Anthropic wants caution but the market wants more AI

The latest news from the Western AI Front is that Anthropic Inc, the US-based creator of Claude, has called for a "global freeze" on the development of AI, so humanity can catch up.

Except it didn't really.

In a long blog post on Anthropic's website headed "When AI builds itself", all about how Claude is not only writing its own code but "proposing its own experiments", it was explained that "the human role is narrowing at each step in the AI development process".

Towards the end of the post, under the sub-heading "What should we do?", the blog suggests that "if it were possible to slow the development of this technology … we think that would likely be a good thing", as long as there is a mechanism for global coordination.

So it might be a good thing, if it were possible. If there were a global coordination mechanism, which there isn't and very unlikely to be, that includes China and Russia, not to mention Iran and North Korea.

Building AI defences

Meanwhile, Anthropic has just finished building the most dangerous AI model yet, called Mythos, which it said is "strikingly capable at computer security tasks" — hacking — so it couldn't be released. Instead, Mythos was confined to a select group of insiders so they could prepare defences for when someone not so friendly invented something similar.

That club of a chosen few is called Project Glasswing, after the butterfly with transparent wings. Last week, a group of undisclosed lucky Australian firms was invited to join, along with 150 others.

How they were selected from the 60 million or so companies in the US, Europe, Japan and Australia is unknown.

Anthropic chief executive Dario Amodei has described it as being aimed at making sure democracies stay ahead of autocracies (that means China, although not explicitly), so presumably all of those in the Glasswing club are from democracies.

It's a little bit reminiscent of the conference of 44 allied countries at Bretton Woods in New Hampshire in July 1944, held to set up the new democratic post-war economic system once the Nazi autocracy had been defeated, although that club did include two autocracies — China and the Soviet Union.

The main difference is that Bretton Woods was all government, while Project Glasswing is all private sector, created and run by Anthropic, which is also now the leading AI business, having recently knocked off ChatGPT developer OpenAI.

Actually, another 1944 event was more relevant to what's happening in 2026: the invention of Colossus, the world's first computer, by Tommy Flowers and Alan Turing as part of the Bletchley Park code-breaking effort, just in time for the D-Day Normandy landings on June 6, 1944.

There is a direct 82-year line from Colossus to Mythos, and from one kind of D-Day to another, as we prepare for machines that build themselves, a moment known as the singularity.

After that British government effort in 1944, the post-war development of digital computing was funded by the US defence department.

But as the 60s turned into the 70s and 80s, technology became a private sector race, mainly between American companies IBM, Apple and Microsoft, and then Google, Facebook and Amazon.

Now, AI is the culmination of Colossus, with Anthropic, OpenAI and SpaceX the new colossi about to hit investment markets with trillion-dollar-plus public floats.

Meanwhile, China has caught up with US technology and, in some ways, passed it, and Dario Amodei is today's protector of democracies instead of Winston Churchill.

The AI bubble has taken off

Project Glasswing and the withholding of Claude Mythos from public release was announced on April 7, which, as it happens, was eight days after the stock market AI bubble took off again.

On March 30, global share prices, led by the listed AI companies, ended a six-month decline and began to surge again. Since then, the world index has gone up 20 per cent, and the eight biggest AI stocks have gone up an extraordinary 32 per cent. The bubble is having its third wind.

There doesn't seem to have been a specific event that sparked this turnaround, just a sudden renewed recognition that AI really is going to change the world and that everyone — individuals, companies and governments — will use it. They already are.

What's more, it's becoming clear that AI is going to cost more than anyone expected, at least for a while, and that the infrastructure build will have to be colossal to support the "compute" required to generate the tokens needed to meet the exploding demand for intelligence of the artificial variety including, according to the SpaceX prospectus, data centres in space.

For that reason, semiconductor shares have gone up the most this year — the big three, Samsung and SK Hynix of Korea and Micron of the US, are up sixfold in six months.

Will the bubble burst this year?

Will the bubble burst?

Well, it's both different this time and the same.

The concentration of the bubbling sector of the market as a whole is the same — 40 per cent — as that which has always in the past preceded a crash.

But there's never been a boom of intelligence before and unlike in the late 1990s, the soaring prices are supported by real earnings and real capital spending. If anything, it's a bubble in profits and capex, and the question for investors is: can they be sustained?

More specifically, it's a question of whether AI will become a low-margin utility business like the electricity that supplies it and other businesses that have been the subject of share market bubbles in the past, like railroads and data transport (the internet).

If so, it is just a matter of how long margin-boosting bottlenecks and cartel-like industry structures last.

Providing the data centre "compute" for AI — that is, generating the tokens that embody it — is already becoming a utility business, but the business of making and selling the tokens themselves, as Anthropic (Claude), OpenAI (ChatGPT) and Google (Gemini) do, is just getting started, and for the moment they seem to be anything but utilities.

Their product is called "inference", defined as the operational phase of AI after training has happened.

In inference, the trained model takes a real-time input (a user's prompt or a photo) and instantly computes an output in the form of tokens, which the company sells by the millions to software agents operated by companies or on subscription to individuals.

How will AI business develop?

It's still early days for this industry, but it seems likely to develop into three layers:

Infrastructure and base models

GPU vendors, cloud providers, and generic Large Language Model Application Programming Interfaces (LLM APIs). These are likely to be solid utility businesses, with pricing determined by the American oligopoly and open‑source pressure from China.

'Smart middleware'

RAG frameworks, orchestration, domain‑specific models, and hosted open‑source platforms. RAG stands for Retrieval-Augmented Generation and is where an LLM first retrieves relevant information from an external knowledge store, then augments its answer with it. In short, it's a pattern for letting models "look things up" at query time instead of relying only on what they were trained on and it can provide higher margins depending on how different and useful it is.

End‑user applications

Vertical products that embed LLMs (copilots, agents, workflow tools, industry‑specific systems). This is where unit economics can be very attractive because customers pay for outcomes, not tokens, and there can be a large spread between price and the underlying inference cost.

But the base LLMs like Claude and ChatGPT could become low-margin utilities through competitive pressures from open-source providers, especially the ones in China (whenever China moves into an industry, profits move out).

There will be higher margins available in AI-enabled products, such as agents that enhance corporate and personal productivity and, in a few years, AI robots that do the housework and keep us company.

China is moving in

On that subject, two weeks ago China became the first country to regulate the potential emotional attachment to humanoid AI robots through something with the catchy title: "Interim Measures for the Administration of Anthropomorphic AI Interaction Services" — that is, humanoid robots with AI inside.

It's the first attempt by any government to deal with the emotional dangers of AI, something I discussed in this column a month ago.

The new Chinese law contains this provision: "Emotional Manipulation and Dependency Red Line: Must not excessively cater to users, induce emotional dependency or addiction, damaging users' real interpersonal relationships; must not induce users to make unreasonable decisions through emotional manipulation."

To that I say: good luck trying to police it!

Back to the bubble, and the problem for investors may be that the big hyper-scalers, and the three firms are doing massive IPOs this year — Anthropic, OpenAI, and SpaceX — are in the first category of future low margin utilities, although for the moment they are charging like unhappy bulls while urgently trying to scramble up the value-add curve.

The number one issue for investors in 2026 is: when will the realisation dawn that AI is just another product sold by just another industry?

Alan Kohler is a finance presenter and columnist on ABC News. He hosts the podcast That’s Business with Alan Kohler in the ABC Business Daily feed on Friday. He also writes for Intelligent Investor.