Anthropic researchers find Claude has a hidden ‘thinking’ workspace: Here’s what it means

Anthropic has claimed that modern large language models (LLMs) such as Claude have a hidden internal workspace which allows them to ‘think’ about concepts without necessarily revealing those ‘thoughts’ in their AI-generated responses.

This small collection of internal neural patterns in Claude has been named J-space. It represents evidence of internal thoughts that do not appear in the model’s outputs, according to a summary of a much longer research paper titled ‘Verbalizable representations form a global workspace in language models’ published by Anthropic researchers on Monday, July 6.

Anthropic also said that it has released a code repository with an open-source implementation of the core methods. It has further developed an interactive demo of the methods on open-weights models in partnership with Neuronpedia.

The researchers emphasised that the existence of the J-space in Claude emerged on its own and was not something designed or programmed by them. “More broadly, these findings have changed our understanding of how Claude’s mind works, revealing a privileged mental workspace that can be used for deliberate reasoning, operating amidst a sea of more automatic, inflexible processing,” Anthropic said.

Anthropic’s latest research is significant because it could advance efforts to make LLMs more interpretable and influence their behaviour. However, the startup’s claims of advanced AI capabilities have previously attracted skepticism, with its researchers consistently raising the possibility that Claude shows signs of human-level consciousness.

What is J-space?

Named after the Jacobian mathematical concept, J-space is essentially a collection of internal neural patterns in LLMs where each pattern is linked to a particular word. When one of these patterns lights up, it means that the model has that particular word “on its mind”.

J-space is not the same as ‘chain of thought’ reasoning in models as “it operates silently, in the model’s internal neural activations, allowing the model to think about a concept without writing it down.” Anthropic claimed that Claude will tell you what it is thinking about in the J-space when asked. These neural activations can also be modulated by Claude when requested.

Story continues below this ad

“Representations in the J-space can be used flexibly for many tasks—for example, once “France” has lit up in Claude’s J-space, the model can recall its capital, or its national currency, or the continent it belongs to,” Anthropic said.

Using its J-space makes Claude smarter. “In experiments where we prevented Claude from using its J-space, it still interacted normally, but lost its higher-order cognitive functions,” the company said. However, the LLM may not rely on neural activations in the J-space for most reasoning tasks and capabilities such as speaking fluently, recalling simple facts, using correct grammar, etc.

How was the experiment conducted?

Anthropic researchers said that the experiments leading to these results were inspired by a prominent theory in neuroscience called the global workspace theory, which proposes to picture the brain as a collection of specialist systems that unconsciously work in parallel and largely in isolation from one another.

“If a thought is consciously accessible to you, you can typically describe it if someone asks. We went looking for representations in Claude with the same property…” Anthropic said.

Story continues below this ad

The researchers further said they utilised a technique called the Jacobian lens, or J-lens, to find the internal activity pattern that makes Claude more likely to say a word in its vocabulary at some point in the future. “Claude processes text through a series of multiple internal stages called layers, and by applying this technique over different layers, we can watch these silent words in the J-space evolve as the model works through what to say,” they said.

Does this mean Claude is conscious?

Anthropic has denied that its latest findings are proof that Claude is conscious like humans and that it experiences feelings. However, it has also added a caveat.

While Claude may not have capacity to have experiences, the model’s neural activity is consciously accessible. “A thought is “access-conscious” (or “consciously accessible”) if you can report it, reason with it, and use it to guide what you do. It remains a contested philosophical question whether or not access consciousness implies phenomenal consciousness, or if the ability to have experiences requires some other property,” Anthropic said.

For now, Anthropic said that it believes the J-space could be a useful tool to offer a glimpse of what Claude is thinking about but not saying. The company plans to use this research to influence and improve Claude’s decision-making skills.