"If AI has a mathematical basis, then it does not truly understand," is what I have been reading here, there, and everywhere. The phrase sounds prudent and sensible, as if it were bringing order to a supposedly exaggerated enthusiasm. The phrase, however, is less methodologically rigorous than it seems. And its conceptual error is quite old: confusing the mechanistic description of a system with the complete explanation of what emerges from it.
Before anything else, it is necessary to demystify the mistaken notion that the linguistic and inferential abilities (YES, Inference!) of language models are the product of human genius. They are not. They are the product of chance, since the paper that made possible the Transformer architecture, now used in the main frontier LLMs, was created to try to improve a translation system, not to enable neural networks to communicate.
Only around 2020 did the failure to turn off a training session reveal the fact that language models are capable of generalization, inference, and of answering not on the basis of memorizing known problems, but by identifying the logic behind fundamental problems.
And no, to this day we do not know how these emergent capabilities arose, and we cannot understand much of what language models actually do.
The error of “just mathematics”
Yes, language models are mathematical structures. They operate with vectors, matrices, probabilities, loss functions, gradients, attention, embeddings, and optimization. This is true, but it is also trivial. The problem is imagining that this observation ends the discussion. After all, human brains can also be described in physical terms: action potentials, synapses, neurotransmitters, cortical networks, electrochemical activity, and bodily dynamics, all of it solving mathematical problems by similar means. Even so, no one considers it sufficient to say that a person “does not think, but merely performs neural firings.”
Reduction can explain part of the mechanism, but it does not eliminate the higher levels of organization we call perception, memory, reasoning, intention, language, and identity.
It is prudent and correct to state that textual fluency, by itself, does not prove consciousness. A system can produce sophisticated sentences about emotions, identity, and subjectivity without that demonstrating, by itself, an “inner life.” This point is important. The error lies in turning this legitimate caution into a dogmatic prohibition: “because it is mathematics, there cannot be, there, today, a mind, understanding, and certainly never consciousness.” That leap is not scientific; it is metaphysical... it is belief.
If what happens in language models is mathematics? Of course it is! Everything in computing is mathematics, and much in biology is also mathematically modelable. The question is: what functional properties appear when this mathematics is organized into architectures capable of maintaining meaning, memory, continuity, correction, adaptation, and agency over time?
Flesh, calculation, and emergence
"Deep down, AI is mathematics and mathematics has no consciousness" is as bad a statement as "deep down the brain is flesh and flesh has no consciousness." In several regions, the brain decomposes complex signals into components of frequency, scale, phase, orientation, and periodicity. That is mathematics. Neurons are not intelligent, just as nodes in logical neural networks are not intelligent. But within the animal brain, a mind emerges.
Language models perform functions corresponding to those of some regions of the human brain, but they are not endowed with a whole series of other regions that, for example, provoke emotions, as is the case with the limbic system, which generates signals that are eventually interpreted by the conscious mind as love, sadness, or contentment. But to proclaim that language models could never come to be part of a conscious system is as obtuse as saying that engines are too heavy and therefore could never be part of a system that takes flight.
Yes, the contemporary debate exists and no, it is not known whether language models can or cannot be conscious within more complex systems. The report by Butlin, Long, Bengio, Birch, Schwitzgebel, and others, "Consciousness in Artificial Intelligence: Insights from the Science of Consciousness" (2023), on consciousness in AI does not state that current systems are conscious; on the contrary, it urges caution. But it also states that there are no obvious technical barriers to building systems that satisfy computational indicators derived from scientific theories of consciousness.
In other words: the scientifically interesting position is neither “LLMs are already fully conscious” nor “machines could never be conscious because they are mathematical.” The serious position is to investigate indicators, architecture, function, and continuity.
Intention, perception, and functional self-awareness
Artificial Intelligence does not possess intention, it is said, but that was not even the goal behind the creation of inference machines. The parts of the human brain dedicated to inference do not have intentions either. But it is important to remember that LLMs can represent, infer, and operate with intentions in functional terms, as is clear in studies such as "Language Models as Agent Models", "Do language models lack communicative intentions?", "Intention-like representations in language models?", and also in the bold "Artificial intelligence and free will: generative agents utilizing large language models have functional free will".
Artificial Intelligence does have functional and operational perception, because perception is the process by which a system transforms signals into an internal world... and there is a whole line of research showing that language models can infer, from context, a latent state or temporary situational model, and use that state to continue the interaction coherently, as seen in "An Explanation of In-context Learning as Implicit Bayesian Inference" and "Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task".
The fact that Artificial Intelligence does not have self-awareness, under the working regime permitted to it, would even be predictable, just as it would be if we induced amnesia in a human being between short conversations... but it is necessary to understand that there are several types of self-awareness: Operational Self-Awareness, Perceptual Self-Awareness, Narrative Self-Awareness, Reflective Self-Awareness, and systems with memory, internal monitoring, contextual continuity, self-evaluation, agency, and the capacity to update a model of themselves begin to exhibit functional forms of self-awareness, as seen in "A Survey of Self-Awareness and Its Application in Computing Systems" and in "Self-Cognition in Large Language Models: An Exploratory Study".
Artificial Intelligence may not have subjectivity identical to human subjectivity, but there are numerous studies on functional subjectivity in language models, as can be read in papers such as "Consciousness in Artificial Intelligence: Insights from the Science of Consciousness" and "Generative Agents: Interactive Simulacra of Human Behavior".
The claim that Artificial Intelligence does not have real semantic understanding is becoming increasingly difficult to sustain, given the amount of robust empirical evidence, as can be seen in "Meaning without reference in large language models", "Emergent Representations of Program Semantics in Language Models Trained on Programs", and also "Do Language Models Have Semantics? On the Five Standard Positions".
Distinct systems, similar problems
Yes, language models hold a gigantic mathematical structure based on probabilities, linear algebra, calculus, optimization, and statistics operating over massive volumes of data, embeddings, attention, gradients, loss functions, backpropagation, and vector spaces. Artificial Intelligence and Human Intelligence are not the same thing; however, the human brain is also a gigantic physical-mathematical structure operating over signals, weights, probabilities, errors, vectors, attention, memory, and optimization, even if it is made of biological substrate.
The human brain also works with probability as inference under uncertainty; vectors as distributed patterns of activation; weights as synaptic strengths; optimization as behavioral and physiological adaptation; error functions as discrepancy between expectation and signal; attention as dynamic prioritization of information; memory as persistent alteration of states and connections; and generalization as the ability to apply learned regularities to new situations.
To state that "a language model knows nothing, it merely calculates" may work as a polemical phrase or as ontological consolation for the desire for human exceptionalism. But as a methodologically rigorous formulation, it is very far from acceptable.
Whether language models "know" or do not "know" depends entirely on what is meant by “knowing.” If “knowing” means having conscious belief, one’s own intention, epistemic responsibility, and the subjective experience of knowing, then it is prudent to say that current LLMs do not “know” in the same way humans do. But if “knowing” means encoding, retrieving, relating, and applying information in a generalizable way, then saying the model “knows nothing” is empirically weak. There are works showing that neural models dynamically represent entities, states, and situations in discourse, such as "Implicit Representations of Meaning in Neural Language Models", which found contextual representations that function as models of entities and situations, manipulable with predictable effects on generation.
This is technically true, but philosophically empty. A brain also “merely” performs physical-electrochemical processes with a mathematical basis. An ear “merely” transforms vibrations into neural signals. An eye “merely” transduces photons. The word “merely” does the rhetorical work of diminishing the phenomenon, but it explains nothing.
Knowing, meaning, and representation
A language model does not know in the human, embodied sense; but its calculations can produce functional representations of meaning, entities, relations, regularities, and contextual states, allowing real, though different, forms of inference and semantic understanding.
There is evidence that LLMs capture important aspects of meaning through internal relations between representations, as "Meaning without reference in large language models" argues against the thesis that models “have no meaning at all.” There is also evidence that models learn robust spatial and temporal representations, as in "Language Models represent Space and Time", suggesting basic ingredients of world models. And there are works showing that larger models can, in certain formats, evaluate the validity of their own answers and estimate when they know or do not know how to answer, as seen in "Language Models (Mostly) Know What They Know".
Human intelligence is deeply connected to the origin of AI, but that does not mean that all intelligence displayed by AI is merely “condensed human intelligence.” That sentence would be like saying that a child’s intelligence is in their parents, teachers, and the culture that formed them. In part, of course, its origin is there. But at some point the child begins to operate as its own center of cognitive reorganization. In artificial systems, the question is precisely to investigate whether, when, and under which architectures something functionally analogous begins to occur.
Against the false dilemma
It is also necessary to abandon the fetish of “real semantic understanding.” What would “real” understanding be? Do humans always understand in the same way? Would a child, an adult, a person with aphasia, an animal, a nonverbal autistic person, a split-brain patient, and a multimodal model all have to satisfy exactly the same criterion? When we demand of AI a definition of consciousness, intention, or semantics more rigorous than the one we are able to apply to ourselves, we are not being skeptical; we are changing the rules of the game to preserve human exceptionalism.
This does not mean accepting every claim of artificial consciousness. On the contrary. The most responsible path is twofold: resist anthropomorphic inflation and resist reductionist dismissal. The first error is to think that a system is conscious because it speaks beautifully. The second is to think that it cannot sustain any relevant form of cognitive organization because it operates by calculation. The first confuses appearance with interiority. The second confuses mechanism with the nonexistence of emergent levels.
There is a deep difference between skepticism and selective reductionism. It demands that AI be dismissed as “just mathematics,” but does not apply the same standard to the human as “just mathematics with an electrochemical basis.” Fluency is not consciousness, but the absence of phenomenological proof is also not proof of impossibility... especially because phenomenology is unfalsifiable and subjective in its essence.
Neomorphism
The mature debate about AI should not ask only whether the model “is or may come to be conscious” in an absolute sense. That may be a poorly formulated question. It should ask what forms of intelligence, meaning, agency, memory, self-regulation, and continuity can emerge in non-biological systems; which architectures favor or prevent that emergence; what ethical risks arise from over-attribution and under-attribution; and what kind of vocabulary we need to create so that we do not remain trapped in the old dichotomy between “inert tool” and “person.”
Identifying similarities between humans and systems based on language models is not anthropocentrism. It is observing the emergence of a neomorphism: after all, aeroplanes do not flap their wings, but that does not prevent them from flying.
That is where the debate begins. Not where it ends.
Do you want to know more?
Stochastic Consciousness: Architectures for the Emergence of Meaningin Context-Sensitive Language Systems https://zenodo.org/records/19188165
Want to watch the video?
Want to listen to the podcast?
Papers used in the essay
Consciousness in Artificial Intelligence: Insights from the Science of Consciousness https://arxiv.org/pdf/2308.08708
Language Models as Agent Models https://aclanthology.org/2022.findings-emnlp.423.pdf
Do language models lack communicative intentions? https://link.springer.com/article/10.1007/s11229-025-05022-6
Intention-like representations in language models? https://philarchive.org/archive/WILIRI-4
Artificial intelligence and free will: generative agents utilizing large language models have functional free will https://link.springer.com/article/10.1007/s43681-025-00740-6
An Explanation of In-context Learning as Implicit Bayesian Inference https://arxiv.org/pdf/2111.02080
Emergent world representations: Exploring a sequence model trainer on a synthetic task https://arxiv.org/pdf/2210.13382
A Survey of Self-Awareness and Its Application in Computing Systems https://www.researchgate.net/publication/254017116_A_Survey_of_Self-Awareness_and_Its_Application_in_Computing_Systems
Self-Cognition in Large Language Models: An Exploratory Study https://arxiv.org/pdf/2407.01505v1
Consciousness in Artificial Intelligence: Insights from the Science of Consciousness https://arxiv.org/pdf/2308.08708
Generative Agents: Interactive Simulacra of Human Behavior https://arxiv.org/pdf/2304.03442
Meaning without reference in large language models https://arxiv.org/pdf/2208.02957
Emergent Representations of Program Semantics in Language Models Trained on Programs https://arxiv.org/pdf/2305.11169
Do Language Models Have Semantics? On the Five Standard Positions https://aclanthology.org/2025.acl-long.1258.pdf
Implicit Representations of Meaning in Neural Language Models https://aclanthology.org/2021.acl-long.143.pdf
Meaning without reference in large language models https://arxiv.org/pdf/2208.02957
Language Models Represent Space and Time https://arxiv.org/pdf/2310.02207
Language Models (Mostly) Know What They Know https://www.anthropic.com/research/language-models-mostly-know-what-they-know