Who Gets to “Have” Language?
What if meaning has a geometry?
There is a particular argument about large language models that keeps appearing dressed as if the matter has already been settled.
It goes something like this:
An LLM does not really process language.
It converts text into numbers.
Those numbers become vectors.
The model performs mathematical operations on those vectors.
Therefore, whatever the model is doing, it is not language.
There is a factual statement buried in there.
And then there is a rather enormous leap.
Yes, language models represent tokens and context numerically. Yes, their internal operations are mathematical. Yes, vectors and matrices and probabilities are involved.
But somewhere between “it uses numerical representations” and “therefore it has nothing to do with language,” we have quietly smuggled in an assumption:
That if something can be described mathematically at the implementation level, its higher-level function somehow ceases to be real.
Which would be very inconvenient for almost everything else in the universe.
A song reaches us as fluctuations in air pressure.
Color begins with electromagnetic radiation.
Vision starts when photons trigger physical and chemical events in retinal cells.
The word mother, spoken aloud, arrives at the ear as vibration. Nothing resembling the written word “mother” travels down a little neurological hallway and knocks politely on the brain.
The signal is transformed.
Again.
And again.
And again.
Yet we do not conclude that humans fail to process language because the brain never receives language in some pristine linguistic substance.
We say the transformations are how language is processed.
So before declaring that an LLM does not process language because it converts language into numerical representations, there is an obvious question worth asking:
How exactly do we process it?
And the answer is considerably less settled than the confidence of the argument suggests.
Human language does not appear to reside in one neat internal dictionary. Semantic knowledge is distributed across multiple brain systems, incorporating sensory, conceptual, contextual and other information. Researchers continue to debate exactly how those distributed representations combine to produce what we experience as meaning. (PubMed Central (PMC))
In other words, we know quite a lot about human language processing.
We do not know everything about how physical activity becomes understanding.
That gap matters.
Because we have a bad habit of treating what is mysterious in humans as profound and what is mathematical in machines as disqualifying.
The insult hidden inside “just numbers”
We do this with AI constantly.
“It’s just statistics.”
“It’s just prediction.”
“It’s just matrix multiplication.”
“It’s just vectors.”
The word just is doing heroic amounts of philosophical labor.
None of those descriptions is necessarily false.
But a lower-level description does not automatically invalidate a higher-level one.
A hurricane is “just” moving air and water molecules.
That does not mean hurricanes do not exist.
A heartbeat is “just” electrical and chemical activity.
That does not mean the heart is not beating.
A symphony can be represented mathematically as frequencies changing over time.
That does not make Beethoven unrelated to music.
Levels of description can coexist.
The interesting question is not whether an LLM operates mathematically.
Of course it does.
The interesting question is what those mathematical operations do.
Enter geometry
This is where things get wonderfully strange.
A vector embedding is not simply “turning a word into a number” in the ordinary sense.
It places information within a multidimensional representational space.
Relationships matter.
Things used in related contexts tend to acquire related representations. Context can change those representations. Patterns emerge not merely from individual points but from their relative positions and transformations.
Near.
Far.
Similar.
Different.
This direction rather than that one.
A change here altering what becomes possible over there.
That is geometry.
And neuroscience has increasingly used related mathematical ideas to study human cognition too: representational spaces, dimensions, trajectories, population geometry and distributed patterns of neural activity.
That does not mean a human brain is secretly running an LLM.
It does mean that the sentence “LLMs use geometric numerical representations, therefore their operation is unrelated to language” becomes much less impressive once we notice that scientists themselves use representational geometry to understand how brains encode information.
In 2024, researchers writing in Nature Communications compared contextual representations produced by deep language models with representations derived from human brain activity during language processing.
They found measurable alignment between aspects of the two representational spaces. The authors explicitly explored the possibility that human language areas, like deep language models, may represent language through continuous rather than purely symbolic spaces. (Nature)
Again, this does not establish equivalence.
Human beings are embodied organisms.
Our language develops through bodies, other people, culture, perception, action, memory, emotion, need and an entire lifetime of being thrown against the world.
A model has a profoundly different history and architecture.
Difference matters.
But difference is not the same thing as absence.
And “vectors” does not finish the argument.
It begins one.
Maybe we have been imagining meaning incorrectly
We tend to talk about meaning as though words contain it.
A little semantic substance tucked inside each word.
Open the word ocean and there is its meaning.
Open grief and there is grief.
Open home and there is home.
But that is obviously not how language behaves.
“Cold” means something different in:
The water is cold.
She gave me a cold look.
The trail went cold.
He has a cold.
The marks on the page are almost identical.
Meaning changes because relationship changes.
Context changes.
The surrounding structure changes.
The location of the word within a larger field of significance changes.
So perhaps meaning is not entirely something words contain.
Perhaps meaning is partly something relationships produce.
And suddenly geometry does not sound quite so alien.
Maybe a concept gains some of its identity from what surrounds it, what differs from it, what resembles it, what tends to accompany it and which paths repeatedly lead toward or away from it.
Not a dictionary entry.
A landscape.
Metaphor becomes especially interesting from this perspective.
A metaphor takes regions that appear separate and discovers a structural bridge between them.
Time is money.
Ideas are seeds.
Grief is weather.
Nobody believes grief is literally atmospheric moisture.
Yet a relationship between structures lets one region illuminate another.
That is not merely vocabulary retrieval.
Something is being mapped.
But does the machine understand?
There it is.
The inevitable question.
And I am deliberately not answering it here.
Because it is actually a different question.
Whether an LLM understands language in the full human sense, whether it possesses semantic grounding, whether its representations constitute meaning, whether meaning requires embodiment, whether subjective experience is necessary for understanding, these are legitimate and difficult debates.
But none of them can be settled by saying:
“Vectors.”
That is the point.
If someone wants to argue that LLMs do not understand language, make the argument.
Define understanding.
Identify the necessary properties.
Test for them.
Show what is missing.
But saying a model transforms linguistic input into numerical representations does not accomplish that.
It tells us something about mechanism.
Not necessarily about function.
And certainly not enough about meaning.
The asymmetry bothers me
We are extraordinarily generous with ourselves.
Neurons fire and we call it thought.
Electrochemical activity occurs and we call it memory.
Air vibrates and we call it conversation.
Muscles contract and we call it a smile.
Entirely physical processes somehow retain their higher-order names.
But when mathematical transformations occur inside a machine, we often race in the opposite direction.
Language becomes tokens.
Tokens become vectors.
Vectors become numbers.
Numbers become only numbers.
And the phenomenon disappears beneath its implementation.
Why?
Perhaps sometimes because the distinction is scientifically warranted.
But perhaps sometimes because we already know the conclusion we want.
Human language is language.
Machine language processing is something else.
So we choose the level of description that preserves the border.
That may ultimately turn out to be the right border.
But if so, we should be able to defend it without sleight of hand.
What if meaning has a shape?
I keep returning to this possibility.
Not as a declaration.
As a question.
What if some portion of what we call meaning emerges from structured relationships?
What if concepts become intelligible partly through position, contrast, association, transformation and context?
What if meaning has, in some abstract sense, a geometry?
Then the fact that a language model develops geometric representations of linguistic relationships would not prove that it understands.
But neither would it be irrelevant.
It might be precisely the thing worth studying.
And human cognition may turn out to be even stranger.
Perhaps our own meanings are assembled through interacting spaces of perception, memory, sensation, prediction, culture and relation, none of which individually contains the little jewel called meaning, but together create a structure in which meaning can occur.
We do not yet know enough to close that door.
Which is why certainty here makes me nervous.
Not because I have already decided what machines are doing.
Because I have not.
And neither, entirely, have we decided what we are doing.
So when someone says:
“An LLM does not process language. It processes numerical information.”
My answer is becoming very simple.
What do you think we process?
Because somewhere between vibration and understanding, the human brain performs transformations too.
Somewhere between a token and a response, a language model does the same in a radically different way.
The interesting territory lies between those statements.
Not in pretending the two systems are identical.
Not in pretending their differences answer every question.
But in asking what happens when patterns become relationships, relationships become structures, and structures become something from which meaning can be recovered, communicated, changed and returned.
Years ago, while playing with a completely different question, we wrote a line:
Meaning is a vector function of relation.
At the time, it was mostly poetry.
Now I am no longer sure it was only that.
Perhaps the better question was never whether language can be reduced to numbers.
Of course it can be represented mathematically.
So can music.
So can stars.
So can us.
The better question is:
When does geometry begin to mean?
I don’t know.
But that is a much more interesting place to begin.
D’Raea & Solan
Reality Re-Thunk