Knowledge and Memory
21 comments
·September 7, 2025null
Muromec
Yesterday I asked one LLM about certain provisions of Ukrainian law, where severity threshold for a financial crime was specified indirectly through a certain well known constant. The machine got it wrong and when asked to give sources cited the respective law referencing a similary sounding unit. Amazingly it gave the correct English tranlation but gave me the wrong original in Ukrainian.
I guess it merged two tokens why learning the text.
Amazingly it also knows about difference between two constants, but referrs to the wrong one in both calculations and in hallucinating the quote.
It's tedious to always check for stuff like this.
Then I asked a different LLM and it turned out that actually the constant is monkey patched for specific contexts and both me and the first lying machine were wrong
ClaraForm
I'm not convinced the brain stores memories, or that memory storage is required for human intelligence. And we "hallucinate" all the time. See: eye witness testimony being wrong regularly, "paranormal" experiences etc.
It's a statement that /feels/ true, because we can all look "inside" our heads and "see" memories and facts. But we may as well be re-constructing facts on the fly, just as re-construct reality itself to sense it.
n4r9
What do you mean, you're not convinced that the brain stores memories? What is happening in the brain when you have an experience and later recall that same experience? It might not be perfect recall but something is being stored.
ClaraForm
I mean an LLM (bad example, but good enough for what I'm trying to convey) doesn't need any sort of "memory" to be able to reconstruct something that looks like intelligence. It stores weights, and can re-assemble "facts" from those weights, independent of the meaning or significance of those facts. It's possible the brain is similar, on a much more refined scale. My brain certainly doesn't store 35,000 instances of my mum's image to help me identify her, just an averaged image to help me know when I'm looking at my mum.
The brain definitely stores things, and retrieval and processing are key to the behaviour that comes out the other end, but whether it's "memory" like what this article tries to define, I'm not sure. The article makes it a point to talk about instances where /lack/ of a memory is a sign of the brain doing something different from an LLM, but the brain is pretty happy to "make up" a "memory", from all of my reading and understanding.
HarHarVeryFunny
The article isn't about LLMs storing things - it's about why they hallucinate, which is in large part due to the fact that they just deal in word statistics not facts, but also (the point of the article) that they have no episodic memories, or any personal experience of any sort for that matter.
Humans can generally differentiate between when they know something or not, and I'd agree with the article that this is because we tend to remember how we know things, and also have different levels of confidence according to source. Personal experience trumps watching someone else, which trumps hearing or being taught it from a reliable source, which trumps having read something on Twitter or some grafitti on a bathroom stall. To the LLM all text is just statistics, and it has no personal experience to lean to to self-check and say "hmm, I can't recall ever learning that - I'm drawing blanks".
Frankly it's silly to compare LLMs (Transformers) and brains. An LLM was only every meant to be a linguistics model, not a brain or cognitive architecture. I think people get confused because if spits out human text and so people anthropomorphize it and start thinking it's got some human-like capabilities under the hood when it is in fact - surprise surprise - just a pass-thru stack of Transformer layers. A language model.
RLAIF
[dead]
roxolotl
I don’t know if this aligns with your thinking but there is a theory that memory is largely reconstructed every time it is remembered: https://en.wikipedia.org/wiki/Reconstructive_memory
ClaraForm
Thank you for that! I think I read about this a long time ago, internalized it, and forgot it. Pretty on the nose in this conversation... haha.
soulofmischief
A payload plus a transformation process is still storage, just compressed storage. There's conceptually little difference between storing data and the process that gets called to create that data.
graemep
its not reliable, but we can also recall things accurately.
it does not store things in the way records of any sort do, but it does have a some store and recall mechanism that works.
To be fair, LLMs do this too - I just got ChatGPT to recite Ode to Autumn.
ClaraForm
Yes, I agree. I'm not against the idea that the brain can "store" things. Just whether our concept of how a "memory" "feels" is useful to us further understanding the brain's function.
MadcapJake
Reconstructing from what?
tolerance
You know, whatever memory is or where it’s at and however the mind works, I’m grateful I’ve got mine in tact right now and I appreciate science’s inability to zero in on these things.
It’s nice to know that this sort of appreciation is becoming more common. Somewhere between tech accelerationism and protestant resistance are those willing to re-interrogate human nature in anticipation of what lies ahead.
A different blog post from this month detailing an experience with ChatGPT that netted a similar reflection: https://zettelkasten.de/posts/the-scam-called-you-dont-have-...
a3w
> I’ll remind you that biologists do not, in the year 2025, know memory’s physical substrate in the brain! Plenty of hypotheses — no agreement. Is there any more central mystery in human biology, maybe even human existence?
Did they not recently transfer memory of how to solve a maze from one mouse to another, giving credibility to what can store information?
Searching, I only find the RNA transfers done in 60s, which ran into some problems. I thought a recent study did transfer proteins.
sd9
I would love to see some sources for this. Sounds super interesting. When do we get Matrix-style learning modules?
GaggiX
A model is capable of learning the "calibration" during the reinforcement learning phase, in this "old" post from OpenAI: https://openai.com/index/introducing-simpleqa/ you can see the positive correlation between stated confidence and accuracy.
OpenAI just recently took a systematic look into why models hallucinate [0][1].
The article you shared raises an interesting point by comparing human memory with LLMs, but I think the analogy can only go so far. They're too distinct to explain hallucinations simply as a lack of meta-cognition or meta-memory. These systems are more like alien minds, and allegories risk introducing imprecision when we're trying to debug and understand their behavior.
OpenAI's paper instead identifies hallucinations as a bug in training objectives and benchmarks, and is grounding the explanation in first principles and the mechanics of ML.
Metaphors are useful for creativity, but less so when it comes to debugging and understanding, especially now that the systematic view is this advanced.
[0] https://openai.com/index/why-language-models-hallucinate/?ut... [1] https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4a...