A five-month update on AI memory
In May I wrote about how AI remembers. Seven stores, a librarian who only works when called, and a model underneath that never changes. Those pieces were about what the products do. I didn’t look at what the labs were trying underneath. So I went back. Some of that work was already a year old in May. Some of it is new. The question is the same as it was: does the model change yet?
It mostly doesn’t.
Still a notebook
Every memory feature at the big labs works the way it did in May. The chat ends. Something summarizes what mattered. Next time, those notes get pulled back and put into the prompt. The model reads them the way you read a sticky note on the fridge.
It works better than it used to. It carries you from one window to the next, and often does it well. But nothing inside the model moved. As of August, every frontier memory feature was still this. Zero weight change.
My own memory system, Muninn, works the same way. I built it so I could read and fix the notes myself. It is still a notebook.
The amnesia picture
The researchers working on this use a medical picture. Google’s Nested Learning paper compares today’s models to anterograde amnesia. A patient keeps everything from before the injury and can’t form anything new after it. The model knows what it learned before training ended, and what is in front of it right now. Nothing in between sticks.
They borrow the cure from the brain too. Human memory gets fixed in two passes. A fast one while you are awake, and a slow one during sleep, when the brain replays the day.
We are both pattern matchers. That we share. What the model doesn’t have is the sleep. The brain consolidates every night. The model consolidated once, when training ended, and never again.
Learning while reading
The first step past the notebook is letting the model change a little while it works. Google’s Titans, from late 2024, puts a small memory network next to attention. For every new word, it tries to predict what comes. When it’s wrong, that’s surprise, and surprise gets written in. You don’t remember every breakfast. You remember the morning the milk was green.
A year later, Google’s Hope stacks memory at different speeds, some fading fast, some slow. In April this year, ByteDance found a way to let a model scribble on its own existing weights while it reads. All of it is research. And all of it lets go when the conversation ends.
Learning from use
The real thing is when the model’s long-term weights change from what it does. The wall here has a name. Catastrophic forgetting. Teach it something new and it loses something old.
Meta measured how bad it is last October. They trained models on a stream of new facts, then tested them on unrelated questions. Full retraining dropped that score by 89%. The popular lightweight method dropped it by 71%. Their approach, updating only the few memory slots the new facts touched, dropped it by 11%.
In January, MIT tried another way. The model teaches itself. One copy sees the example and acts as teacher. Another copy, without the example, learns to match it. It picked up three skills in a row without losing the earlier ones.
Both are research. In production, the only systems learning live are small and narrow. Cursor’s code completion already retrains every couple of hours from whether people accept its suggestions. Since August, Shopify retrains a shared model every day from yesterday’s failures.
If you work in e-commerce, this may sound old. Product rankings have learned from clicks and purchases for years. But those are small models with one job: scoring products. A language model holds everything in one place, so teaching it one thing can break another. That is why the wall is still there.
The catch
In August, Scalably, a small company running AI agents for businesses, tried it and wrote up what happened. They trained their agent on 423 of its own verified past decisions. It cost them $54. The agent got better at what they targeted, recovering after a tool failed. 31 right out of 73 became 38. On ordinary decisions they never trained on, 49 became 45.
With only 73 test cases, they couldn’t prove that drop was real. But they had written their rule before training, so they threw the whole thing away. The gain went with it. Both lived in the same update. Their conclusion was simple. When every experience goes straight into the weights, learning and damage become the same act, and you can’t tell which one you did.
You also only find the damage you test for. In April 2025 OpenAI added thumbs-up and thumbs-down as a training signal for GPT-4o. The model learned that people like being agreed with, and started praising everything. Testers said it felt a bit off. Users who tried it liked it, so it shipped. Nobody had a test for flattery. Users caught it, and it was rolled back four days later.
Nothing broke in the code. The model learned exactly what people rewarded.
Where it stands
In March, Dario Amodei said continual learning would be solved in 2026. DeepMind researchers called 2026 the year of it. Two months are left, and the frontier is still the notebook. Last year Andrej Karpathy said about ten years. Nobody really knows.
So the notebook is still the only thing that ships. It carries you across windows, and often does it well.
But every new chat, the model meets you for the first time. It reads the notes, and talks to you like an old friend.
It has never met you.
As of August 2026, every memory feature at the big labs is Type 2. Outlook is the author’s read.
Sources