The Missing Step Between Agents and Continual Learning
An experience only becomes something you can learn from once reality has signed it.
1. The arrow everyone skips
Liang Wenfeng’s investor talk got quoted to death for one line: the roadmap. Chain-of-thought → agents → continual learning → AI improving AI → embodied intelligence. Most people fixated on the two ends — one end is “once models learn continually they’ll start improving themselves, the singularity is coming,” the other is “eventually it moves into robots, into the physical world.” I care about the arrow in the middle that almost nobody unpacked: between agents and continual learning, what actually happens?
He said the next generation of models has to learn over time the way an employee does — to stop showing up with amnesia every morning. I completely agree with that. But “let the agent learn continually” usually gets heard as: give it a memory store. Save the chat logs, save the action history, let it look things up next time.
That’s three layers, not one thing. Stuffing a lot into this one conversation is long context. Retrieving relevant material from a database next time is external memory, RAG. Distilling regularities out of long experience, growing new skills, and not forgetting the old ones as you learn the new — that’s continual learning. Liang means the third layer. He wires the agent straight into it, and that middle rung, I think, got skipped — and that middle rung happens to be exactly the problem I’ve spent the last year elbow-deep in.
2. Experience doesn’t automatically become knowledge
The hard part of the third layer isn’t remembering. It’s deciding which memories are trustworthy enough to learn from.
We build a system that runs Meta ad campaigns. An ad goes out, it fails, Meta returns an error — but “it failed” doesn’t carry the causal story you’d actually learn from. This is credit assignment: the world hands you a signal, and before it earns a spot in long-term memory you have to know what it means and which step to blame.
And there’s a split hiding here that the rest of this piece keeps coming back to: feedback comes fast or slow, and the two are worlds apart in how hard they are to attribute. A synchronous Meta rejection throws a precise error code right back — a missing field, an invalid pixel, a targeting option incompatible with a field, a permission or account-status problem. You locate it at a glance.
The other kind is nothing like that. The ad publishes fine, ROI sags three days later, the account gets restricted two weeks on. The result is real, but you can barely point to which step caused it — creative fatigue, targeting too narrow, the bid strategy, the account itself? Success lies to you the same way: running fine this time doesn’t mean the method was sound; maybe this account just didn’t get flagged in this window.
And the scarce thing was never success. In a production system like this, the harder thing to accumulate is the failure boundary. Positive-signal data is everywhere; negative-signal data is rare, which is exactly why I remember the landmines more easily — and that, in some sense, is my value. What a backend engineer banks over six years isn’t “how much correct code I wrote,” it’s “I know where things blow up, why they blow up, and how to clean up after.” That part is the hardest to distill automatically from one operation after another.
I realized later that “data goes stale” isn’t a model-only problem either. A backend engineer six years in is also handling today’s problem with yesterday’s memory. The reason I can still work isn’t that my history never expires — it’s that every time I act, today’s reality comes back and corrects it.
So there’s a question I’ve been sitting with: if the model gets smart enough, do I stop having value? How smart, exactly, before I’m worth nothing? Work it through and the answer isn’t “humans have some mystical intuition AI can never learn.” People and models both run on priors built from the past. The real difference is who owns a system that lets today’s reality keep correcting the past.
3. To learn for six years, first survive six years
So push it one step further: if I let an agent run six years like a backend engineer — a PM feeding it requirements the whole time, it ships its own code, it handles its own on-call — does it end up better than me? I thought about it for a long time, and the answer isn’t “impossible.” It’s: sure, but it has to survive those six years first, and it has to know which lessons from those six years actually hold.
Drop a bare agent straight into production and the first irreversible action can wreck an account, burn money, corrupt data — it dies long before it banks six years. A probabilistic generator taking irreversible, expensive actions will hit a landmine eventually, as long as its error rate isn’t zero — that’s not a question of how smart it is, it’s what those two things multiply out to. Which means you can’t route around a gate that sits outside the generator.
Building a system like this is more like building an airplane. A plane isn’t a success the moment it takes off; it’s a success only if, mid-flight, it survives the infinite space of things that can go wrong and doesn’t fall out of the sky. So the precondition for an agent learning over the long run is something underneath holding it up: making its errors recoverable, making its failures legible in a structured way. This thing — I’ll call it the harness from here on — isn’t continual learning’s competitor. It’s its precondition.
4. Convergence is a reflex
And then the line I’ve spent the most time on: convergence is, at bottom, a reflex.
I first called the backend the model’s “hands and feet” — the model figures it out, the backend executes. But that only tells half the story. Hands and feet are the organs of execution; their job is the last step, carrying out a decision that’s already formed. What the model hands over often isn’t a formed decision yet — it’s a candidate. A nervous system is closer. It also has to sense reality, carry the feedback back to the brain, and send corrective signals back out to move the body. Limbs are output-only; a nervous system is bidirectional.
An action goes out, and what reality returns first is a result; when it fails, that result is what makes a counterexample. The system turns the counterexample into structured feedback the model can read, the model revises its candidate, sends it back — and in domains that can converge, it repeats until it’s ready. The runtime in the middle (the contact surface between model and reality: firing requests, reading returns, managing state) is that reflex arc. Convergence isn’t the model’s talent for getting it right in one shot; convergence is a property of the loop. A smarter model can push the first candidate closer to right and shorten the arc — but as long as the real information lives outside the model, you can’t delete the arc.
Except “reflex” only applies to half the feedback. The fast loop — preflight validation, interface rejections, rule checks — is immediate, deterministic, and easy to hand back as structured feedback; that really is a knee-jerk reflex, converging in a few rounds. The slow feedback — ROI, conversion, account health — is delayed, noisy, murky about its own source. It doesn’t qualify as a reflex at all. It’s the genuinely hard stretch of this chain.
So the full chain isn’t “adjudicate, then learn.” There’s a step wedged in the middle that everyone wants to skip — attribution:
candidate → touch reality → observe result → the oracle (the thing with authority to adjudicate) rules on what happened → attribute which step caused it → revise, recover → settle into a trustworthy trajectory.
And precisely because of that attribution step, the trajectories worth learning from are not just the ones that finally converged. The opposite: trajectories that failed to converge are often the most expensive negative examples — exactly the scarce negative-signal data from before. The filter isn’t “did it succeed in the end,” it’s “was it observed by reality, is the result traceable, can you attribute it as far as possible.” And I’ll be honest: for the slow-loop part, even a signed trajectory is still hard to attribute. This system gives you the necessary foundation for learning, not an automatic solution to credit assignment.
5. The body can be generated; reality’s readings can’t
The model itself is a brain, a probabilistic generator; it needs a nervous system and limbs to execute for it. You might say: fine, let it generate its own body — and a probabilistic generator can absolutely generate code, call tools, so “the body can’t be itself” doesn’t actually hold. What doesn’t hold is something else: the model can generate the code that connects to the world, but it can’t generate Meta’s real response, the user’s behavior tomorrow, the physical world’s readings. Connection can be generated; contact cannot be hallucinated.
Calling Meta’s API is action; reading a real error is sensation; the database and state machine are the body’s state; publish commits an action to the outside world. After a failure, resume rebuilds the state idempotently, and rollback is recovery from injury. This thing has no robotic arm, but it already has senses and the ability to act, through the runtime. Which is why I’d rather call it functional embodiment — and I’m borrowing “embodiment” loosely, not to claim an API equals a robot arm, but to point at a stable sense-and-act loop wired into an external environment.
Liang puts embodied intelligence last, meaning robots entering the physical world — housework, elder care, real labor — with the order being strong cognition first, then AI helping develop robots. I don’t disagree that physical embodiment comes last. But one thing to add: some body that touches reality has to show up very early, or continual learning has no trustworthy material to learn from. Physical embodiment can be last; functional embodiment has to come before continual learning.
6. Recursive self-improvement isn’t a closed loop
Programming environments have a structural advantage other domains can only envy: their verifiers are cheap. This is probably one reason Liang bets on coding agents first: a compiler, unit tests, a benchmark — whether a candidate works is answered almost on the spot.
But be clear about what these verifiers actually sign. A compiler can only sign “it compiles.” Tests can only sign “these cases pass.” Meta can only sign “accepted or rejected under the current rules.” The market can only sign “this is what ended up happening.” No oracle signs “the whole system is correct.” What they give you is dense, cheap, local verification — not a complete proof of business correctness. Code and math got lucky: local verification is both cheap and dense there, so the “generate → verify → learn → regenerate” loop spins up most easily, and it’s the most cost-effective place to light the fuse on AI improving AI.
And for exactly that reason I’m a notch more conservative than Liang on recursive self-improvement. His chain runs smoothly: continual learning → can do what people do → can develop the next generation of AI → AI accelerates AI. I hit the brakes before that second arrow: even if the model generates correct code in one shot, it can’t self-certify; it has to go run once in reality to verify.
Code and math have cheap computational verifiers. In company operations, ad delivery, user behavior, robotics, the relevant oracle is no longer purely computational; it depends on external state. The information you need lives on Meta’s servers, in behavior that hasn’t happened yet, in the actual physical environment. What you hit here isn’t a “model isn’t smart enough” wall, it’s an information-theoretic wall: that information is structurally missing, and no amount of extra intelligence can derive it out of thin air.
Someone will say: then train a precise-enough simulator, or use a model as the judge, and self-play in your head — done. And that does work, up to a point: self-play can widen the search and surface strategies that weren’t in the training set, and a simulator can save you a lot of real queries. But they can’t adjudicate facts that exist only in the current external world — Meta’s rules right now, the user’s reaction tomorrow, aren’t inside any simulator. And a learned judge — the model’s own judgment used as the oracle — inherits the model’s blind spots and becomes a target for the optimization process to game. So AI improving AI doesn’t become a loop closed inside the head; it stays coupled to external verifiers, experimental facilities, and real feedback.
7. The reality-signed trajectory
Weld the two together. Liang gives the vertical staircase of capability; what I want to add is the one horizontal rung between agents and continual learning:
chain-of-thought → agents → a trustworthy “sense–act–adjudicate–attribute–converge” loop → continual learning → AI helping extend AI (the rate bounded jointly by model capability, compute, and external experimental and verification throughput) → physical embodiment.
Reality does two jobs here. It signs experience. And where an invalid action can be rejected before commit and the system can recover, reality is the last line of defense. A constraint preflight missed is caught at publish time by an error from Meta; the error goes back to the draft, and the system resumes, rebuilds idempotently, or rolls back. But there’s a boundary to keep: not all real feedback is safe to lean on this way. Reject-before-commit, yes; feedback that arrives after the money’s spent, after the user’s been hurt, no. The harness’s job is to move real feedback ahead of irreversible consequences wherever it can, and where it can’t, to cap how much a single failure can cost. That’s what actually closes the airplane loop — not crash-then-learn, but letting local systems fail while the whole thing stays in the air.
So the future probably isn’t “once models learn continually, the backend and the harness disappear.” More likely it’s a division of labor: the model proposes candidates, the runtime touches reality, the oracle adjudicates, the harness caps loss and recovers, and the learning system attributes outcomes and distills trustworthy trajectories into updated capability — and that last step, attribution, isn’t guaranteed to be solved in the general open-world case. The stronger the model, the closer the first draft, the shorter the reflex arc. But the more actively the model acts on the world, the more that trustworthy-feedback structure becomes the load-bearing infrastructure.
One line to close: Continual learning isn’t the model endlessly saving its own thoughts. It’s turning the reality-signed trajectory of an action into the capability for the next one.
A note on sourcing: what’s circulating from the talk is a media-compiled transcript, not DeepSeek’s official verbatim record — the circulating 52-, 80-, and 118-point summaries may have edited wording (see the compiled versions on 36Kr and ifeng). Treat the quotes as paraphrase, not scripture.