Why this is a big problem
A self-evolving agent is one that gets better at its job by changing itself while it runs. The interesting version is not the one that retrains its model. It is the low-dimensional one: the agent edits a small, inspectable set of things it owns, and those edits compound over a long run.
Framed that way, it stops being one feature and becomes a bundle of hard problems that are usually studied apart:
- Autonomy. How much the agent decides for itself, and how that scope should grow with evidence.
- Long-horizon operation. Runs measured in days, across many context windows, not one session.
- Memory optimization. What to keep, what to compress, what to forget, and who decides.
- Drift. Long tasks quietly wander from the goal. Self-modification makes that worse unless something anchors it.
None of this belongs to one model or one harness. The same questions show up whichever model sits underneath and whichever loop drives it, which is exactly why they are worth studying on their own.
Two layers, one of them out of scope for now
Model layer
Pretraining data and post-training alignment shape how well a model can take advice from its own past. Real work lives here, but it is not the subject of these notes yet.
Harness layer
Everything around the model: the loop, the memory, the tools, the permissions, the review. This is where an agent can evolve today, with any model. This is the layer these notes are about.
The boundary question: constrain or encourage
Most harness design is about constraints: what the agent may not do. That is necessary and it is the smaller half. The larger half is encouragement, and encouragement is not a softer kind of rule. It is attention. Telling an agent "spend effort here" changes what it notices, what it retries, and what it writes down, far more than another prohibition does.
So the design problem is where to draw the line between the two:
- Constrain the actions whose cost is irreversible or lands on someone else: deletes, payments, anything outward-facing, anything that rewrites the agent's own guardrails.
- Encourage nearly everything else, and be explicit about it. An agent that is only told what not to do will optimize for not being wrong, which is a different objective from doing the job.
- Move the line with evidence. The scope of autonomy should be a function of what the agent has already shown it can do, recorded somewhere both sides can read.
What an agent can evolve without new weights
If the model is fixed, the agent still owns three things it can change mid-run, and each is a different kind of self-improvement:
- Memory. The cheapest and most common. Facts, preferences, corrections, and the reasoning behind them. The hard part is not writing memory but curating it: marking what has gone stale, resolving contradictions, and keeping the index small enough that recall stays sharp.
- Tools and skills. Procedures the agent writes for itself once a workflow has been done twice. A skill is a compressed, replayable memory of how a task went. It evolves by being run again, failing in a new way, and being patched with the reason.
- Its own components. The loop, the prompts, the checks, the sub-agents. This is the least explored and the most dangerous, because a change here alters how future changes get evaluated. It needs a review step the agent cannot edit.
Together these form a small state space. Small enough to diff, version, and roll back. That is the property that makes agent-layer evolution tractable in a way weight-level evolution is not.
Open questions
- What does the agent measure itself against, so that "improved" is not just "changed"?
- How does memory stay honest across models and harnesses, when each one reads it differently?
- What anchors a multi-day run to its original goal without freezing it?
- Which self-modifications should require a second opinion, and from whom?
- How much of this transfers when the underlying model is swapped?
There is a lot left to explore here. If any of it overlaps with what you are working on, reach out.