The extremely smart intern

Delegating software work to coding agents without giving up responsibility.

I have recently found it useful to think about coding agents as extremely smart interns. They can do things that would make an experienced programmer stop and take notice, yet they can also make mistakes which seem almost absurdly inexperienced. The contradiction disappears if you take both parts seriously. They are capable of a great deal of reasoning, but they know almost nothing about the history of this system, the failed ideas which preceded it, the conventions nobody wrote down, or the reasons why apparently odd pieces of code have survived for ten years.

In my own work, current models will usually get somewhere between 90 and 99 per cent of the way to a correct change at the first attempt. I have yet to find one that can be relied upon to go the rest of the way without a separate step in which someone checks and validates what it has done. Most of what follows is about how to handle that final step without throwing away the benefit of the first.

That changes how I think we should work with them. The answer cannot be to supervise every edit, because then much of the advantage disappears. But neither can it be to describe a task, go away and accept whatever comes back. A good programmer would not manage a very bright new colleague either way. You would spend time making sure they understood what mattered, discuss the important choices before they committed too far, give them room to work, and then review what came back.

This sounds less like prompting and more like delegation.

Start with the outcome

A lot of agent instructions are still written as sequences of activities: change this class, replace that API, add these tests, refactor these methods. That can work, but it leaves the agent making local decisions without much sense of why one answer is better than another.

It is usually more useful to describe the outcome. There should be one obvious way for this package to update a customer. Existing behaviour must remain unchanged. The transaction boundary should remain visible at the level of the business operation. Callers outside the package should not need to understand how the update is implemented.

That is still a specification, but it is a different kind. It tells the agent what must be true when the work is finished, and gives it some freedom over how to get there.

This is where The Art of Action started to seem relevant to me. Stephen Bungay’s argument is about the gaps that inevitably appear between plans, actions and results. We never know everything, instructions are never understood perfectly, and actions do not always have the effects we expected. The usual organisational response is to close those gaps with more detail and more control, but beyond a point that simply makes the organisation slower while leaving the uncertainty in place.

Software agents have the same shape of problem. We cannot put everything we know about a system into their context, and trying to do so would probably be a mistake anyway. Much of what an experienced programmer knows is tacit, local and historical. We therefore need to give the agent enough intent to make sensible decisions without pretending we can specify the entire job in advance.

Acceptance criteria become especially useful here. They are not a disguised implementation plan. They are a description of what we will later use to decide whether the work succeeded.

Check that you are solving the same problem

One of the ideas in The Art of Action is the backbrief. Someone given an objective explains back what they think the objective is and how they intend to approach it. It is a way of finding disagreement before effort has turned it into code, equipment movements or organisational momentum.

That seems almost made for agent work.

Before a substantial change, I want the agent to tell me what it thinks the goal is, what constraints it has noticed, what assumptions it is making and which parts of the system it expects to affect. I also want it to say how it thinks we will know when the task is complete.

This is not the same as asking permission before every step. The purpose is to align understanding at the beginning so that supervision is less necessary later.

There is probably another conversation at this point too. The programmer is still responsible for the design, and there will often be choices worth discussing before the agent starts changing things. An agent can make the old advice to design something twice much cheaper. Ask for genuinely different approaches, not three variations on the same decomposition, and then talk about the consequences.

One design might preserve an existing module boundary while another changes it. One might extend the schema while another derives the information at read time. One may require a new abstraction while another deliberately avoids introducing one. The value is not in generating alternatives for their own sake. It is that the alternatives expose decisions while they are still cheap to change.

The programmer can then choose a direction, combine parts of two approaches, or decide that none of them are acceptable. That is a good use of human attention. Watching an agent mechanically rename twelve methods is not.

Then let it work

Once the goal, constraints and important design choices are reasonably clear, the agent needs room to do the job.

This is surprisingly difficult because agent interfaces make observation almost free. We can watch commands run and files change in real time, and the temptation is to intervene whenever the implementation takes a turn we did not expect. But if every unexpected local decision requires approval, we have not really delegated anything.

There should still be boundaries. Do not change the public API. Do not add a dependency. Stop if the schema needs to change. Do not cross this module boundary without coming back. Those are decisions whose effects extend beyond the piece of work being delegated.

Inside those boundaries, however, the agent should be able to discover that an idea does not work, inspect another part of the system, revise its approach and continue. Otherwise we have taken something capable of independent reasoning and reduced it to an unusually elaborate editor.

The point at which it says it is finished

The next problem is that agents are very willing to declare success.

In my experience the usual failure here is an omission rather than a false claim. The agent solves the difficult part of the problem and then simply forgets a step it would certainly have taken if asked, most often running the tests. The work may well be correct, but nobody has yet shown that it is.

Humans do this too. We solve the interesting part, see some green tests and quietly change the question from “is this complete?” to “is there any reason I cannot stop now?”

This is where The Checklist Manifesto seems useful. Atul Gawande’s point was not that experts need step-by-step instructions for doing their jobs. It was that complicated work contains small, important things which competent people still forget, especially when attention has been consumed by the difficult part of the problem.

A completion checklist for an agent should therefore be short. It should not describe how to write software. It should ask about the things a capable implementer is liable to overlook once it believes the main problem has been solved.

Run the relevant tests and show the output. Show how each acceptance criterion has been demonstrated. Describe what happens at the failure boundaries. Check whether any unrelated behaviour has changed. Identify any dependency or abstraction that was introduced, and why it was needed. List anything that is being assumed rather than shown.

The phrasing matters. Asked whether it ran the tests, an agent that forgot may answer from its impression of what it did. Told to run them and show the output, it just does the step it missed, and the output becomes evidence I can look at rather than a statement I have to take on trust.

The checklist may need a few items specific to the kind of change. A database transaction deserves questions about isolation and failure. A security change deserves another set. A schema change another.

The useful distinction is that the checklist comes after the agent thinks it has finished. It is a deliberate interruption to confidence, not a recipe for doing the work.

The programmer still has to understand the code

Even that is not enough.

The code can satisfy every acceptance criterion and still leave us with a problem: the programmer who is responsible for the change may know less about the system than they would have done had they written it themselves.

That is a new problem, or at least a much larger version of an old one. Writing code has always been one of the ways programmers build a mental model of a system. If an agent can disappear for twenty minutes and return with a substantial working change, that incidental transfer of understanding no longer happens.

The problem also becomes harder as the models improve. When an agent got 70 per cent of the way, the programmer had to work closely with the code to finish the job, and built a mental model of the change as a side effect. When it gets 99 per cent of the way, the sensible response is to skim, and the remaining 1 per cent is harder to see precisely because everything around it is right. Aviation and other heavily automated fields have known this for a long time: the more reliable the automation, the less attention people pay to it, and the less well placed they are to catch the rare failure when it arrives.

A generated summary of files changed does not solve it. Git already knows which files changed. What I want from the agent after the work is closer to another backbrief, but this time about the decisions it actually encountered.

It should explain what changed in structural or behavioural terms, why it changed that way, what plausible alternatives it considered and deliberately did not change, and where it is still uncertain.

The “not changed” part may be particularly important because it is invisible in a diff.

An agent might tell me that it considered moving a transaction into a repository, but left the boundary at the transaction-script level because the alternative would split one business operation across two independently callable methods. Or that it considered introducing an interface around a component, but found one implementation and no requirement for substitution, so introducing another public concept would have bought nothing.

Those are the kinds of things I would have thought about if I had written the code myself. They are also the things I need to know if I am expected to own the result.

I am not sure “code summary” is the right name for this. “Decision backbrief” is closer. The point is not to narrate the coding session or justify every line. It is to reconstruct the decision surface: the choices that mattered, including the roads deliberately not taken.

Forgetting can be useful

There is another property of agents which may be unusually valuable for review: we can create a reviewer who was not there.

Human reviewers usually share some context with the author. They may have attended the refinement meeting, heard the design argument and know what the author meant. Often that is useful, but it also means they inherit some of the same assumptions.

A fresh subagent need not.

Give it the acceptance criteria and the finished system, but not the implementing agent’s explanation, and ask it to test the behaviour from the outside. Give another fresh agent the code and ask it to look for incorrect assumptions, unnecessary complexity or boundary violations. Give another the original problem and the resulting interface and ask whether the solution appears to solve the problem that was actually stated.

The implementing agent knows why the solution ought to work. The fresh agent does not, which is exactly the property we often want from an independent tester.

For years we have tried to preserve useful separation between implementation and verification while deadlines and shared meetings steadily pull everyone towards the same mental model. Agents let us manufacture a new reviewer with none of that shared history at almost no cost.

Their ability to forget is not always a weakness.

Delegation does not transfer responsibility

None of this makes the agent responsible for the software. The programmer still decides whether the change belongs in the system, still needs to understand the important decisions, and still puts their name behind the result.

Agents make a distinction visible that was easier to ignore when authorship and understanding usually arrived together. Typing the code is not the same thing as understanding it, but writing it forced us to spend time with the problem. If the typing and much of the exploration are delegated, then we need another way to preserve that understanding.

The resulting workflow is not especially complicated. Agree the goal and acceptance criteria. Discuss important design alternatives. Ask the agent to backbrief its understanding. Let it implement within agreed boundaries. When it thinks it is finished, make it run a completion checklist. Then ask for a decision backbrief, use fresh agents where useful for independent verification, and finally decide whether you understand the change well enough to accept responsibility for it.

Some of this will date quickly. The specific checklist items, and the particular ways agents fall short, reflect the models I am using now, and those improve every few months. I expect the argument about responsibility to last much longer. Whoever merges the change still has to own it, and as the agents get better, the pressure to accept their work without understanding it will only grow.

Small changes need very little ceremony; dangerous changes need more. The point is not to replace programming with a new process.

It is to move human attention to the places where it matters.

That may be why “prompt engineering” has never felt quite right as a description of this work. The difficult part is not finding a magical sequence of words that makes the model obey. It is deciding what to delegate, making the intended outcome clear, preserving the important design decisions for the human, and constructing a way to know whether the result deserves to be accepted.

That sounds much more like delegation engineering.

A practical test

Take a change an agent has made that is waiting to be merged. Can the programmer who will merge it explain what changed, why it changed that way, what important alternatives were rejected, and what evidence shows that the result is correct?

If not, the agent may have finished coding, but the programmer has not finished the job.

References

• Stephen Bungay, The Art of Action: How Leaders Close the Gaps between Plans, Actions and Results (2011), on the gaps between plans, actions and outcomes, directing by intent, and the backbrief.
• Atul Gawande, The Checklist Manifesto: How to Get Things Right (2009), on short checklists for the steps competent people forget. The completion checklist here is what he calls a DO-CONFIRM checklist: the work is done from experience, then stopped and checked.
• John Ousterhout, A Philosophy of Software Design (2018), whose chapter “Design It Twice” is the advice to consider radically different designs before choosing one.
• Peter Naur, Programming as Theory Building (1985), on the understanding programmers build while they work, which the code does not carry on its own. https://gwern.net/doc/cs/algorithm/1985-naur.pdf
• Lisanne Bainbridge, Ironies of Automation (1983), on why automating most of a task leaves the operator worse placed to take over when the rare failure arrives. https://doi.org/10.1016/0005-1098(83)90046-8
• David L. Parnas, On the Criteria To Be Used in Decomposing Systems into Modules (1972), on the design decisions a module boundary hides, which are what the boundaries given to an agent protect. https://doi.org/10.1145/361598.361623