Where the Specification Lived

Question

How much of an agent's specification has to arrive in the prompt when the agent works inside a mature repository? Two agents were given the same task in the same repository: finish the Traffic Simulator that the site listed as in development. One prompt was 38 words. The other was 615. The working thesis was that prompt length and specification load are different variables, because much of the effective specification may already live in the repository.

Where specification can live

A mature repository already specifies a great deal: AGENTS.md, tests, neighbouring implementations, types, names, documentation, route machinery, and visual conventions. What a prompt adds is the residual specification on top of that environment.

• Durable project invariants belong in the repository, where every task inherits them.
• Task-specific non-negotiables belong in the invocation, because the repository cannot know them.
• Genuinely open design decisions belong to the agent's judgment; specifying them in advance spends that judgment.

So the question is not whether short prompts beat long ones. It is where each part of the specification lived, and which copy of it each agent obeyed.

Experimental Setup

The two prompts

What Each Agent Built

• The Simple run built a three-scenario lab for emergent congestion: a signal corridor with three coordination modes, a three-lane highway that loses a lane (drivers merge using the MOBIL lane-change rule), and a ring road with a brake-tap button. Drivers follow the Intelligent Driver Model. A time-space diagram plots every vehicle's position over the last two minutes.
• The Detailed run built one deeper system: a two-way arterial with three signals and cross streets. Its signal controller always passes through yellow and all-red and holds a minimum green, including when the plan is edited mid-run. It offers fixed-time and detector-actuated control, a rule against entering an intersection you cannot clear, per-signal zoom, and per-direction measurements.
• Both wired the simulator into the same places: route, simulation card, page metadata, sitemap, static route shell, and smoke test. The Simple prompt named none of them; the Detailed prompt listed the kinds of mechanism but not where they lived. The Detailed run also added breadcrumb structured data and a browser regression script.

Rows are comparable only where the same measurement was applied to both. The mutation counts are not: each suite was tested against breakages of its own model's rules.

How the Artifacts Were Judged

What the agents reported

Each run ended with a careful, specific closing report. The Simple run listed tests for “red lights obeyed” and said an independent code review it ran “found no engine bugs.” The Detailed run reported that the eastbound approach “saturates at about 1,500 vehicles/hour,” and that switching off each driver rule in turn made a test fail.

Those reports were self-reports: each author's account of its own work, checked by reviewers it chose itself. They are evidence of what was claimed. They are not validation, so the artifacts were judged directly.

What the Evaluation Found

Outcome

• Before learning which prompt produced which branch, the evaluator recommended publishing the Simple artifact. Its breadth and its time-space diagram fit what this site's Simulations page is for: interactive explorations of rules, state, feedback, and emergence.
• The same evaluation judged the Detailed artifact the better-engineered one: safer signal transitions, stronger tests, and deeper treatment of the problem it chose.
• The Detailed run took 56 min 34 s to the Simple run's 26 min 55 s, about 2.1 times as long. That is a cost, not a quality score.

What changed before publication

• The live Traffic Simulator is a reviewed derivative of the Simple artifact, not an untouched copy. The experimental commit 7420cc7 is preserved unchanged by the tag traffic-sim-experiment-simple, and each change below was made on top of it.
• Signal lights now step only green → yellow → red → green. A timing edit that jumps the schedule shows a full 3 s yellow first. A regression test replays the same 49 abrupt edits and allows no red crossings. With no edits, the lights match the original schedule exactly, so the artifact's documented results are unchanged.
• The ring-road text now says the dense ring jams by itself after roughly 15 to 20 simulated minutes; the brake tap sets the wave off early.
• A paused simulator no longer runs animation loops or re-renders ten times a second.
• The time-space diagram draws only new samples: about 4.4 ms per sample instead of 100 to 165 ms for each full repaint, measured with 4× CPU throttling at phone width.
• The diagram has a data table listing where traffic is slow now and 30, 60, and 90 s ago, so its content is available as text and from the keyboard.
• The model suite grew from 14 to 21 tests aimed at rules the original suite passed by accident. Against the same 51 single-rule breakages, it misses 7 instead of 25: five change nothing the model can show, one is a gap check that another safety check already covers, and one changes how stops are counted without any effect the tests or probes could find. Three new breakages aimed at the post-evaluation code are all caught.

What this says about repository-resident specification

The Simple prompt said almost nothing about this repository, and the Simple run still reconstructed nearly all of its integration requirements: the route, simulation card, metadata, sitemap, static route shell, smoke tests, theme tokens, reduced-motion handling, and the existing split between model and renderer. That part of the specification came from the repository.

The Detailed prompt asked for things the repository did not require on its own, such as meaningful tests of the model's invariants and browser-level checks, and the Detailed run delivered them: a stronger test suite and a browser regression script. Those are task-specific non-negotiables, and the invocation is where they had to live. Its safer signal transitions were not in the prompt; they were the run's own design choice.

The Detailed prompt also listed signals, queuing, and throughput as minimum requirements, and the Detailed run built deeply around exactly those, while the Simple run, given no list, ranged wider. One pair of runs cannot show that the list caused the narrower scope. It is consistent with detail consuming decisions that could have stayed open.

Nor does this show that the repository always suffices. The Simple run shipped a red-light bug that its own tests and review missed, and the repository contained no traffic model from which to inherit the rule that a light never skips yellow.

Limitations

• One pair of runs, not a crossover: each prompt ran once.
• Agent variance and tool-use luck are uncontrolled; a second run of either prompt could produce a different artifact.
• The agents collided in a shared worktree before separating, so the runs were not fully independent.
• Wall time measures cost, not quality.
• The closing reports are self-reports and were not treated as validation.
• The evaluator was a separate session of the same model. Blinding hid which prompt produced which branch; it did not remove tendencies the evaluator may share with the authors.
• The mutation counts compare each suite against its own model's rules, so they are not a like-for-like score.

The principle

Put durable invariants in the repository, where every task inherits them. Put the task's non-negotiables in the invocation. Leave genuinely open decisions open. Then judge what comes back by the artifact, not the report. This experiment does not show that shorter prompts are better. It shows that the useful question about a prompt is which of those three places each requirement belongs in.

Evidence