When a coding agent fails, teams often change the prompt or swap the model. Often the missing capability sits elsewhere: the agent could not find a rule, a tool returned noise, a permission boundary was vague, the runtime drifted, or “done” had no independent test.
Harness engineering gives those problems names and owners. A practical harness can be inspected as six layers.
One stack, six different contracts
The previous article treated a coding agent as a whole system. This model makes that system actionable. Each layer answers a different question:
- Context: What can the agent know?
- Tools: What can it ask the system to do?
- Policy: What is it allowed to do?
- Sandbox: Where may side effects happen?
- Feedback: What did the environment return?
- Verification: What proves the outcome is acceptable?
Policy constrains a tool before execution; sandbox boundaries shape what feedback is possible; verification may request more context or another action. Teams can improve one contract without pretending every concern belongs in one agent prompt.
Layer 1: context makes the environment legible
Context is more than the conversation. It includes repository maps, local instructions, architecture decisions, relevant code, tests, issue history, task state, and the observations accumulated during the run.
A good context layer practices progressive disclosure. It gives the agent an entry point, reliable search, and paths toward deeper material. It also protects provenance and freshness. An instruction should be distinguishable from an old plan; generated output should not masquerade as source; a cached repository summary should not quietly outlive a refactor.
OpenAI’s harness engineering case study describes this as making the repository the system of record and giving agents a map rather than a monolithic manual. Context succeeds when the agent can locate the smallest trustworthy slice needed for the next decision.
Layer 2: tools turn intent into precise action
Tools translate a model proposal into operations the environment understands: search, read, edit, shell, browser, database, version control, deployment, or domain-specific APIs.
Tool quality lives in its contract. Names should be distinct, parameters typed, destructive effects obvious, errors structured, and output bounded. The tool interface should support narrow actions such as reading a file range or running one test before forcing a broad, expensive operation.
OpenAI’s description of the Codex agent loop shows tools as first-class items in the trajectory rather than hidden implementation details. That visibility matters: an engineer should be able to tell what the model requested, what actually ran, and what came back.
Layer 3: policy turns access into authority
A model can propose any action expressible in text. Policy decides which proposals receive authority.
Policy includes allowed paths, network rules, secret access, command classes, spending or token budgets, approval requirements, and stop conditions. It should distinguish reading from writing and reversible changes from destructive ones. A blanket “allow tools” switch is not a mature policy model because it erases the risk differences among actions.
Policy also expresses ownership. A human may delegate routine edits while retaining decisions about production, money, privacy, or architectural taste. The boundary should be explicit enough that the system can pause and explain what decision is needed.
Layer 4: the sandbox bounds real effects
Policy is a decision; a sandbox is enforcement. It constrains where files can change, which processes can run, what network destinations are reachable, how credentials are exposed, and how resources are limited.
A generated instruction cannot enforce its own security boundary. OpenAI’s account of building the Codex Windows sandbox frames the problem directly: coding agents need enough access to build and test software, while the environment must automatically constrain writes and network use within safe bounds.
A useful sandbox is also reproducible. Worktrees, containers, disposable databases, seeded fixtures, and isolated browser sessions let an agent inspect the same state it changes. Isolation reduces ambiguity: a failure can be connected to this task rather than an unrelated local mutation.
Layer 5: feedback makes consequences legible
Feedback is the observation returned after an action. It might be command output, an exit status, a diff, a screenshot, a DOM snapshot, a metric, a compiler diagnostic, or an approval response.
More output is not necessarily better feedback. A ten-thousand-line log can hide the one fact that changes the next move. Harness design should compress observations without removing provenance: preserve the failing assertion, file location, relevant log window, changed state, and status that distinguishes timeout from failure.
Feedback closes the operational loop. Without it, the agent may continue acting but cannot update its plan from the environment. That is open-loop execution wearing the appearance of autonomy.
| Signal | Required evidence |
|---|---|
| Request | Authorized action |
| Execution | Bounded observation |
| Completion | Independent check |
Layer 6: verification defines completion
Feedback says what happened. Verification asks whether the requested outcome is true.
A command can exit successfully while the product behavior remains wrong. A unit test can pass while the page overflows on mobile. A generated diff can compile while violating an architectural boundary. Verification therefore needs independent acceptance criteria: tests, static checks, visual inspection, accessibility audits, performance budgets, policy checks, or human review where judgment remains irreducible.
The generator should not be the only evaluator. Anthropic’s work on long-running application-development harnesses separates planning, generation, and evaluation to make quality criteria concrete and reviewable. The broader principle is stable: completion should be grounded in evidence produced against the goal, not in the agent’s confidence.
Control plane, data plane, and ownership
The six layers become easier to operate when two cross-cutting planes are visible.
The control plane carries intent, policy, budgets, approvals, and stop rules. It governs the trajectory. The data plane carries actions, observations, changed files, trace events, and verification evidence. It records what the trajectory did.
Persistent task state connects both. A resumed session needs to know not only the current files but also the accepted goal, remaining work, approvals already granted, checks already run, and evidence still missing. OpenAI’s Codex App Server architecture treats threads, turns, items, events, and bidirectional approval requests as durable protocol concerns—useful evidence that continuity belongs in the harness rather than a lucky context window.
Turn failure into infrastructure
A mature harness does not merely recover from one failed run. It captures the reason and changes the environment so future runs inherit the correction.
A wrong file suggests a better repository map. An invalid call suggests a sharper tool schema. An unsafe request suggests a policy guard. A machine-specific failure suggests a reproducible sandbox. A noisy result suggests a better observation formatter. A false success suggests verification closer to the user-visible goal.
This is the heart of harness engineering: convert repeated correction into reusable capability. The next article focuses on the final layer, because it is where confident activity becomes accountable delivery—verification first, with evidence rather than self-reported certainty.
Sources
- OpenAI — Harness engineering
- OpenAI — Unrolling the Codex agent loop
- OpenAI — Unlocking the Codex harness
- OpenAI — Building a safe, effective sandbox to enable Codex on Windows
- Anthropic — Harness design for long-running application development
