Agentic Harness Engineering: The Real Job Is Building the Harness

The industry spent two years arguing about which model writes the cleanest code. That argument misses half the system. A raw model can’t hold state, run a test, or open a PR. It becomes an agent only once you wrap it in a harness — and the harness is where the real engineering now lives.

Agent = Model + Harness. If you’re not the model, you’re the harness. — Vivek Trivedy, LangChain

A harness is everything around the model: system prompts and AGENTS.md files, tools and MCP servers, sandboxes and filesystems, orchestration, hooks, and feedback loops (Addy Osmani). A decent model with a great harness beats a great model with a bad harness. The gap between what models can do and what you see them doing is largely a harness gap.

The discipline has a name now

Harness engineering treats that scaffolding as a first-class artifact that tightens every time the agent slips. The mindset: every agent mistake is a signal, not a one-off. The agent didn’t know a convention, so it goes in AGENTS.md. It ran a destructive command, so a hook blocks it. It got lost in a 40-step task, so you split it into a planner and an executor (Addy Osmani).

Martin Fowler’s Birgitta Böckeler frames the same loop as feedforward and feedback controls (source):

• Guides (feedforward) steer the agent before it acts — AGENTS.md, skills, conventions. They raise the odds of a good first attempt.
• Sensors (feedback) observe after the agent acts so it self-corrects — tests, linters, type checkers, AI code review. The most powerful sensors emit signals optimized for the model to consume.

Both come in two flavors: computational (deterministic, fast, cheap — run on every change) and inferential (semantic judgment, LLM-as-judge — slower, probabilistic, run selectively). The human’s job is the steering loop: whenever an issue recurs, you improve the harness so it’s less probable next time.

What it looks like at the frontier

This isn’t theory. OpenAI’s Codex team shipped an internal product with zero manually-written code — every line written by agents — and estimates they built it in roughly 1/10th the time (source). Their lesson: early progress was slow not because the model was incapable, but because the environment was underspecified. The engineering job became enabling the agents — making the app, logs, and metrics directly legible to the agent, treating repo knowledge as the system of record, and enforcing it mechanically with linters and CI.

Anthropic’s harness work converged on the same primitives for long-running builds: decompose work into tractable chunks, hand off context between sessions with structured artifacts, and separate the agent doing the work from the agent judging it (source). A generator that grades its own work skews positive; a skeptical, standalone evaluator gives it something concrete to iterate against.

The fix is almost never “try harder.” It’s “what capability is missing, and how do we make it legible and enforceable for the agent?”

The next frontier: the harness has to improve itself

Here’s the problem hiding in plain sight. The steering loop is manual. A human inspects trajectories, spots recurring failures, and hand-edits prompts, tools, and middleware. That loop is expensive, hard to scale, and doesn’t keep pace with model capability.

Recent research points at the way out. Agentic Harness Engineering (AHE) automates harness-level evolution by instrumenting three observability pillars — component, experience, and decision — so every edit becomes a falsifiable contract verified against the next run’s outcomes. Ten iterations lifted pass@1 on Terminal-Bench 2 from 69.7% to 77.0%, beating the human-designed harness, and the improvements transferred to other benchmarks and model families (arXiv).

The takeaway is bigger than a benchmark: the harness itself is something you can observe, evaluate, and improve automatically — if you have the observability to attribute outcomes back to changes. The discipline stops being “a human gardening prompt files” and becomes “a system that learns from every accepted outcome.”

Why most teams can’t run this loop yet

The frontier labs have an advantage most teams don’t: every task, run, log, artifact, PR, review state, and accepted outcome lives in one legible place. For everyone else, that signal is scattered. You’re running Copilot, Cursor, Claude Code, and Devin across different repos, trackers, and CI. The runs are invisible. Nothing connects this ticket to that agent run to this merged PR to did it actually hold up in production.

Without that connective tissue, you can’t close the loop. You can’t tell which harness change helped, which agent is worth the spend, or which patterns to encode as the next guide or sensor. You’re stuck hand-editing AGENTS.md and hoping.

Where Superfactory fits

Superfactory is the harness layer for your whole SDLC — a control plane that sits above the agent layer and stays neutral across coding agents, IDEs, issue trackers, Git hosts, and CI.

It does for your organization what frontier teams built by hand for a single repo:

• Ingest the signals — issues, incidents, reviews, telemetry — that should drive agent work.
• Put agents to work with full run visibility — every task, run, log, artifact, and PR in one place, so agent work stops being invisible.
• Close the loop — connect ticket → agent run → PR → review → deploy → outcome, so you can attribute results back to what changed.
• Learn from every accepted outcome — turn recurring failures into better guides and sensors, so the next run starts smarter.

That’s the AHE thesis applied at org scale: a self-improving software factory, not a pile of disconnected agent runs. Harness engineering is the discipline. Superfactory is where you run it.