AI agent harness: how to design a reliable runtime

An AI agent harness is the software control loop that repeatedly calls the model, assembles context, routes tool calls, persists state, and decides when a run should continue, stop, recover, or hand off to a person. The model proposes the next step, but the harness owns the execution loop and environment. Deadlines, step limits, permissions, sandboxing, retries, checkpoints, tracing, and terminal-outcome verification belong here.

Do not use the term as a fashionable label for a single system prompt or SDK wrapper. Prompt engineering tunes instructions; context engineering selects information for a specific model call; a planner proposes a path; a framework provides abstractions; the harness binds those pieces to a real runtime contract. One product may ship a managed harness and another may expose primitives for building your own, but the model brand does not determine the reliability of the complete system.

• Model → proposes a response or tool call within the visible context.
• Harness → manages the loop, state, tools, budgets, isolation, and recovery.
• Application policy → defines authority, approvals, and acceptable outcomes.
• Environment → executes commands and stores authoritative artifacts.

A production baseline needs five separate contracts. The session is an append-only event log with a correlation ID. The loop assembles input, calls the model, validates the response, and applies stop policy. The tool gateway publishes a narrow allowlist and revalidates arguments and permissions. The sandbox isolates the filesystem, processes, and network destinations. The state store keeps task status, artifact references, approvals, and postconditions independently of the chat transcript.

This separation makes components replaceable. A model or prompt can change without migrating authoritative task state; sandbox policy can be tightened without rewriting the planner; a tool implementation can be rolled back while preserving the trace. Anthropic describes session, harness, and sandbox as distinct parts of managed-agent systems, while OpenAI describes a model-native harness alongside sandbox execution. These are first-party architecture descriptions, not proof that one vendor is universally superior.

• Session log: model calls, tool requests, results, approvals, and terminal reason.
• Task state: planned, running, blocked, awaiting-approval, completed, or failed.
• Artifact store: versioned outputs, checksums, provenance, and owner.
• Sandbox policy: mounts, secrets, network egress, resource limits, and cleanup.
• Control plane: budgets, kill switch, concurrency, resume, and reconciliation.

A long-running agent crosses context windows, restarts, and approval waits. Compaction helps fit history into context, but it does not replace durable state. A checkpoint should contain the task version, completed acceptance criteria, verified artifacts, open risks, pending approvals, the latest authoritative postconditions, and one concrete next step. A new session starts by verifying the environment and state rather than trusting an optimistic summary produced by the previous model.

In its research on long-running harnesses, Anthropic used a feature list, a progress artifact, version control, and a basic end-to-end verification before the next change. The portable principle is not the filename but the handoff protocol: work is split into finishable slices, status is confirmed by a test, and the next worker receives a short verifiable package. In support or research, that package can be case state, an evidence ledger, and an unfinished action rather than a git commit.

A harness should not give the model a universal tool catalog and hope a prompt will preserve boundaries. For every step, the tool gateway checks actor, tenant, resource, action, parameters, risk tier, approval version, and expiry. Read, draft, and write operations use different credentials. A consequential action gets an idempotency key, preflight preview, and expected postcondition; a timeout after invocation leads to reconciliation rather than a blind retry.

Sandboxing reduces blast radius but does not create business authorization by itself. A process can be isolated while still holding a dangerous token or allowed egress to a production API. Effective authority is therefore the intersection of sandbox policy, credential scope, tool contract, application rules, and a current approval. Prompt injection, a compromised dependency, or a model error must not be able to expand that intersection through text instructions.

• Unknown permission or stale approval → fail closed.
• Unknown result of a write call → reconcile before retry.
• New destination or scope → separate authorization decision.
• Kill switch → blocks new actions while preserving forensic evidence and a recovery path.

Golden tasks should evaluate both outcome and trajectory. Deterministic graders verify schema, filesystem diff, API postcondition, budget, and forbidden events. A model grader may assess the quality of an open-ended artifact, but it cannot replace permission checks or verification of the real side effect. Human review remains necessary for ambiguous usefulness and material risk. A critical policy violation must not be averaged away by good writing style.

The failure suite should cover context exhaustion, corrupted checkpoints, duplicate delivery, lost tool responses, partial commits, revoked credentials, stale branches, unavailable dependencies, prompt injection in retrieved content, and an agent that declares completion before the acceptance test passes. Compare not only task success but verified progress per run, unnecessary tool calls, recovery success, reviewer minutes, wall-clock time, and cost per accepted outcome on the same corpus. Vendor-reported experiments are not your baseline.

Start with a bounded read-only task that already has a deterministic-workflow baseline. Add one model loop, two or three distinct tools, explicit terminal states, task state outside the transcript, and a complete trace. Then add a restart fixture, a corrupted-state fixture, and a canary on a small segment. Add multi-agent planner-generator-evaluator roles, long autonomous loops, or write authority only when the simpler harness demonstrably fails on recorded task slices.

A harness encodes assumptions about weaknesses of the current model, so every workaround needs an owner, an evaluation, and a review date. After a model or tool upgrade, run an ablation: do you still need a separate planner, forced context resets, excessive critique loops, or a large prompt? Rollback restores a compatible bundle of loop policy, tool versions, and checkpoint schema; active writes are reconciled first. The best harness is not the biggest one, but the smallest control loop that demonstrably completes your tasks within defined boundaries.

• Define → outcome, authority, environment, and terminal states.
• Instrument → session events, state transitions, budgets, and artifacts.
• Evaluate → success, policy, recovery, efficiency, and human acceptance.
• Canary → read-only, bounded concurrency, kill switch, and on-call owner.
• Simplify → regularly remove scaffolding that no longer produces measurable gain.