Introduction

Strip an AI agent down and you find two things: a model that predicts text, and a harness — the code around it that plans steps, calls tools, enforces budgets, handles failures, and records everything. The model gets the demo. The harness survives production.

Harness engineering is the discipline of building that loop well. It's unglamorous, and it's where reliability lives.

What the Harness Owns

┌──────────────── HARNESS ────────────────┐
│  planner → context → model call → parse │
│     ↓ tools ↓ budgets ↓ retries ↓ logs  │
│  verifier → checkpoint → answer/escalate│
└─────────────────────────────────────────┘

Everything outside the raw inference call is harness territory:

ResponsibilityExample
Step loopMax 12 steps, then summarize-and-ask
Tool gatewaySchema validation, timeouts, auth, rate limits
BudgetsTokens, wall-clock, dollars per task
StateConversation, plan, scratchpad, checkpoints
RecoveryRetry once, repair loop (max 2), escalate
ObservabilityTrajectory log of every step and token

The Tool Gateway Is the Real API

Models should never call the outside world directly. Every tool goes through a gateway that enforces contracts:

@dataclass
class ToolResult:
    ok: bool
    data: dict
    latency_ms: int
    error: str | None = None
 
def execute_tool(call: ToolCall) -> ToolResult:
    spec = registry.get(call.name)          # unknown tools rejected
    args = spec.schema.validate(call.args)  # malformed args rejected
    with timeout(spec.timeout_ms):          # hung tools killed
        return spec.handler(args)

Budgets: Three Numbers Every Harness Needs

const TASK_BUDGET = {
  maxSteps: 12,        // loop iterations
  maxTokens: 24_000,   // total in + out
  maxCostUsd: 0.50,    // real money per task
};

When a budget trips, the harness doesn't just stop — it summarizes progress and asks a pointed question. A budget trip is a handoff, not a crash.

Evals Are the Harness's Test Suite

You can't assert agent == correct. You can:

  1. Golden tasks — 20–50 realistic scenarios with checkable outcomes, run in CI on every harness change.
  2. Trajectory review — sample real runs weekly; classify failures (wrong tool, bad retrieval, premature stop, format drift).
  3. Cost per task — tracked per run, alerted on drift. A harness regression often shows up in spend before quality.
def grade_run(run: Trajectory, expectations: Expectations) -> Verdict:
    checks = [
        expectations.outcome_achieved(run.final_state),
        run.steps <= TASK_BUDGET.maxSteps,
        run.cost_usd <= TASK_BUDGET.maxCostUsd,
        all(t.validated for t in run.tool_calls),
    ]
    return Verdict(passed=all(checks), checks=checks)

Build Order That Works

  1. Hardcode the happy path — script the workflow end to end with fixed steps.
  2. Add one decision point — let the model choose at exactly one branch.
  3. Add budgets + logging — before widening autonomy, instrument everything.
  4. Add evals — golden tasks in CI.
  5. Widen autonomy gradually — convert scripted steps to model-chosen, one at a time, each guarded by evals.

Key Takeaways

  • The harness — loop, tools, budgets, recovery, logs — is the product; the model is a component.
  • A tool gateway with schemas, timeouts, and auth is non-negotiable.
  • Budget trips should hand off gracefully, never crash silently.
  • Golden-task evals in CI are what let you widen autonomy safely.