Skip to main content

Harness Engineering

The system of context, tools, control loops, safety checks, and feedback mechanisms that turns a model into a usable agent.

Model Versus Harness​

A model generates outputs from the context it receives. A harness decides what context reaches the model, which actions are available, how actions are authorized and executed, when execution stops, and how results are checked. Instruction files and skills cover part of that system.

It is useful to separate two boundaries:

  • Tool-owned harness: runtime behavior implemented by the coding agent or agent platform, including tool execution, context limits, permissions, retries, and stop conditions.
  • User-controlled harness: repository instructions, scoped rules, prompt assets, skills, memory files, evals, and integrations that a project or user can configure.

The exact boundary varies by tool. Document observable controls and public interfaces rather than assuming how a product implements its private runtime.

Anatomy of an Agent Run​

StageHarness responsibilityRelated conventions
Request intakeParse the task, identify applicable policy, and reject unsupported inputAGENTS.md, tool-specific rules
Context assemblySelect relevant instructions, project facts, prior state, and capabilitiesAGENTS.md, MEMORY.md, Memory Bank, SKILL.md
Reasoning and action selectionDecide whether to answer or request a bounded tool actionPrompt assets, skills
Authorization and executionCheck scope, permissions, input shape, timeout, and side effects.aiignore, scoped rules, MCP
Result ingestionValidate and sanitize tool output before adding it to contextRuntime guardrails
Loop controlStop on completion, an unrecoverable blocker, or an explicit budgetPLAN.md, runtime policy
Result validationTest outputs against the task and expected behaviorEVAL.yaml
Observation and improvementRecord safe traces and use failures to improve instructions, tools, or evalsOpenTelemetry GenAI, OpenInference

Context Assembly​

A harness selects a working set from the files and memory available. It may draw from:

  • Procedural context: instructions, rules, skills, and playbooks that describe how to work.
  • Semantic context: durable facts about the project, domain, or user preferences.
  • Episodic context: relevant events and outcomes from earlier tasks or sessions.

Agents vary in how many of these memory categories they implement. Prefer explicit, reviewable project files for facts that must be portable across tools, and retrieve only what helps the current task.

The Tool Loop​

An agent run commonly alternates between model reasoning and tool execution. Each tool result becomes new context for the next decision. A safe loop defines:

  • which tools are available and what each tool can access;
  • authorization rules for reads, writes, network access, and destructive actions;
  • schemas for tool inputs and validation for tool results;
  • per-call timeouts, retry limits, and total action or token budgets;
  • completion and blocked conditions that prevent an unbounded loop.

MCP can standardize the tool boundary, but it does not define the host's complete loop, permission model, or success criteria.

Guardrails​

Guardrails operate at several points:

  1. Input checks reject or narrow unsafe and malformed requests.
  2. Context checks prevent secrets, irrelevant data, or untrusted instructions from being injected silently.
  3. Action checks enforce permissions before a tool changes external state.
  4. Result checks treat tool output as untrusted input before returning it to the model.
  5. Output checks verify format, evidence, and task-specific acceptance criteria.

A Markdown rule can express policy, but enforcement belongs in the runtime or tool boundary when failure would have material consequences.

Evaluation and Observability​

Evaluation asks whether the harness produced an acceptable result. Observability explains what happened during the run. Each answers a different question.

Useful measurements include completion rate, tool failures, retries, latency, token usage, estimated cost, policy denials, evaluator scores, and the stage where a task became blocked. Trace content can include sensitive prompts, responses, retrieved documents, and tool payloads; capture only what is needed, apply redaction, and define retention before enabling content-level telemetry.

A Minimal Adoption Path​

  1. Add AGENTS.md with verified build, test, style, and safety instructions.
  2. Exclude secrets and noisy generated content with .aiignore or supported scoped rules.
  3. Add a reviewable PLAN.md workflow for multi-step or high-risk changes.
  4. Extract repeated specialized workflows into SKILL.md files.
  5. Add evals when behavior must remain stable across prompt, model, or tool changes.
  6. Add tracing when failures span multiple model or tool calls and cannot be explained from final outputs alone.
  7. Add MCP, A2A, or ACP only when the corresponding integration boundary exists.

Start with the smallest layer that addresses a measured problem. More context, tools, memory, and telemetry increase capability, but also increase cost, attack surface, and maintenance.