Claude Code, Codex CLI, Cline, Hermes Agent, Kilo Code and OpenClaw all run the same commodity tool-calling loop — as their own docs and independent analyses describe — and Pi strips the loop down to four tools on purpose. Mapped against the eleven-layer harness architecture, here's what each one built around its loop, and what all seven still leave thin. Read the loop-first version on Design a Loop →
Each project reports its own headline numbers in its own units — stars, users, tokens — so the cards below show each on its own terms. Star counts and release versions were read from the GitHub API on 6 Oct 2026; Kilo's user and token figures are from kilo.ai as read on 24 Sep 2026 and differ from the 3M+ developers Anaconda cited in July.
Each agent's documented behavior mapped against the same eleven canonical layers used across Build A Harness — not an ad hoc feature list.
Scroll the table sideways — eight columns, Aielia last.
| Harness layer | Hermes Agent | Kilo Code | OpenClaw | Pi | Claude Code | Codex CLI | Cline | Aielia (reference) |
|---|---|---|---|---|---|---|---|---|
| 1. Caller State | ○Not described | ◐Agent "modes" (code/ask/plan/debug) fix tools + instructions per task | ◐Messaging channel/persona selects context — not a formal constraint system | ◐AGENTS.md / SYSTEM.md context files set per-project instructions; no formal mode or constraint system | ◐Permission modes (Manual / Accept edits / Plan / Auto) fix what Claude may do; CLAUDE.md and rules files scope instructions — no formal constraint system | ◐Approval policy + sandbox mode per session; AGENTS.md sets per-project instructions; no formal constraint system | ◐Plan / Act modes (Plan is read-only) and .clinerules rules, optionally path-conditional; no formal constraint system |
◐Per-turn intent classification sets risk level + tool scope; not a formal mode system |
| 2. World Model | ◐Static profile + identity-document config; no typed belief model | ◐Static AGENTS.md / rules files (Memory Bank is deprecated) | ◐Static persona/bootstrap files | ◐Static context files plus the session tree; no typed belief model | ◐CLAUDE.md hierarchy, auto-memory notes and a git-state snapshot; no typed belief model | ◐AGENTS.md plus opt-in memories; no typed belief model | ◐Static rules files; Memory Bank is a user-built convention, not a built-in; no typed belief model | ●Typed beliefs with provenance + staleness, rebuilt each turn from the persisted fact store |
| 3. Reasoning | ○Not described | ○Not described | ○Not described | ○Not described | ○Not described | ○Not described | ○Not described | ◐Typed evidence + contradiction layers run each turn; hypothesis generation is opt-in and off by default (AUDIT_SEMANTIC_HYPOTHESES), with no measurable benefit shown |
| 4. Control State | ◐Dangerous-command detection + approval callback; binary, not a tiered resolver | ◐Per-tool allow / ask / deny rules (edits, shell, MCP); not a tiered risk resolver | ◐Exec approvals (deny / allowlist / ask / auto / full) with per-agent allowlists; command-level, not a tiered risk resolver. Run queueing separately prevents races | ○Explicitly omitted — no permission prompts by design; gates only via optional extensions | ◐Deny → ask → allow rules, permission modes, an auto-mode classifier and managed policy; the docs say rules aren't a boundary around a program (use the sandbox); not a tiered risk resolver | ◐OS-enforced sandbox mode (read-only / workspace-write / danger-full-access) × approval policy, execpolicy rules, hooks, optional auto-review subagent; not a tiered risk resolver | ◐Eight auto-approve categories; the model flags each command requires_approval; YOLO mode removes all checks; the CLI auto-approves by default; not a tiered risk resolver |
●A turn-local control state, built from this turn's tool outcomes and folded with the harness's 5-tier NORMAL / CAUTIOUS / BLOCKED state, gates every read-only tool call; a real retry loop is blocked once 8 failures are counted. Write / shell / email are always staged for approval |
| 5. Planning | ◐Subagents get their own capped budget via delegate_task — no full dependency graph | ◐Subagent delegation for isolated subtasks | ○Not described | ○Plan mode, sub-agents and built-in to-dos are deliberately omitted (PLAN.md files instead) | ◐Plan mode and a read-only Plan subagent; subagents nest up to 3 deep and run up to 20 at once by default — no full dependency graph | ◐Plan mode; subagents on explicit request or project instructions, inheriting the parent's sandbox and permission mode — no full dependency graph | ◐Plan mode, parallel read-only subagents (no nesting), todo list that survives summarization — no full dependency graph | ◐Task-graph decomposition (code plan templates are opt-in and off), plus a hierarchical goal graph with next-step proposals (on by default in the CLI and browser); dependency-aware, not a full scheduler |
| 6. Execution | ◐70+ tools, ~28 toolsets, 7 terminal backends, concurrent dispatch | ●Git-snapshot checkpoints (captured at each model-call step, revert per user message) + diff-repair + tool-call translation | ◐Wide action surface (device/OS tools), gated by exec-approval policy | ◐4 core tools (read / write / edit / bash), optional grep / find / ls; 15+ providers; extensions add more | ●Five tool categories (files, search, shell, web, code intelligence); file snapshots before every edit with Esc-Esc rewind. Checkpoints cover file changes only, not remote actions | ◐Shell commands and patch-based edits run inside an OS-enforced sandbox; no built-in file-level checkpoint or rewind (an open request, #11626, asks for one) | ●Shadow-git checkpoint after every tool use; restore files, the task, or both. Tools cover files, shell, browser and MCP | ◐9 tools (read / list / write / shell / web_search / fetch_url / send_email / two reminder tools) — file writes, shell commands and email staged for approval; applied actions are logged for undo (binary or oversized content is marked not undoable) |
| 7. Verification | ○Not described | ◐Informal completion check + diff-repair | ○Not described | ○Not described | ◐Loop's third phase is "verify results"; Stop hooks (prompt- or agent-based) can force another pass; on-demand /code-review (alias /review) and /security-review for changes; no automatic structured verifier |
◐On-demand /review of uncommitted changes, commits or branches; no automatic verification step |
◐Reports compiler and linter errors as it works; no structured verifier | ◐9 checks defined, but only evidence sufficiency and output contract normally return a result in a chat turn; consistency, syntax, unit, integration and goal correctness report SKIPPED, and requirements / assumptions can only fail on an empty output |
| 8. Recovery | ◐Provider fallback, fixed iteration budget | ◐Checkpoint rollback, diff-repair | ◐Auto-compaction retry + provider fallback; no checkpoint rollback | ◐Session-tree branching and rewind, automatic compaction; no checkpoint rollback described | ◐Checkpoint rewind, session resume and fork, auto-compaction that stops after repeated thrashing instead of looping | ◐Resume recent sessions, automatic compaction past a token threshold; no file-revert | ◐Checkpoint restore, including to a state from before an auto-compact summary | ◐Replanning + one retry of a system-error failure + semantic failure-mode matching (both on by default) + crash-safe checkpoint resume + a Trajectory Supervisor for stalled runs (on by default) |
| 9. Memory | ●SQLite+FTS5, context compression with a 20-message floor | ◐AGENTS.md project context (Memory Bank deprecated), context condensing | ◐Raw JSONL transcript, token-limit + compaction reserve | ◐JSONL session trees, compaction, AGENTS.md; long-term memory left to extensions | ◐Saved JSONL transcripts, CLAUDE.md and auto memory (first 200 lines / 25KB load each session); every session starts with a fresh context | ◐Opt-in memories (off by default) distilled from past chats into ~/.codex/memories; AGENTS.md; session resume |
◐Task history, auto-compact summaries near the context limit, rules files; Memory Bank is a user-built convention | ◐Cross-session transcript + typed fact store; model-inferred facts need confirmation unless high-confidence; linear search (an inverted index is planned) |
| 10. Learning | ●Self-improving skills — the project's headline differentiator | ○Not described | ◐Skill Workshop / self-learning drafts and patches skills from corrections and successful work; auto mode by default, propose-only mode optional | ○Not described (extensions can edit the agent itself) | ◐Auto memory saves learnings and patterns from your corrections and preferences itself — notes, not a skill-authoring loop | ◐Opt-in memories extract durable entries from past chats; no self-authored skills described | ○Not described | ◐ExperienceStore is written only when learning is switched on (off by default), and then only reorders recovery strategies; bookkeeping, not shown to change an outcome (AUDIT_EXPERIENCE_LEARNING) — internal, not authored skills |
| 11. Output & Reviewer Pass | ○Not described | ○Not described — human review substitutes | ○Reply shaping is cosmetic, not a quality gate | ○Not described | ◐No built-in gate before a reply (on-demand /code-review and /security-review review changes); a configured Stop hook can ask a model if the work is complete and make Claude continue (capped at 8 blocks in a row) |
◐No built-in gate; a configured Stop hook can block the final reply and continue with the reason as a new prompt | ○Not described — human approval substitutes | ◐Implementer and reviewer lenses on every turn, plus an adversarial lens only on consequential (non-LOW risk) or multi-step (3+ task) turns, and an abstraction-fit check before a reply leaves, as advisory output; findings that revise the reply are opt-in and off, and no outcome benefit has been measured. A grounding check on tool-backed replies is on by default |
Reading the pattern: Across the seven agents, Execution, Memory, and Learning are the only layers where any reaches full coverage — checkpoints for Kilo, Claude Code and Cline on Execution, and Hermes's persistent memory and self-improving skills. Pi reaches full coverage on none, deliberately; so does Codex CLI, whose main strength, an OS-enforced sandbox, lands in the Control State and Execution cells as partial. Reasoning gets zero coverage from all seven. Output & Reviewer Pass has no built-in gate in any of them; Claude Code and Codex let you configure a Stop hook that approximates one, so both rate partial. Of the remaining layers, none is built out to full coverage by any of the seven — they're a patchwork of static substitutes, partial mechanisms, and outright gaps, with World Model and Recovery the only two where all seven have at least something, thin as it is. Which is a fair description of where most production agents are today, popular or not. The Aielia column is the reference point — Build A Harness's own assistant, included to show what the thin layers look like when something actually ships them (Control State at full coverage; Reasoning and the Reviewer Pass partial, since hypothesis generation and revising findings are opt-in and off by default; Verification is partial — only three of its nine checks run in a chat turn); it is the newest and least battle-tested of the eight.
Beyond the layer-by-layer mapping, each project reads differently in practice:
Scroll the table sideways — eight columns, Aielia last.
| Practical dimension | Hermes Agent | Kilo Code | OpenClaw | Pi | Claude Code | Codex CLI | Cline | Aielia (reference) |
|---|---|---|---|---|---|---|---|---|
| Model / provider breadth | 18+ providers, open-weight models | 500+ models via the Kilo Gateway | Claude, GPT, DeepSeek, or local models | 15+ providers (Anthropic, OpenAI, Google, Azure, Bedrock, Mistral, Groq…); switch model mid-session | Claude models, reached through a Claude plan, the Anthropic Console, Amazon Bedrock, Google Cloud's Agent Platform, Microsoft Foundry or an LLM gateway | OpenAI models via ChatGPT sign-in or an API key; other providers (e.g. OpenRouter, Ollama, LM Studio) through config | Claude, GPT, Gemini, Ollama and 200+ other models via OpenRouter and compatible APIs | Anthropic, OpenAI, OpenRouter (direct, bring your own key), or a self-hosted proxy |
| Primary reach | CLI, 25+ messaging and platform adapters, IDE (ACP), API, cron | VS Code, JetBrains, CLI, cloud agent, mobile | WhatsApp, Telegram, Discord, Signal, iMessage, Slack, Teams, native device apps | Terminal TUI, print/JSON mode, RPC, and SDK embedding | Terminal, VS Code, JetBrains, desktop app, web and mobile, Slack, CI (GitHub Actions, GitLab), Remote Control, scheduled routines | Terminal, IDE extensions (VS Code, Cursor, Windsurf), desktop app, Codex Web, cloud tasks | VS Code, JetBrains, CLI, desktop app, Node SDK, cron-scheduled runs | CLI, browser (hosted /try), Tauri desktop — no messaging or IDE surface yet |
| Business model | Free, MIT; runs on a few-dollar-a-month VPS | Free/BYOK or Kilo credits, zero markup on provider rates; acquired by Anaconda (announced 15 Jul 2026, terms undisclosed) | Free, MIT, self-hosted; foundation stewardship after the founder joined OpenAI | Free, MIT; company-backed (Earendil Inc.) | Proprietary (all rights reserved, Anthropic Commercial Terms); needs a Claude subscription or Console / cloud-provider billing; Teams and Enterprise per-seat plans | Free, Apache 2.0 (OpenAI); sign in with a ChatGPT Plus / Pro / Business / Edu / Enterprise plan or use an API key | Free, Apache 2.0 (Cline Bot Inc.); bring your own model or provider | Free, Apache 2.0, self-hosted; bring your own API key; no funding round |
| Known risk | Newest of the seven by public launch (Feb 2026) — memory and self-improving skills lack a long track record. 2026 CVEs include an injection issue in context scanning (CVE-2026-9366) and a sandbox issue in code execution (CVE-2026-9368), both CVSS 7.3 (v3.1) | Published 2026 CVEs include CVE-2026-79403 (CVSS 8.4), local code execution via a permission endpoint in versions before v7.4.1, and a low-severity information disclosure, CVE-2026-8766 (CVSS 4.3) | Critical CVEs up to 9.9 CVSS, 341 malicious marketplace skills disclosed, prompt-injection susceptible | No permission system by design — full filesystem and command access by default; sandboxing is left to the user (containers). Four advisories published 8 Jun 2026, including a high-severity local privilege escalation via predictable extension install paths (CVE-2026-54328) and project-local extensions loaded without approval (CVE-2026-54325) | Closed source — the public repo holds the issue tracker, plugins and changelog. Permission-system CVEs have been patched, e.g. CVE-2026-33068 (CVSS 4.0: 7.7), where a malicious repo could skip the workspace-trust dialog | Two CVEs published 1 Sep 2026: CVE-2026-19591 (8.8), a command-safety parser misclassification that skipped an approval prompt, and CVE-2026-19592 (7.3), a repo-local Git helper that ran outside the sandbox | Supply chain: in Feb 2026 a prompt-injected issue-triage workflow leaked an npm token, and an unauthorized cline@2.3.0 (postinstall installed OpenClaw) was live ~8 hours. YOLO mode and the CLI's auto-approve default remove approval checks. Also CVE-2026-44211 (CVSS 9.6), a cross-origin WebSocket hijack in the Kanban server | Newest and least battle-tested of the eight; early-stage (v0.3.x), not an audited security product — the harness reduces blast radius, it doesn't eliminate it |
All 11 layers ship as drawable, composable canvas nodes — Reasoning, a tiered Control State, Verification, and a formal Output & Reviewer Pass included, the layers thinnest across all seven agents above. See the full architecture →
Pi's empty cells are a design position, not an oversight. Its author argues that with four tools (read, write, edit, bash) and a system prompt under 1,000 tokens, frontier models already know how to be coding agents. Pi lists MCP, sub-agents, permission popups, plan mode, built-in to-dos and background bash as deliberately omitted, and ships extensions, skills, prompt templates and installable packages instead. On permissions its stance is explicit: it runs unrestricted by default and treats prompts as
"security theater"
unless the agent is sandboxed. That is the opposite bet to a harness that gates actions — so the coverage table above measures Pi against a taxonomy it rejects, not against its own goals.
On its own terms Pi does well. In Databricks' July 2026 benchmark on its internal multi-million-line codebase, Pi with Claude Opus 4.8 sent about 3x less context per turn than Claude Code or Codex (both covered above) and cost less at comparable quality; Databricks calls the benchmark non-comprehensive. Aielia has no comparable coding benchmark — its feature evaluation is being rebuilt and its earlier results are offline — so this comparison is about governance, not which agent is cheaper or better at coding.
Every layer above wraps around a loop that, on its own, isn't a differentiator — a point the projects' architecture docs and independent write-ups broadly agree on. Mapped against the same seven-stage loop, the pattern is clear. Pi's public docs describe its tools and extension points rather than a step-level loop, so its column is read from those, and no step list is given for it below. Claude Code documents its loop as three phases rather than numbered steps, and Codex's step list below is read from OpenAI's own write-up of its loop plus its docs.
Scroll the table sideways — seven columns.
| Loop stage | Hermes Agent | Kilo Code | OpenClaw | Pi | Claude Code | Codex CLI | Cline |
|---|---|---|---|---|---|---|---|
| 1. Perceive | ◐Session history + static profile/identity config; no typed beliefs | ◐Static AGENTS.md / rules files (Memory Bank is deprecated) | ◐Static bootstrap/persona files | ◐AGENTS.md / SYSTEM.md context files plus the session tree | ◐CLAUDE.md files, auto memory and a git-state snapshot, plus files it searches | ◐AGENTS.md plus the full conversation history re-sent on every request | ◐Rules files (optionally path-conditional), @-mentions and open files |
| 2. Reason | ○Not described | ○Not described | ○Not described | ○Not described | ○Not described | ○Not described | ○Not described |
| 3. Decide | ◐Dangerous-command approval gate; no risk-tiering | ◐Per-tool allow / ask / deny approval; no risk-tiering | ◐Exec approvals (allowlist / ask / auto); no risk-tiering | ○Explicitly omitted — no permission prompts by design | ◐Deny → ask → allow rules plus an auto-mode classifier | ◐Sandbox mode × approval policy, optional auto-review subagent | ◐Per-category auto-approve; the model flags requires_approval |
| 4. Act | ◐Concurrent dispatch; binary dangerous-command gate only | ●Git snapshot at each model-call step | ◐Broad device/OS action surface | ◐4 core tools (read / write / edit / bash); unrestricted by default | ●File snapshot before every edit | ◐Tool calls run inside an OS-enforced sandbox | ●Shadow-git checkpoint after every tool use |
| 5. Verify | ○Not described | ◐Informal check + diff-repair | ○Cosmetic reply shaping only | ○Not described | ◐"Verify results" phase; on-demand /code-review; Stop hooks can force another pass | ◐On-demand /review; Stop hook can force another pass | ◐Compiler and linter feedback |
| 6. Recover | ◐Fallback provider, iteration budget | ◐Checkpoint rollback, diff-repair | ◐Auto-compaction retry + provider fallback | ◐Session-tree rewind and automatic compaction | ◐Checkpoint rewind, resume / fork, auto-compaction | ◐Session resume, automatic compaction; no file-revert | ◐Checkpoint restore, auto-compact |
| 7. Learn & Repeat | ●Persistent memory + self-improving skills | ◐AGENTS.md persists project context (Memory Bank deprecated) | ◐Skill Workshop drafts and patches skills from experience (auto by default) | ○Not described (extensions can edit the agent itself) | ◐Auto memory notes from your corrections and preferences | ◐Opt-in memories distilled from past chats (off by default) | ○Not described |
Reading the pattern: Execution and Memory/Learning are the only two stages where any agent reaches full (●) coverage — the checkpoints of Kilo, Claude Code and Cline, and Hermes's persistent memory, respectively. Reasoning is not described by any of the seven, and Pi's column is read from its tools and extension points rather than a published loop. This isn't a knock on any one project; it's a reasonably faithful reading of what's in their own public documentation, and it's exactly the gap the eleven-layer harness above is meant to close.
{runId, acceptedAt} immediatelyprevious_response_id.clinerules, AGENTS.md; path-conditional ones activate on matching files) and @-mentioned contextPi's docs describe four core tools (read, write, edit, bash), JSONL session trees, compaction and extension hooks, not a numbered step-level loop, so none is reconstructed here.
Its loop is a refined ReAct implementation — Observation → Reasoning → Action — the same call-model / run-tools / append-results shape Claude Code and most agents use. Nous's own framing puts the differentiation in memory and self-improving skills, not the loop.
Kilo began as a fork of Roo Code, itself a fork of Cline; since April 2026 its VS Code extension and CLI run on an OpenCode-based core. Either way the loop is a standard tool-calling loop, and Kilo's stated edge is pricing, model breadth and reliability plumbing, not the loop.
Independent architecture write-ups describe OpenClaw's runtime as "just a ReAct loop, a filesystem, sessions, and plain API calls" — the simplicity is deliberate, and by broad agreement the loop is not where its traction comes from.
Pi doesn't pretend the loop is special. It keeps it as small as it can, with four tools and a system prompt under 1,000 tokens, and treats permissions, planning and sub-agents as optional extensions rather than core.
Its own docs say the agentic loop is "powered by two components: models that reason and tools that act," and call the layer around the model that supplies tools and manages context "the agentic harness." The loop is three phases — gather context, take action, verify results — and the differentiation is the machinery around it: permissions, checkpoints, hooks, subagents and memory.
OpenAI describes the agent loop as "the core logic in Codex CLI that is responsible for orchestrating the interaction between the user, the model, and the tools the model invokes." It is a plain call-model / run-tool / append-result cycle; what Codex invests in around it is an OS-level sandbox and an approval policy.
Cline's SDK describes its loop as "model request → tool calls, if any → tool results → repeat until complete." Its distinctive layer is human-in-the-loop control — Plan / Act modes, per-category approvals and a checkpoint after every tool use — and the loop itself is the commodity part.
They're seven of the most-used AI agents running today — six open-source (Codex CLI, Cline, Hermes, Kilo Code, OpenClaw, Pi) and Claude Code, which is proprietary — each with public documentation detailed enough to map against a common harness taxonomy. Claude Code, Codex CLI, Cline, Hermes, Kilo Code and OpenClaw run a conventional tool-calling loop, per the projects' own docs and independent analyses. Pi is included as the deliberately minimal counterexample: it leaves most governance layers out on purpose.
None covers more than 2 of 11 canonical layers at full strength. Hermes leads on Memory and Learning. Kilo Code, Claude Code and Cline reach full coverage on Execution only, all through reversible edits (checkpoints). Codex CLI covers none at full strength; its main strength is a default OS-enforced sandbox (Seatbelt on macOS, bubblewrap on Linux). Both it and Claude Code had permission-bypass CVEs disclosed in 2026 (Codex, Claude Code). Pi covers none at full strength by design, yet was the most cost-efficient harness in Databricks' July 2026 benchmark. OpenClaw leads on reach, but its broad action surface lines up with its documented security incidents — critical CVEs including a 9.9-severity privilege-escalation-to-RCE flaw and an 8.8 one-click RCE, plus a marketplace campaign that disclosed 341 malicious skills in a single sweep.
The canonical architecture used across Build A Harness: Caller State, World Model, Reasoning, Control State, Planning, Execution, Verification, Recovery, Memory, Learning, and Output & Reviewer Pass. Full definition →
Build A Harness implements all 11 layers as drawable, composable canvas nodes — so you can build the layers these seven agents leave thin (Reasoning, a tiered Control State, Verification, Output & Reviewer Pass) without writing that infrastructure yourself.
This comparison draws on each project's own architecture docs and independent reporting. Claude Code, Codex CLI and Cline were researched on 6 Oct 2026; the other four were last checked on 24 Sep 2026, apart from star counts, release versions and token volumes, which were refreshed on 6 Oct. Token-volume figures are from OpenRouter's global app rankings — those numbers move daily and will drift from this snapshot. The Known-risk row cites example vulnerabilities, not counts: advisory totals depend on how each project discloses, so they aren't comparable across projects.
The layer-by-layer coverage mapping above is our own reading of each project's public documentation against Build A Harness's canonical taxonomy, not an official audit or a claim endorsed by any of the seven projects.
The mapping above is a taxonomy exercise unless something actually ships those layers. Aielia — Build A Harness's own assistant — wires the 11-layer harness into every non-trivial turn (Hypothesis, Diagnostics and experience learning are dormant or opt-in by default), and is the same class of tool as the seven above: an everyday chat/coding assistant you install and talk to.
Honest caveats: Aielia is the newest of the eight and the least battle-tested. It is an early-stage project (package v0.3.x), not an audited security product — the harness reduces the blast radius of a bad decision, it does not eliminate it. Apache 2.0, provided as-is.
Reasoning (hypothesis generation opt-in), a tiered Control State, Verification (2 of 9 checks return a result in chat), and a formal Output & Reviewer Pass — the layers where these agents are thinnest — ship in Aielia today, and as drawable canvas nodes if you want to build your own. Apache 2.0.