Attention was always the bottleneck

In April, OpenAI open-sourced Symphony, a spec for orchestrating fleets of coding agents off an issue tracker. Around the same time, Peter Steinberger — who burned through 603 billion tokens in a month running ~100 agents to build OpenClaw — said the quiet part out loud: first his constraint was tokens, then CPU, and now it's attention.

I want to gently point out: it was always attention. Tokens and compute were just the constraints you hit first, before you had enough agents running to notice the real one.

I know because I started building a tool around that exact premise in February, watched the platform absorb most of it over the following months, and shut the service down this week. This post is the postmortem — and the one idea from it that I think matters more now, not less.

The thing I built

HUD started in mid-February as version zero: a board over tmux panes, because that's what existed. When codex app-server shipped, it became the transport, and the project evolved fast — v0.1 by March 1, a v2 collaboration flow two days later, v3 cut on March 6. The mental model stayed constant through every rewrite: a kanban board that happened to be alive. Each runtime was a card. Each card showed a synopsis — what the agent was doing, its progress, what it needed from me — and clicking a card dropped you into the real conversation. One orchestrator conversation per runtime, plan mode, steering, halt, cleanup. About five thousand lines of Python and JavaScript, plus a Tron-rain canvas behind the fleet — black field, point-emitted neon — that I will defend to anyone as the correct aesthetic for watching a fleet of agents work at 2 a.m.

The HUD console: a dark interface with a runtime rail on the left showing two agent cards with READY status chips and synopsis text, against a black field of falling neon glyphs in cyan, blue, and violet. — **FIG. 1** — The console, resurrected for one last screenshot in July. The runtime rail carries the fleet: each card is a synopsis (state chip, progress line, working directory, age), not a transcript. The rain is the actual rain.

The premise was never "I need more agents." It was: I can only attend to a few things at once, so the system's job is to decide what deserves my attention and present it in a form I can act on in seconds. A card that says Progress: migrated 14 of 30 call sites — Next: needs decision on the auth shim is worth more than a thousand lines of streaming transcript.

The cards existed because the agents wouldn't stop — by design. Before goal-directed loops were a first-class feature, before /goal was a thing people used, some very early Codex build briefly supported JSON hooks, and I saw an opening: hook the end of the turn and inject a prompt — keep working until the goal is met. That's it. That's the whole mechanism. It was ratchet, and it worked: agents that ran indefinitely, each loop writing its own ledger as it went. And once your agents literally don't halt on their own, the question of what the human should look at stops being theoretical.

hooks.json — reconstructed

{
  "hooks": [
    {
      "event": "turn.completed",
      "action": "inject_prompt",
      "prompt": "Read EXPERIMENTS.md. If the goal is not met, pick the
        next hypothesis, run it, and record the outcome under a new
        numbered experiment. Update the Active Experiment heartbeat.
        Do not stop."
    }
  ]
}

FIG. 2 — The entire loop mechanism, reconstructed from memory (the original was lost to a spring cleanup — a lesson the rest of this post will make ironic). One event, one injected prompt, infinite agent.

The HUD autoloop rail listing experiment loops with status chips: OBSERVED, WAITING_ON_OPERATOR, and COMPLETED, each with a working directory and age, against the neon rain. — **FIG. 3** — The autoloop rail, July 2026: forty-five loop ledgers across projects, each one an `EXPERIMENTS.md` on disk. Note the states — `OBSERVED`, `WAITING_ON_OPERATOR`, `COMPLETED` — still legible four months after the last loop ran. Files outlive services.

What the dogfood log actually taught me

I kept a running log of findings from operating it daily. Rereading it now, the entries sort cleanly into two piles.

Pile one: plumbing pain. Threads going stale after the app-server restarted. Orphaned in-progress turns after a crash. Resume and reattach flows. Client refresh storms that could time out the health check. Keeping transcripts fresh across tabs. Probably 70% of the effort went here — into keeping conversations alive and current, which is exactly the problem Steinberger's "keeping threads moving" era was about, and exactly what the platform has since stabilized underneath everyone. Codex's app server matured; Symphony now handles dispatch, restarts, and stalls at the orchestration layer. That code of mine is dead weight now, and it should be.

A HUD runtime detail view: a state strip showing Ready / Answer / a pulse line, and a green-on-black live feed where the orchestrator describes fixing a resume glitch, with a Show Internals toggle and a composer below. — **FIG. 4** — A runtime view. State strip up top (`State / Phase / Pulse`), final answers in the feed, sixteen internal items folded behind `SHOW INTERNALS`. The last transcript in the fleet is, fittingly, the agent describing its own fix to a resume glitch. Bottom left: an authentic `Reconnect failed` — pile one, forever.

Pile two: attention design. These findings still hold, and they were all versions of one discovery:

You cannot project an agent's status by scraping its transcript. The agent has to author its status, as a first-class artifact, in a format you can hold it to.

My first cards summarized transcripts heuristically — grab the last completed answer, guess at a "mission" from the first user turn. The summaries were plausible and constantly, subtly wrong. Plausible-but-unverifiable is the most expensive kind of wrong, because it spends your attention and your trust. The fix that stuck was a contract: every runtime maintains a synopsis.md with fixed frontmatter — title, progress, next, attention, updated_at — and the UI renders only that. Long-running autonomous loops got a sibling contract, an EXPERIMENTS.md with one mutable "active experiment" heartbeat and an append-only ledger of numbered, immutable checkpoints: hypothesis, outcome, decision, next hypothesis.

EXPERIMENTS.md — a real ledger, March 2026

## Experiment 18
title: Fresh Saturday-only rebuild from rollback baseline
status: completed
updated_at: 2026-03-16T20:38:00Z

## Hypothesis
- Starting from the clean rollback baseline and filling only the
  genuinely open Saturday cells from exact `GetAvailableStaffByTask`
  pools should remove the prior class drift.

## Outcome
- Harvested and assembled `45` open Saturday hidden-endpoint
  responses into `artifacts/.../fresh_saturday_open_live_availability.json`.
- Fixed `_validate_live_availability_coverage` so occupied seed slots
  with external staff are preserved, then added regression coverage
  in `tests/test_call_soft_avoids.py`.
- Built a Saturday-only exact draft at
  `artifacts/.../proposed_call_schedule_manifest.json`.

FIG. 5 — Checkpoint 18 of a real loop (a hospital scheduling task), verbatim from disk. Every claim points at an artifact that either exists or doesn't. This is what an agent status report should look like.

Same shape everywhere: the agent writes a small, structured, falsifiable claim about its own state, and the human attends to the claim — not the stream.

Too early, and what "too early" buys you

HUD was too early twice over. I was forcing autonomous loops with hooks before /goal made durable objectives a native feature, and I was hand-building thread stabilization months before the platform shipped its own. That second one is the part I'd caution anyone against repeating — don't build the layer the platform vendor is visibly about to eat.

But being early bought me the weeks of operating experience that produced pile two. And here's what I think the current moment gets slightly wrong: Symphony's framing, and most of the discourse around it, treats attention as a supervision problem. Stop watching sessions, let the board dispatch work, review finished PRs. That's real — OpenAI's own engineers reported that three to five concurrent sessions was the ceiling before context switching broke them, and moving review to the end of the pipeline is how you get past it.

The harder question is what you're reviewing against. A finished PR from an agent you didn't watch is a pile of diffs plus the agent's own account of what it did. If that account is prose in a comment, you're back to plausible-but-unverifiable — just batched.

Shared surfaces of verifiability

So here's the thesis I'd extract from the wreckage, the one that I think gets more true as models get stronger:

The things you attend to have to live at shared surfaces of verifiability.

Unpacking each word, because they all pull weight:

Shared — the human and the agent read and write the same artifact. Not a dashboard the human sees and the agent doesn't know exists; not an internal scratchpad the agent keeps and the human never audits. When the surface is shared, the agent can be held to it — "your synopsis says done, but the checkpoint test isn't in the ledger" is a conversation you can have.

Surfaces — small, fixed-format, glanceable. A frontmatter block, a checkpoint entry, a ticket state, a diff. Attention is spent in seconds-per-glance; the surface has to be sized for that.

Verifiability — every claim on the surface is checkable against something the agent can't vibe into existence: a commit, a passing test, a diff, a file that exists. Progress: 14 of 30 call sites is verifiable. Making good progress on the migration! is not.

This is why I've become convinced the substrate is git, and that agent workflows need to become git-native rather than session-native. A commit is a checkpoint that can't be retroactively reworded. A diff is a status report that can't exaggerate. An EXPERIMENTS.md that's committed at every checkpoint is an audit log with tamper-evidence for free. Symphony gets partway there by making the issue tracker the control plane — the ticket is a shared surface — but tickets record intent and state, not evidence. The repo records evidence. The strongest version of this pattern makes every unit of agent work terminate in a commit whose message, diff, and updated synopsis are the entire interface the human attends to.

Session state, by contrast — the thing I spent 70% of HUD keeping fresh — is none of these things. It's unshared (trapped in a process), unbounded (a transcript, not a surface), and unverifiable (prose). The lesson wasn't "manage sessions better." It was "stop making sessions the thing you attend to."

What survives

I shut down the HUD service this week. What I'm keeping is small: a 230-line parser for the experiments contract, two Markdown specs, and a page of operating rules (fail closed on unparseable contracts; never leave a command in a synthetic running state; every cleanup emits a reflection note). The next version won't be a console that mirrors live threads. It'll be closer to nothing: agents on branches, committing structured checkpoints, and a thin view over the repo that renders the synopses. The fleet dashboard I hand-built in March collapses into git log plus a hundred lines of rendering.

Models will keep getting better, and each generation makes the transcripts more plausible — which makes unverifiable status more dangerous, not less. Your attention was always the bottleneck. The fix isn't watching less. It's making everything you watch checkable.

Sources:
OpenAI — An open-source spec for Codex orchestration: Symphony
InfoQ — OpenAI open-sources Symphony
Better Stack — Orchestrating AI coding agents with an issue tracker
Tom's Hardware — OpenClaw creator's $1.3M token month
Peter Steinberger — "My attention is the bottleneck"

Abstraction Labs — think hard