six lessons · measured, dated, cited
Six lessons I measured
None of the six below is a hypothetical risk. Each has a date, a count of how many times it was
measured, and a file carrying the rule that came out of it. I ordered them by the stage they
fell out of rather than by severity, because what matters is where the lifecycle keeps leaking.
The part worth reading is the fix and what it costs. None of the fixes was free.
Three agents on one checkout lose work without throwing an error.
I ran three sessions against one checkout and nothing threw an error. An afternoon of work
vanished, and it took me another afternoon to find the cause.
I took the blunt fix: every spawned agent that can run state-changing git gets its own
checkout, including the ones that only read. The read-only exemption was the assumption I had
believed, and it was wrong. The cost is one more worktree on disk per Builder and a few seconds
of setup. I pay it, because that afternoon cost more.
Seven stages mark seven places where something already fell out.
The process started with two stages: the agent builds, I review. Review ran five to fourteen
rounds, and most of the later rounds went on clearing up what the earlier ones left behind. The
reviewer was not where it broke. It broke further upstream: decisions that had never been
settled went straight into code, and got settled at the most expensive point in the process.
I moved the loop to the front and cut what was left into seven named stages. The seven stages
are named so that every measured failure attaches to exactly one of them; without names, the
next failure lands somewhere nobody can point at. The cost is a longer pipeline, and three of
the seven are still empty because I have not measured a failure in them.
claim
The queue drained and nothing asked whether a slot was free
Measured on an adapted project running continuously by the owner's instruction: **after two merges, two of four Builder slots sat idle while twelve claimable tickets waited.** Nothing errored. Every step performed was performed correctly.
The router's loop was notification → verify → merge → report → wait, and no step in it asks how many Builders are working. …
measured once, one project — 2 of 4 slots idle after two merges, 12 claimable tickets waiting, nothing errored.
Twelve claimable tickets were waiting while two of four Builder slots sat idle, and nothing
errored. The router’s loop was notification, verify, merge, report, wait, and no step in it asks
how many Builders are working.
I fixed it by giving the frontier query a target rather than only a trigger: after every merge
the router has to ask how many more tickets it should claim, not only which ones are claimable.
The cost is a busier router that sometimes over-claims. I take that trade, because an idle slot
produces no signal at all.
brief
no failure measured here yet
build
TERMINAL:done means the turn ended, not that the work finished
Found by workspace-app-inception Thomas, measured three times on one pane in one session. A builder launches a long background process (a test suite, a build), ends its TURN while waiting for the completion notification, and the pane reads done. The watcher faithfully reports TERMINAL:done. …
measured three times, one pane, one session — the dispatcher nearly reported it abandoned before checking further.
A Builder started a long background process and ended its turn while waiting. The pane read
done, the watcher reported TERMINAL:done, and dispatch-ticket’s branch table said the
builder had finished. I came close to reporting that ticket abandoned while the Builder was
twenty minutes into honest work.
What saved this was not the protocol but the artifact contradicting itself: the modified file
contained one added comment and nothing else. I dropped my trust in pane status entirely after
that, and every done now has to read the diff before concluding anything. The cost is one
extra pass every time a ticket closes.
Builder stops after writing code but before committing — pane reads done, cleanup deletes the…
Measured in the field: 5 instances, 3 different Builder sessions, runtime claude, dispatched via herdr pane. Each time the Builder wrote substantial work (93-433 lines), pane status settled to done or idle, watcher returned TERMINAL:done, but git status on the worktree showed uncommitted changes.
The danger: Thomas's cleanup in dispatch-ticket runs git worktree remove, which silently deletes all uncommitted files. …
measured five times, three Builder sessions — 93 to 433 lines lost each time, never recovered.
The same word again, and this time it cost more. A Builder wrote the code and stopped before
committing, the pane settled to done, and cleanup ran git worktree remove over the top of
it. Five instances across three sessions, 93 to 433 lines each time, none recovered.
I put the guard at the dangerous step rather than relying on Builders to commit more carefully.
Cleanup reads git status on the worktree before removing it; anything dirty stops and goes
back to a person. The cost is orphaned worktrees piling up and the occasional manual sweep. Next
to losing a day of work, that price is cheap.
code-review
no failure measured here yet
simplify
no failure measured here yet
arm
Export step committed live secrets + buyer PII
Same-vendor correctness review PASSED it; cross-vendor caught it (P1) — the two lenses catch different classes. A value that touched a tracked file is burned — rotate it. Bound: .claude/rules/no-secrets-in-exports.md (always-on). CI secret-scan still TODO.
one measured incident — a same-vendor review passed it; a cross-vendor reviewer caught it before it shipped further.
An export step committed live secrets and buyer PII into a tracked file. The same-vendor
correctness review read it and passed it. The cross-vendor round caught it and filed it P1.
That is why the phase still ends with an arm round run on a different vendor, even though it
costs more money and more time. The two lenses catch different classes of defect, and the class
same-vendor misses is the expensive one to let through. A value that has touched a tracked file
is burned and has to be rotated. Downstream there is nothing cheaper.
merge
A tracker measured only against itself cannot detect its own drift
Reported upstream from an adapted project's own Thomas, via a handoff, rather than found in this package directly — the first entry with that provenance, recorded because the class it names is general and the reachability checks have no way to see a tracker's *content* going stale, only a document's.
Four tickets in that project sat claimed and in-progress with a live assignee **after their code had merged to the base branch**, the oldest by a full day. …
measured once, one project — 4 tickets stale, the oldest by a full day, before anyone noticed.
Four tickets sat in progress with a live assignee after their code had merged, the oldest by a
full day. Nothing errored: the merge ran, the frontier write-back after it did not, and no
artifact recorded the omission.
No tracker-only check catches this, because a wrong state is perfectly consistent with itself.
The tracker has to be reconciled against Git after every merge instead of confirming itself. The
cost is a reconcile step nobody enjoys running, and most of the time it finds nothing.
A blocking edge expresses order, not exclusion
The frontier asks which tickets have no open blocker and no assignee. It never asks what each ticket will WRITE, and nothing else did either. Two tickets went out together, correct by every rule the package stated. One existed to correct WIRE-CONTRACT.md; …
measured twice in one day — a naive merge would have reverted already-reviewed work.
Two tickets had no blocking edge between them, correct by every rule I had written, and both
edited the same three rows of one file. The first merged. The second was based on the commit
before that merge, produced two conflict blocks, and a merge in the wrong direction would have
reverted reviewed work with no signal.
One worktree per Builder solves the checkout collision and nothing else. It relocates the
collision to the merge, where it is found late and by hand. Tickets now declare a write-set, and
two tickets whose write-sets overlap are serialised even when nothing orders them. The cost is
fewer tickets running at once, which is the exact thing this whole system exists to increase.
Explore the full ledger →