Learn AI Engineering
Lessons
← Loop Engineering

Worktrees: Isolating Parallel Runs, and What They Don't Isolate

A worktree gives every parallel run its own files, its own HEAD and its own state file, but not its own merge, so a green run still has to be rebased and rerun before it opens a pull request.

Last lesson ended on a problem: two fixes, and only one working copy. The obvious answer is to give each run its own copy. Does that fix it? Half of it. This lesson is about both halves.

What actually collides

"They share the repo" is too vague. With one checkout and two agents, three specific things collide.

  1. The files. Both runs edit one tree at the same time. Every test run either of them starts includes the other's half-finished edit, so a green run proves nothing and a red one blames the wrong change.
  2. The branch. One checkout has one HEAD. Both fixes commit onto whatever is checked out, which here is master: exactly where Lesson 2.1's command said not to push.
  3. The state file. Lesson 1.3's .loop/state.json lives in the checkout. One checkout means one state file, with two sessions writing to it.

The third one quietly breaks your ceiling. The hook from Lesson 1.2 resets its count when the session_id changes. With two sessions alternating, each one's first call looks like a new session, so they take turns wiping each other's count. The PreToolUse hook still fires on every call. It just never reaches the cap.

One worktree per run

Give each run its own worktree. It's two commands:

$ git worktree add ../loopdemo-pricing -b fix/apply-discount
$ git worktree add ../loopdemo-flaky -b chore/quarantine-retry
$ git worktree list
D:/github/loop-engineering-demo  9f2c1ab [master]
D:/github/loopdemo-pricing     9f2c1ab [fix/apply-discount]
D:/github/loopdemo-flaky       9f2c1ab [chore/quarantine-retry]

Each command gives you a new directory on its own branch. Underneath, there is still one .git: one object store, one set of refs. A worktree is a second checkout, not a second clone, so it's cheap and takes seconds to create. To undo one, use git worktree remove.

Then start the loop inside the new worktree:

$ cd ../loopdemo-pricing
# .loop/guard.py is already here — it's committed. .loop/state.json is not.
$ claude
/loop 1d "Check the repo for newly failing tests. Rerun each 3x to rule out flakiness. If a real failure remains, fix it and keep going until the suite passes, then push the branch (not master)."

That last line is Lesson 2.1's command, word for word. This primitive adds no words to it at all. What changed is the directory it runs in.

Per worktreeWhat you get
SharedOne .git: one object store, one set of refs.
Its ownWorking copy and HEAD. One branch per directory, and git refuses to check the same branch out twice. That's a guard rail, not a nuisance.
Its own.loop/state.json, so each run gets a fresh counter.

Only tracked files come across

A worktree takes tracked files only. Lesson 1.3 committed the scripts and kept the state file out of git, so the guard comes across and the counter doesn't. For the state file, that's what you want. The same rule means no .venv and no .env either.

It bites if the guard isn't committed. An untracked .loop/guard.py doesn't come across. If .claude/settings.json is committed, the hook fires, finds no script, and blocks every call. If the hook is in settings.local.json, which is gitignored by default, the worktree has no cap at all.

Commit the scripts and the settings. Ignore only the state.

What they don't isolate: the merge

In the morning, both branches are green. It looks like two clean pull requests.

But both runs touched the same file, pyproject.toml, and neither knew the other existed.

Case one: a textual collision, which git can see. The first pull request merges cleanly. The second now conflicts with a change it never saw. That's loud and annoying, but safe: nothing lands until someone resolves it. A file-overlap check catches this one.

Case two: a semantic collision, which git cannot see. Take a different pair of runs with no shared file at all. One changes what a function returns; the other adds a caller in a file the first never touched. There's zero path overlap, so a file-overlap check clears them both. Both diffs apply with no conflict, and the suite goes red on master anyway. The merged tree is a combination neither run ever executed. This is the one that reaches your users, and only a rerun sees it.

Green in a worktree is a measurement of a tree nobody will ship.

Two ways out

Neither is clever. Pick one before you run two loops, not after.

ApproachWhat you doCostsCatchesMisses
Detect, then serializeEach run declares the files it expects to touch before it starts. Intersect the sets; if two runs overlap, they run one after the other in one lane.A plan stepShared configs and fixturesAnything the run didn't predict
Rebase, then rerunImmediately before opening the pull request, rebase onto current master and run the suite on the rebased tree.One more suite runBoth collision classesNothing that lands after it

"Immediately" is doing real work in that sentence. A green run starts aging the moment another branch lands.

Which to use? Rebase-and-rerun by default, because a predicted file list is a guess and a rerun is a measurement. Serialize only for the handful of files you already know every change touches. And if your loop can't rerun a suite unattended, it was never ready to open pull requests unattended either.

The honest scoreboard

Isolated by a worktreeStill shared, still yours to handle
The working copy: no run tests another's half-finished edit.The merge: both collision classes live here, and only a rerun sees the second one.
The branch: one HEAD per directory, so nothing lands on master by accident.The remote: one master, one pull request queue, one review backlog.
The state file, and so the iteration cap that depends on it now counts one run at a time.Anything outside the tree: one CI runner, one test database, one rate-limited API key. Yours will differ.
Cheap: one object store, seconds to create, git worktree remove to undo.Two windows in the same folder.

After this lesson

You will be able to:

  • name the three things that collide when two runs share one checkout: the files, the branch, and the state file;
  • explain why a shared state file silently stops the iteration cap from ever tripping;
  • give each run its own worktree, and know why the guard and settings must be committed for the cap to come with it;
  • tell a textual merge collision from a semantic one, and say which check catches each;
  • choose between detect-and-serialize and rebase-and-rerun before running loops in parallel.

What comes next

Every run still starts from nothing: which branch names you use, which command runs the suite, which files it must never touch. Four runs means four re-derivations of the same house rules. Lesson 2.3, "Skills: Codifying Project Knowledge the Loop Won't Relearn", writes them down in one place.

Companion article

Every hook, script, and command in this course, written up and executed: How to build an agent loop in Claude Code