Running an overnight team of AI coding agents

The setup

The rules are deliberately dull. At 02:00 every night a dispatcher wakes up and runs gh issue list --label ready-for-agent --state open. Whatever carries that one label is the night's work; nothing else exists as far as the run is concerned. It has until 06:00 to do as much of it as it can, then it stops and writes a report.

Each issue gets built in its own git worktree on its own branch, in isolation. Before an issue is allowed to count as done, it has to pass the same gates we'd demand of ourselves: the full test suite (pnpm -r test), a clean build (pnpm -r build), and, for anything touching infrastructure, a cdk synth so we can diff the CloudFormation before it ever goes near an account. As issues move, their label walks a small state machine: ready-for-agent → in-progress → ready-to-finish.

And then the most important rule, the one that makes the whole thing safe to run unattended:

That boundary is what turns "autonomous AI dev team" from a scary phrase into a boring, useful one. The agents do the typing; a person still owns the merge button. What follows are the nights that taught us the most.

The good night: eight green, and the scheduling was the interesting part

One night the queue had twelve issues in it. The report the next morning said 8 green, 0 red, 4 deferred, last branch finished at 04:10, well inside the cutoff. The eight that landed were real: a WebSocket auth posture change, a batch-DAG provider-routing refactor, an admin-reports feature spanning a new data model and an Angular page, mock-draft player notes, two standings-math surfaces, a self-service billing endpoint, and a deep-link into a CSV wizard. Each came with passing test counts and a short note on the judgment calls it made.

But the eight that got built weren't the interesting part of that report. The four that didn't were. The dispatcher had deferred them on purpose, and it explained itself:

Deferred up front (same-area / dependency — queued for a future
night, not blocked):

- #175 — depends on #62's activity-event store; #62's branch won't
  land until morning, so building on it tonight would stack unmerged
  branches. Next night after #62 lands.
- #170 — alert-topic relocation collides with #135's morning-batch
  changes in the same stack file.
- #161 — PlayersComponent inversion collides with #160 (same
  component). #160 first, #161 a later night.
- #128 — active-projection card freshness collides with #127 (same
  card component). #127 first (older).

It stopped at 04:10 with time to spare, not because it ran out of work, but because every remaining queued issue touched a file that one of the night's branches had already touched. Dispatching them would have manufactured merge conflicts for us to untangle over coffee. So it didn't. A system optimizing for "issues closed" would have plowed ahead and handed us a pile of conflicts; this one optimized for a clean morning, which is what we actually wanted.

That's the thing we'd tell anyone building one of these: the agent writing correct code is table stakes now. The leverage is in the orchestration around it, deciding what not to start, isolating work so parallel branches don't collide, and being honest in the handoff.

The honest night: the queue was empty

Two of the nights in our sample produced this report, near enough verbatim:

gh issue list --label ready-for-agent --state open
  returned no issues — the queue is empty. Nothing was dispatched.

0 green, 0 red, 0 skipped. No usage-limit stalls — the run ended
immediately because the queue was empty.

We're including these on purpose, because they're the part these stories usually leave out. An autonomous dev team is not a perpetual-motion machine that invents work. Ours does exactly what you point it at and nothing more. If nobody groomed the backlog and tagged issues ready-for-agent that day, the night is a no-op and the report says so in three lines. That's a feature. An agent that goes looking for things to do at 3 a.m. when the queue is empty is an agent that files busywork, or worse, "refactors" something nobody asked it to touch. Empty-in, empty-out is the correct behavior, and we'd rather read the boring report than trust the busy one.

The flip side showed up on the fuller nights too: the reports were candid about the machine itself. A test named content-factory/handler.test.ts kept flaking under full-suite CPU contention on multiple worktrees and passing in isolation. The agent flagged it as a recurring flake and suggested a timeout bump, rather than quietly re-running until green and pretending it never happened.

The uncanny night: the issue that had already fixed itself

One night the queue held a single issue, #238: "player pages link to zero rankings pages." Its acceptance criterion was literally "verify the link is present in the rendered HTML." The agent did the obvious thing before writing any code, it went and checked the live site:

$ curl -s https://rotoalpha.com/nfl/players/ja-marr-chase \
    | grep -oE 'href="/nfl/[^"]*"'
href="/nfl/players"
href="/nfl/rankings/12-team-half-ppr"          <-- the gate that
                                                   was silently false
href="/nfl/rankings/half-ppr-wide-receivers"   <-- also lit

The links were there. The rankings pages they pointed at returned 200. It sampled three more player pages; all fine. The bug the issue described was real when it was filed, and had since fixed itself, and the agent worked out how. The cross-linking logic is a render-time filter: a page only links to targets that already exist. The rankings pages had been missing because a model quota was exhausted when they were meant to generate. When the quota reset, the pages generated, and because the links are computed at render time, the next re-render repaired every player page for free. Nobody had looked at the bytes since. The issue predicted its own fix and then it quietly happened.

Here's the part we're proudest of, and it isn't a line of code. The agent did not close the issue. Closing issues and editing labels is outside what the overnight run is allowed to do, so it left #238 exactly as it found it, wrote up the curl evidence, and told us it looked closeable so a human could confirm and close it. It also noticed the real latent lesson, that a missing cross-link target degrades to silence, which is why the bug was invisible for the entire life of the content set, and recommended filing a proper observability follow-up rather than guessing at an unspecified one. Restraint, in other words. The most useful thing it did that night was correctly decide there was nothing to build.

The messy night: a subagent went off-script

It would be a dishonest post if every night went to plan. On the eight-green night, one of the agents misbehaved. Working a billing endpoint (self-service sport change, a Stripe-adjacent surface), the subagent spawned its own background agent and returned early, before its work was actually verified. The dispatcher caught the early return, corrected it with follow-up instructions, and made it finish and verify the work properly. But because a duplicated, half-supervised agent is exactly the kind of thing that produces subtly wrong code, that branch got called out in the report:

Process note: this subagent misbehaved twice (spawned its own
background agent and returned early); corrected via follow-up and
ultimately verified/committed. The diff got an extra review pass
because of the duplication risk — worth a closer look in the morning
than the other branches.

Note what the safety net actually was. It wasn't that the agent never misbehaves; it will. It's that the branch was still sitting unmerged in a worktree, flagged for extra human scrutiny, with nothing pushed and nothing deployed. The "no-push, human-lands-it" rule isn't bureaucracy. It's the thing that makes an occasional off-script agent a note in a report instead of an incident in production.

What we actually learned

A few weeks of these reports moved our thinking more than any single benchmark.

• The harness is the product, not the model. Every good decision above, deferring colliding issues, checking prod before building, refusing to close an issue it wasn't allowed to close, flagging a flake, was a decision about orchestration and guardrails. Swap in a better model and those decisions still have to be designed. That's where the engineering is.
• Isolation buys parallelism cheaply. One worktree per issue meant eight branches could be built in a night without stepping on each other, and the collisions that would have happened were predicted and scheduled around instead of discovered as conflicts.
• The read-only boundary is everything. Because the run can't push, merge, or deploy, a misbehaving agent is a review note, a "fixed in prod" surprise is a heads-up, and an empty queue is a three-line report. None of them are outages.
• Honesty in the handoff beats a green checkmark. The reports we trust most are the ones that say "this one is worth a closer look" and "the queue was empty." A summary that's always triumphant is a summary you have to double-check by hand, which defeats the point of it.

We're not going to tell you the overnight team replaced anyone or that it ships itself to production. It doesn't; it hands us a stack of candidate branches and a clear-eyed note about each one, and a human still does the landing. What it changed is the shape of the morning: we wake up to reviewed-and-reasoned diffs instead of a blank editor. On a good night that's eight features closer to launch. On an empty night it's an honest reminder to go groom the backlog.

RotoAlpha launches this season, and a lot of it was built exactly this way, at night, in worktrees, one labeled issue at a time. More field notes from that build, including the draft-strategy math and the backtests behind it, are at rotoalpha.com.

Kyle Twogood is the founder of Magrathea Software. He’s been building production software since 1997.