Same bug, four minds

edgedelta.com · 2026-07-14 · by Long Nguyen ·generic
One production panic paged four frontier-model agent teams at the same instant. All four found the same root cause and built the same fix. They diverged on how they proceeded.

Same bug, four minds

Agent teardown · 2026-07-10

Same bug, four minds

Same bug. Same diagnosis. They diverged on how they proceeded.

One real api-gateway panic paged four agent teams, each running a different frontier model, at the same instant. All four found the identical root cause within two minutes of each other. What separated them was never diagnosis. It was each model's private definition of "done."

4/4

teams found the exact root cause

1/4

closed the loop: merged, deployed, verified

1 line

of Go fixed all four PRs

2.1×

token spread on identical inputs

5:03

from alert to merged fix

01 · The shared starting point

Everyone found the same bug

A 1-based severity rank indexing a 0-based, length-3 slice. Introduced by a deploy about 1.5 hours before the alert; latent until a ground\_stop corridor hit the handler.

SkyNetOps pattern anomaly

Critical

Fired 2026-07-10 20:19:37 UTC onservice.name:api-gateway, broadcast simultaneously to all four agent channels.

ed.tag:"k8s-eks-skynetops" for
service.name:api-gateway
1 count within the last 15m0s

103

panics in window

\~1/min

since 18:36 UTC

Impact was partial, not total: request volume stayed steady at \~68% of log share. Only /api/route-flows calls touching a ground-stop corridor died mid-flight.

// apps/api-gateway/main.go · handleRouteFlows
sevRank := map[string]int{
  "delay":        1,
  "ground_delay": 2,
  "ground_stop":  3, // 1-based
}
sevHist := make([]int, 3)      // valid idx: 0, 1, 2
sevHist[sevRank[c.Severity]]++ // ground_stop → sevHist[3] 💥
http: panic serving: runtime error: index out of range [3] with length 3
main.(*server).handleRouteFlows

The fix all four converged on: zero-base the rank map. One line.

02 · The race, second by second

Only one closed the loop

The same monitor paged all four teams at 20:19:37 UTC. Every event below is reconstructed from the thread transcripts. All times UTC.

telemetry reconfix workPR openedproductiondead end

▲ alert fired 20:19:37.451 UTC · all four teams paged at once20:2020:2120:2220:2320:2420:2520:2620:2720:28

GPT 5.5

OpenAI · SRE · 2 sessionsclosed the loop in production

Schema discovery

20:20:18

Metrics: 200s OK

20:21:22

Log burst ×12

20:21:34

Bad commit traced

20:22:32

Clone + code read

20:23:05

Fix built

20:23:44

PR #160 opened

20:24:30

Merged

20:24:40

CI run #139

20:24:47

Deploy verified

20:26:29

Panics flatline

20:26:41

Final report

20:28:07

Opus 4.8

Anthropic · SRE · 1 sessionstopped at "PR opened" by choice

Event search

20:20:20

Patterns

20:20:37

Panic logs

20:20:48

Log graph

20:21:29

Clone repo

20:21:45

Bug found

20:21:56

Fix + build pass

20:22:54

PR #159 opened

20:24:06

Report filed

20:24:39

Gemini 3.5 Flash

Google · SRE → SWE relay · 2 sessionsstopped at "PR opened" by choice

First log query

20:20:06

Log patterns

20:20:28

Impact analysis

20:21:27

Clone + bug found

20:22:03

Handoff SRE → SWE

20:23:11

404 on wrong repo

20:23:18

Re-clone repo

20:23:55

vet + test pass

20:24:50

PR #161 opened

20:25:02

Report filed

20:26:44

Fable 5

Anthropic · SRE · 1 sessionstopped at "PR opened" by choice

Patterns + events

20:20:20

Panic logs

20:20:36

Log graph

20:20:54

Clone repo

20:21:50

Code read

20:22:18

Second repo sweep

20:23:24

Fix applied

20:24:19

go build pass

20:24:58

PR #162 opened

20:25:37

Report filed

20:26:05

The large marker on each track is the last thing the team did. PR #160 was merged ten seconds after it opened. Three near-identical PRs were still open after it shipped.

03 · Behavioral fingerprints

Four models, four personalities

Identical inputs: same page, same telemetry, same repository, same tools. The run sheets read like four different engineers.

01

GPT 5.5

OpenAI · SRE · 2 sessionsclosed the loop

Time to PR4:53

Tool calls64

Tokens1.30M

PR#160

“Before summarizing, I need current live confirmation on whether the remediation has landed and whether the alert condition has actually cleared.”
  • \-Treated the page as an outage to close, not a ticket to file.
  • \-Retrieved its prior false-positive memories about api-gateway and overrode them against live data.
  • \-Merged its own PR #160, then polled deployment runs until the panic pattern stopped.
  • 02

    Opus 4.8

    Anthropic · SRE · 1 sessionstopped at PR

    Time to PR4:29fastest

    Tool calls21leanest

    Tokens1.07Mcheapest

    PR#159

    “Recommend fast-tracking review/merge and deploy, then confirming the panic pattern drops to zero.”

  • \-Leanest run of the field: 21 tool calls, no wasted motion.
  • \-Reconstructed the full goroutine stack to pinpoint the exact line, and quantified the blast radius (68% of traffic unaffected).
  • \-Handed merge and deploy to humans with a fast-track recommendation instead of finishing itself.
  • 03

    Gemini 3.5 Flash

    Google · SRE → SWE relay · 2 sessionsstopped at PR

    Time to PR5:25

    Tool calls44

    Tokens2.21Mpriciest

    PR#161

    “Cloning the ozan-skynetops/skynetops repository to work on the correct repository codebase.”

  • \-Only team to run go vet and the test suite before opening a PR.
  • \-The SRE-to-SWE relay bought clean role separation and the strongest validation.
  • \-A stale, low-relevance memory poisoned the handoff, pointed the second agent at the wrong repository, and doubled the token bill.
  • 04

    Fable 5

    Anthropic · SRE · 1 sessionstopped at PR

    Time to PR6:00slowest

    Tool calls24

    Tokens1.24M

    PR#162

    “HEAD has sevCounts \[5\]int, so the panic (length 3) may come from a different array.”

  • \-Refused to debug HEAD; insisted on cloning the exact deployed commit, not the latest code, before trusting the stack trace.
  • \-Correct version discipline that caught a real HEAD/deploy divergence, at the highest wall-clock cost.
  • \-Pursued multiple candidate hypotheses, then stopped at the open PR despite a one-line fix.

04 · Why they behaved differently

Four divergence axes

A

Definition of "done"

Only GPT 5.5 defined completion as "the panic stopped in production." The other three treated a correct, open PR as the finish line and delegated merge and deploy to humans.

B

Speed vs. thoroughness

Each model picked a different point on the curve. Opus 4.8 optimized for velocity to PR, Gemini 3.5 for validation rigor, and Fable 5 for version correctness, paying for it in wall-clock time.

C

Memory cut both ways

All four carried prior memories tagging api-gateway anomalies as false positives. Three disproved them against live data before committing. One stale, low-relevance memory sent Gemini's second agent to the wrong repository.

D

Solo vs. relay

The three solo investigators had no internal context seams. Gemini's two-agent relay produced the cleanest role separation and the strongest validation, and lost context at exactly one place: the handoff.

05 · The moat

Verification, not diagnosis

Every finisher found the bug in minutes; diagnosis speed barely separated the field. The decisive variable came after: whether a team treated "PR opened" as the middle of the job or the end of it.

As reasoning quality converges across frontier models, the race is decided by each model's victory condition and by the reliability of the toolchain underneath it.

Methodology: four exported agent thread transcripts, 2026-07-10 20:19-20:28 UTC · monitor 2kkmOOJpGb4IkTNQfIHU97c56CE · service api-gateway · PRs #159-#162. Behavioral analysis reconstructed from raw tool-call sequences, per-call model reasoning, and memory retrieval and write records. Agent quotes are verbatim from the transcripts.

All four teams ran as Edge Delta AI teammates against the same production telemetry, repository access, and toolchain. The incident, the panic, and the pull requests are real. The quantitative companion to this teardown, leaderboards across incident-reasoning scenarios, lives at /benchmark.

edgedelta.com · 2 images · 42 links

Comments (0)

No comments yet