On the morning of March 29, 2024, a database engineer named Andres Freund sent an email to a security mailing list with the subject line “backdoor in upstream xz/liblzma leading to ssh server compromise.” He opened with a disclaimer: “I am not a security researcher, nor a reverse engineer.” He had been microbenchmarking PostgreSQL, and to reduce noise he went looking for stray load on the machine. He found sshd processes burning CPU on logins that were failing immediately. He profiled them, saw the time disappearing into a compression library with no symbols attached, and remembered an odd valgrind complaint from automated Postgres testing a few weeks earlier. The number that made him keep pulling: ssh logins that used to take 0.299 seconds were taking 0.807.

Half a second, on a machine he happened to be quieting down for an unrelated benchmark.

What the half second turned out to be was the most sophisticated supply-chain attack in the history of open source. An account named Jia Tan had spent three years becoming the co-maintainer of xz. First patch in October 2021. Sock-puppet accounts pressured the project’s exhausted solo maintainer through 2022. One wrote, “Progress will not happen until there is new maintainer.” By 2023, Jia Tan had full control of releases. In February and March of 2024, Jia Tan shipped xz 5.6.0 and 5.6.1 with a backdoor targeting sshd. CVE-2024-3094 scored a perfect 10.0. The payload was close to landing in major stable distribution releases.

Two details in the disclosure matter more than the drama. The build process, in the words of the CVE description, “extracts a prebuilt object file from a disguised test file.” The payload lived in the test fixtures, the directory nobody reads because it is only tests. And the line that activated it existed, as Freund noted, “solely in the distributed tarballs.” It was never in the git repository at all. Every commit was reviewable. The dangerous release artifact was not. Ordinary source review was looking at the wrong thing. It did not catch it, anywhere, for five weeks of exposure.

A measurement caught it. The most consequential code review of the decade was not a reading. It was a benchmark, run by someone with a memory of what normal looked like.

I need to be honest about why this story will not leave me alone. I have spent this year prescribing code review. In February I gave the standard advice: schedule dedicated review hours, cap PR sizes, add bots to clear the brush. In May I went further and argued that doubt is a paid position now. Fund reviewer roles. Calibrate the bot reviewers. Treat verification as an institution. I run my own version of that institution. My agents’ operating instructions contain a section titled “code review discipline.” It has three standing rules distilled from bug classes that review kept missing: where the audit row goes relative to the side effect, which status writes belong inside the failure handler, whether a guard added to one path reached its siblings. The newest rule exists because one of those classes shipped four separate P1 bugs in a single pull request. After review. With the checklist in the reviewer’s hands.

And earlier this month, a change in my own fleet crashlooped the first canary batch of its rollout. An agent wrote it. Review approved it. Promotion began. The wave halted itself. The pointer rolled back to a month-old known-good image. The whole event consumed none of anyone’s morning. The review had passed it. Batch one caught it.

This post is me conceding the pattern. Not that review is dying. That it is being found out. Modern code review kept the belief that it is a defect gate while shedding the function decades ago. The strange gift of AI-scale code generation is that the two can no longer impersonate each other.

The room was real

The belief has an honest origin. When Michael Fagan formalized inspection at IBM in his 1976 paper, it was unambiguously a defect gate. He was blunt about it: the inspection “is intended just to find errors!”, exclamation mark his. The process was heavy machinery: four roles, preparation against written checklists of known error types, sessions capped at two hours because attention decays. The economics were explicit. Fixing an error at inspection cost 10 to 100 times less than fixing the same error after release. And in its one carefully measured application study, the process caught 82 percent of the defects in the product, with zero surfacing in six months of production use.

The room worked because it was expensive: roughly ninety person-hours of structured attention per thousand lines. I told the story of what happened next in May. The room lightened into mailing-list patches, then into pull requests, then into a glance and a green checkmark. Reviewers stayed expensive while the volume of work grew. What I missed in May is that the lightening did not just compress the form. It swapped the function out and kept the label. The pull request compressed the room into four letters, and the compression was lossy. LGTM carries the 1976 authority with none of the 1976 mechanism. The mistake is treating the button as if it still means the room.

What the button actually does

We do not have to speculate about what pull-request review does. Researchers spent the 2010s measuring it, and the numbers never matched the story. When Microsoft watched its own reviewers work, finding defects was the reason everyone gave for reviewing. It was the fourth most common thing review actually produced, at 14 percent of comments. Most of those were “small logical low-level issues.” The researchers called relying on review for quality assurance “fraught.” The follow-up, two years later, dropped the hedge straight into its title: “Code Reviews Do Not Find Bugs.”

Google measured nine million of its own reviewed changes: the median change is 24 lines, seen by one reviewer, approved within hours. And review was never installed there as a defect gate in the first place. It was introduced “to force developers to write code that other developers could understand.” Of 44 Googlers asked about their most recent review, two said it had found a bug.

None of this is an indictment. Comprehension, norms, ownership, education: review manufactures real goods. They are goods for the team. The thing standing between a defect and production was always something else, and the industry’s own data said so before AI wrote a line. DORA found in 2019 that formal approval gates had no detectable relationship with change failure rates. None. Organizations using them were 2.6 times more likely to be low performers. What did correlate with stability is a list worth reading slowly: “continuous testing, continuous integration, and comprehensive monitoring and observability.”

The receipt is dated 2019. Approval felt like safety, so nobody cashed it.

The counterfactual fits in a single artifact. The commit that introduced Heartbleed, two years undetected in the most-eyeballed cryptography library on earth, carries the line “Reviewed by: steve” in its message. Steve was not the problem. Even Linus never claimed reading was the mechanism. Eric Raymond’s own footnote to Linus’s Law records his correction: “Somebody finds the problem, and somebody else understands it. And I’ll go on record as saying that finding it is the bigger challenge.” Finding is behavioral: a crash, a benchmark, a fuzzer, a monitor. Finding is what happened to xz.

The flood broke the alibi

At human production rates, the two functions of review could run on the same ritual: maintaining comprehension and gating defects. Both operated at the speed of one person explaining themselves to another. That is the hidden assumption agents broke. In January 2023, developers merged about 25 million pull requests a month on GitHub. The figure now tops 90 million. On June 17 of this year, GitHub shipped something that would have sounded absurd at any earlier point in the platform’s history: rate limits on pull requests. These are caps on how many PRs a contributor may have open, with agent-authored PRs explicitly counting against the cap. The world’s largest code host, rationing contributions. Their explanation is the whole argument in two sentences: “Creating a pull request has never been easier. Reviewing one still takes a human about as long as it ever did.”

I wrote in June that every saturated channel eventually has to choose between admission rules and pretending, and that the honest response to a flooded receiver is back-pressure on the producer. Seven days after that post went up, GitHub shipped producer back-pressure for pull requests. I take no credit and considerable comfort.

The telemetry agrees. Across 22,000 developers, median review time is up 441 percent at high AI adoption, and “31% more PRs are merging without any review” at all. DORA’s 2025 report has AI adoption finally correlating with higher throughput, and still with degraded delivery stability. So the gate is being run around at exactly the moment the traffic multiplied. Here is the tell: if review were what kept defects out of production, this would be a catastrophe. What the numbers show is a tax. A tax is what you would expect if the gate had been ceremonial for years and the real load was carried elsewhere.

Reading faster is still reading

The industry’s answer is to scale the reading: AI code review. The money believes. Well over a hundred million dollars of venture funding went into the category in 2025. The adoption is real too: GitHub’s Copilot reviewer now touches more than one in five code reviews on the platform. Some of what these tools flag is real. But this scales the instrument whose defect yield, when humans held it, was 14 percent. Three problems do not yield to a faster reader.

First, the reviewer now shares a prior with the author. A model reviewing its own family’s output is not a second opinion; it is the same opinion with different formatting, and LLM evaluators measurably favor their own generations. The blind spots correlate exactly where you need them not to.

Second, the verdicts are unscored. An approval that is never checked against what happened in production is a confidence number without a Brier loop. It is a feeling with a checkmark. OpenAI’s auto-review replaces synchronous human approval at the sandbox boundary, approves around 99 percent of what it sees, and says of itself that it “should not be treated as a guarantee of security.” Even the builders of the fastest readers locate safety somewhere other than the reading.

Third, reading has an adversarial floor. Ken Thompson marked it in 1984: “No amount of source-level verification or scrutiny will protect you from using untrusted code.” The eerier version is newer. In 2015 a commenter on Pieter Hintjens’ blog posed the exact xz scenario: a patient contributor building trust to land an obfuscated backdoor. Hintjens was open source’s great advocate of merging without review. His reply, nine years before Jia Tan: “Then careful code review won’t help you. Mallory simply has to gain enough trust to become a maintainer, which is a matter of how, not if.”

Armin Ronacher, arguing in “The Final Bottleneck” that machine review must pre-clear machine code, lands honestly on the residue: “for as long as I carry responsibilities and am accountable, this will remain true.” He is the bottleneck. That is the right question left standing: if accountability is what humans still hold, what should we attach it to? For seventy years the answer has been “the diff,” and the diff was never where the safety came from.

Doubt that runs

Every save in this essay has the same shape. The xz backdoor: a benchmark with a baseline. My fleet’s bad image: a canary with rollback authority. Heartbleed’s class of bug is hunted today by fuzzers that execute the parser rather than squint at it. What these share is not automation. They are executable and scored. Each one runs the change against reality. Each one produces a verdict that reality gets to grade, so the verdict accumulates calibration. Review, human or AI, is doubt that reads and is never scored. The primitives that work are doubt that runs.

I have already written about the pieces one at a time: rollback as the first thing you build, the branch as the agent’s blast radius, the system as its own eval. What I had not done is admit what they add up to: the assurance role we keep assigning to review already moved out. The question that replaces “did a qualified human approve this diff?” is “what would have to be true for this change to promote itself?” It forces every primitive the old question let you skip. Which invariant did it satisfy. What environment did it survive, against what production-shaped data. Which monitor would catch its failure mode, with what rollback authority. What blast radius was it allowed. What evidence survives the merge.

That question is also where the accountability lands. The alarm budget ended by asking who signs the number, and I think this is the answer: the human signature moves from the artifact to the harness. You stop signing line 47 and start signing the promotion contract: the invariants, the gates, the blast radius, the rollback SLO. A reviewed diff is attention spent once, on one change, unverifiable even in principle after the fact. A reviewed gate runs ten thousand times, and every run scores it.

None of which says “trust the tests.” The xz payload lived in the test directory; agents already write tests that bless their own code; Kent Beck’s red flag for agentic work is the moment the agent weakens a test to get green. Tests are claims. What caught xz was a scored baseline the attacker did not know was being watched. The gate is not any single artifact. It is the property of being executed and scored. And comprehension debt is still real. Review is still how teams manufacture comprehension. That function survives, smaller and more deliberate, spent on intent, contracts, architecture: the questions with no executable proxy. Review is not dead. It has been promoted out of a job it was failing at and into the only one it was ever good at.

What I am still figuring out

Which doubt has no executable proxy. “Does this fit the system we are building” and “should this exist at all” have no harness today, and I spend real hours on them. But I notice the category keeps shrinking. Five years ago “does this migration work against production data” was architectural judgment; now it is a branch and a dry run. I do not know how much of what I currently call judgment is actually a harness nobody has built yet. I also do not know how I would tell.

Whether the bottleneck recurses. If humans review gate designs instead of diffs, review has moved up a level, and the receiver-capacity problem comes with it. I think the trade is real rather than cosmetic. A diff is read once and never scored. A gate runs continuously and is scored on every firing. But “the flood moved up a tier, pre-aggregated and harder to audit” is exactly the failure I warned about in June, and I would be foolish to assume my own proposal is immune.

The experiment nobody has run. My correlated-blind-spot argument rests on evaluator self-preference research and on theory. The direct study does not exist in public: same-model author and reviewer, miss rates against cross-model and human baselines. If someone runs it and same-family review holds up, I will say so. You should know which parts of this essay are measurements and which are bets.

Andres Freund was not assigned to review xz. He had no checklist, no diff in front of him, no LGTM to grant. What he had was an instrument, a baseline, and the itch that something half a second slow was worth an afternoon. The most heavily defended backdoor in the history of open source had passed every reading it would ever face. The reviewers were the attacker. It failed its first honest measurement.

Fagan’s room was a gate, and it earned its authority two expensive hours at a time. The button we inherited kept the authority and quietly lost the mechanism. For twenty years the gap did not matter, because the volume was human. The real gates, the tests, the canaries, the monitors, the rollbacks, were accumulating underneath while we credited the ritual. Agents did not break code review. They raised the volume until the ritual and the gate came apart in our hands, and made us look at which one had been holding.

Review was never the gate. The gate is whatever keeps score.