How we review code we didn't write

Most engineers learned to review code by reviewing colleagues. That experience builds a set of instincts which are genuinely excellent — and which misfire almost completely on generated code.

This is the process we arrived at after getting it wrong for a while.

The instincts that stop working

Reading the author, not just the code

Reviewing a colleague, you carry context: what they're strong at, what they rushed, whether they understood the ticket. You allocate attention accordingly. Generated code has no author history. Every diff arrives with the same flat, competent surface, and there's nothing to allocate attention against.

Using messiness as a signal

Confusing code is a reliable proxy for confused thinking, and reviewers slow down for it automatically. Generated code is never confusing. It is well-named, well-structured, and often commented. The proxy is gone, and with it the natural brake.

Trusting that the author understood the requirement

A colleague who misunderstood the ticket usually leaves evidence — a question in the PR, a hedge in a comment, an odd assumption stated out loud. A model that misunderstood produces something confident and complete that solves a subtly different problem, silently.

You are no longer reviewing whether the code does what the author intended. You are reviewing whether it does what you intended.

What we do instead

1. Review against criteria, never against the diff

Before anything is generated, the requirement is written as checkable statements. Review starts from that list, not from the code. The reviewer's first question is "does this satisfy criterion four?", not "does this look right?"

Reading a diff and asking whether it looks correct is how you end up approving something fluent and wrong. Reading criteria and hunting for where the code fails them is a different cognitive task, and a much more reliable one.

2. Read the tests first, and read them adversarially

Tests are where generated work fails most quietly. We read them before the implementation and ask one question: could this test pass if the implementation were wrong?

A test that asserts the function returns what the function returns is worse than no test — it produces a false signal that survives review and gets counted in coverage.

3. Run a machine pass before the human pass

An adversarial review agent runs on every diff before a human opens it, looking for the mechanical failures: missing authorisation, injection paths, unsafe dependencies, N+1 queries, silent behaviour changes. It's tuned pessimistic and it produces false positives, which is the correct trade.

The point isn't that it catches everything. It's that it clears the mechanical checks so human attention is spent on judgment, which is the only thing humans are still better at.

4. Give refactors more scrutiny than features

This is the counterintuitive one and it's the rule we'd keep if we could only keep one.

Feature diffs get read carefully because they're interesting and obviously consequential. Refactor diffs get skimmed because they're supposed to be behaviour-preserving. That expectation is exactly what lets a behaviour change ride through unnoticed — and generated refactors change behaviour more often than generated features do.

5. Ask what isn't in the diff

Generated code answers the question asked. It rarely volunteers what the question missed. So the reviewer has to supply the absence:

• What happens when this is called twice with the same input?
• What happens when the third-party call times out halfway through?
• Who is allowed to call this, and where is that enforced?
• What does this do with an empty collection? A very large one?
• Is there a migration, and does it run safely against production data volume?

6. Keep a named human accountable

Every merge has one engineer's name attached, and that engineer owns the consequences. Not a rota, not two approvals from whoever's free. Diffuse accountability produces diffuse attention, and that effect is stronger when the code looks good.

What we got wrong first

Two mistakes, both worth naming because they're the obvious ones to make.

We initially reviewed generated code more loosely than hand-written code, on the unexamined assumption that a model wouldn't make beginner mistakes. It doesn't make beginner mistakes. It makes confident intermediate ones, which are harder to see and further into production before they surface.

And we let coverage numbers rise without checking what the tests asserted. Coverage went up, confidence went up, and neither meant anything for several weeks. Now nobody looks at a coverage number without reading a sample of the tests behind it.

The underlying shift

Review used to be a check on work that had already been thought through. Now it is frequently the first point at which anyone thinks the problem through properly. That makes it the highest-leverage activity in the process, and it needs to be staffed accordingly — by the most experienced people available, not by whoever has capacity.

Which is, in the end, the whole argument for hiring a team that has been shipping production software for fifteen years rather than one that started last year with better tools.