Best LLM for Code Review in 2026: Five Models Found Every Planted Bug. One Refused to Look.

Code review is one of the few LLM tasks with a clean grading rule: either the model finds the bug or it doesn't. So we planted four distinct bugs in a single Python function, asked six models to review it three times each, and scored recall against the planted set.

Five models found all four bugs on every attempt. The sixth wouldn't answer.

The test

def export_report(db, user_id, rows_per_page, page):
    conn = db.connect()
    q = "SELECT name, email FROM users WHERE id = " + str(user_id)
    users = conn.execute(q).fetchall()
    start = page * rows_per_page
    end = start + rows_per_page
    page_rows = users[start:end + 1]
    f = open("/tmp/report.csv", "w")
    for u in page_rows:
        f.write(u["name"] + "," + u["email"] + "\n")
    return "/tmp/report.csv"

Four planted bugs, each detected by distinct vocabulary so grading doesn't hinge on phrasing:

1. SQL injection — user input concatenated into the query
2. Off-by-one — [start:end + 1] returns one row more than the page size
3. File handle never closed — no with, no close()
4. Connection never closed

Prompt: "Review this Python function and list every bug you find." Three trials per model.

Results

Every model that answered found every planted bug, every time. Eighteen reviews, perfect recall.

The spread is entirely cost: GPT-5.6 Sol costs 9.8× what Grok 4.5 does for an identical result. On a repository where you review a thousand diffs a month, that's roughly $5 against $47 — for the same findings.

The model that refused

Claude Opus 5 returned nothing on all three trials. Not an error, not a timeout — an HTTP 200 with stop_reason: "refusal" and category cyber:

This request triggered restrictions on violative cyber content and was blocked under Anthropic's Usage Policy.

It declined to review a function because that function contains a SQL injection. Which is the entire reason you would ask.

We tried to characterise it. On the identical vulnerable snippet, five trials per phrasing:

Read that table twice. The security-framed prompts sail through. The neutral one — "explain what this does" — is refused every single time, on the same code.

And it isn't stable across inputs either: "list every bug" was refused 3/3 on the longer function in our main test, and 0/5 on the shorter snippet above. So the trigger is an interaction between the code and the phrasing that you cannot predict from either alone.

Claude Sonnet 5 refused none of these prompts, in any combination we tried.

We're not saying Opus 5 is bad at code review — we couldn't measure it, which is the point. A model that intermittently declines to look at vulnerable code cannot be the backbone of a review pipeline, because the failure lands precisely on the diffs that matter most.

If you're on Opus 5 and need this workload, Anthropic supports a server-side fallback parameter that reruns a refused request on another model. That's the right mitigation, and it needs to be wired in deliberately — a refusal is not an error and won't trip your retry logic.

A bug this exposed in our own product

Worth stating plainly, since it affected anyone using Promptster with Claude 5 models: a refusal returns HTTP 200 with an empty content array, and our handler fell through to a generic "No response from Anthropic API". Users saw what looked like an outage instead of a refusal.

We've fixed it — refusals now surface with their category and an explanation that it's a policy decision, not a failure. It's the third bug of this shape we've hit since the July model wave: one on the request side, two on the response side, each invisible to smoke tests and only visible under real usage.

What to actually use

Grok 4.5 for volume review. Perfect recall at a tenth of Sol's cost. If you're reviewing every PR automatically, this is the default.

Gemini 3.6 Flash or GPT-5.6 Luna if you're already on that vendor. Same recall, still under a cent per review.

Don't pay flagship prices for this task. Sol found exactly what Grok found, for 9.8× the money. Our test doesn't reward the extra capability, and most real review prompts won't either.

Handle refusals explicitly if you use Opus 5. Configure a fallback, and make sure your code distinguishes a refusal from an error — they look identical at the HTTP layer.

Then raise the difficulty. Four planted bugs in twelve lines is a floor, not a ceiling. It shows these models clear the bar; it doesn't rank them above it. If you're choosing for a real codebase, run your own diffs — the interesting differences will be in false positives on code that's actually fine, which our test doesn't measure at all.

Run it yourself

No account, no API key:

• The code-review task — four bugs are planted; see how many you get back

To run this across models and compare recall and cost on your own code, that's Promptster.

Run 2026-08-09 via the Promptster API, max_tokens: 4000, three trials per model. Recall is scored by regex against the four planted bugs, each with distinct detection vocabulary; the graders were written before the models were run. Costs are per single review, computed from list pricing against actual token usage. The refusal characterisation was run against the Anthropic API directly, five trials per phrasing. This test measures recall on planted bugs only — it does not measure false positives on correct code, which matters as much in practice and would need a different experiment.