AI code review works when you stop asking "is this good?" and treat it as a high recall defect linter. How I wired LLM review into a Flutter + Firebase PR flow.
For about a year I used AI code review the dumb way. I'd paste a diff into a chat window and ask, "Is this code good?" The answer was always yes — confident, well-formatted, subtly useless yes. The model praised my naming, suggested a doc comment I didn't need, and sailed right past the off-by-one that took production down two days later. I filed AI code review under toys.
I was wrong, but not about the toy part. I was asking the wrong question. "Is this code good?" is a judgment call, and judgment is the one thing these models are worst at. Once I stopped asking for opinions and started asking for a specific, boring, high-recall sweep, the bot became the most reliable member of my review rotation. Not the smartest. The most reliable. Those are different jobs, and confusing them is why most teams bounce off AI code review after a week.
This is the entire reframe: an AI reviewer is not a senior engineer, it's a linter with reading comprehension. Everything below is how I operationalized that on a real Flutter + Firebase codebase with a small team in Dubai — the context bundle, the severity contract, the exact prompts, and the places I deliberately switch it off.
"Good" is contextual. Good for a throwaway prototype is different from good for the payments path. A model doesn't know which one you're in, so it defaults to a generic idea of goodness it absorbed from a million public repos — mostly tutorials, mostly overengineered. Ask a large language model to judge and it will confidently reshape your pragmatic code into something that looks like a design-patterns textbook: interfaces nobody implements twice, a factory for a thing you construct once, a comment restating the line below it.
What it's genuinely great at is the opposite of judgment: exhaustive, tireless pattern-matching. It never gets bored on line 400 of a diff. It never thinks "the author is senior, I'll skip this bit." It has no ego and no Friday-afternoon fatigue. So I reframed the whole thing. I don't ask the AI whether my code is good. I ask it to find specific classes of defect and shut up about everything else.
The mental model that made it click: treat the model like static analysis that happens to read English. A linter doesn't have opinions about your architecture; it has rules, and it applies every rule to every line. The AI's edge over a traditional linter is that its "rules" can be fuzzy and written in prose — "flag any Future in a widget lifecycle that isn't awaited or explicitly fire-and-forget" is a check no ESLint plugin ships, but a model enforces it happily. Its weakness over a linter is that it's non-deterministic and eager to please. The rest of this post is about leaning into the first property and engineering around the second.
I wrote this split on a whiteboard and it hasn't changed much since. The bot owns recall on mechanical defects. Humans own judgment.
| The bot's job (high recall, low stakes per miss) | The human's job (judgment, high stakes) |
| --- | --- |
| Null/late-init hazards, unawaited futures | Is this the right feature to build at all? |
| Error paths that swallow exceptions | Does this fit the product and the roadmap? |
| Off-by-one, boundary and empty-collection cases | Is the abstraction worth its weight? |
| Copy-paste drift (fixed in one place, not the other) | Will this be painful to maintain in a year? |
| Missing await, resource leaks, unclosed streams | Is the tradeoff acceptable for this module? |
| Obvious security tells: logged secrets, injection | Should this ship now or wait? |
The line is simple: the bot catches things that are objectively wrong given the code in front of it. Humans decide things that require knowing where the company is going. If a review comment needs context that isn't in the diff — business priorities, team conventions we haven't written down, the fact that this service is deprecated next quarter — that comment belongs to a human. Everything else is fair game for the machine.
A useful test when you're deciding which bucket a check falls into: could you, in principle, write a deterministic rule for it if the language gave you a rich enough type system? Unawaited futures, empty catch blocks, logged tokens — yes, those are rules. "Is this the right abstraction?" — no, that's a conversation. Keep the bot strictly on the left column and it stops embarrassing you.
The single biggest jump in quality came from feeding the model more than the diff. A raw diff is a keyhole. You see three changed lines and none of the function they live in. Of course the review is shallow — the reviewer is functionally blind. Most people's disappointing first experience with AI code review is really a context problem wearing a capability costume.
On our setup, the review step assembles a context bundle before the model ever sees the change:
REVIEW.md describing our conventions — how we handle errors, how we structure Riverpod providers, what we consider a leak.That last one does more work than the other three combined. Here's a trimmed version of ours:
## Review conventions- Every Future in a widget lifecycle must be awaited or explicitly fire-and-forget with a comment saying why.- No `print` in lib/. Use the logger. Never log tokens or PII.- Firestore writes that touch user data need a matching security rule in the same PR. Flag if rules/ is untouched.- Prefer failing loud over silent catch. An empty catch block is a bug until proven otherwise.
Feeding conventions in text is how you turn a generic reviewer into your reviewer. The model stops flagging our intentional patterns as problems and starts flagging violations of the patterns we actually care about. Context is not a nice-to-have here; it's the difference between signal and a machine that argues with your house style.
One practical note on assembling that bundle: keep it lean and relevant. Stuffing the entire repository into the prompt is not the goal and usually backfires — the more unrelated code you include, the more the model's attention gets diluted and the more likely it is to comment on things outside the diff. I include the changed functions, their direct callers, the security rules that govern the touched collections, and the conventions file. That's it. Precision in what you feed it is as important as volume.
If you use a hosted review tool instead of rolling your own, the same principle holds: the ones that meaningfully outperform "paste a diff into chat" are the ones that pull surrounding function bodies, git blame, and repo-level config into the model's window before asking for a verdict. When you evaluate a tool, that's the feature to interrogate — not the marketing around model size.
The fastest way to kill an AI reviewer is to let it comment on everything. The first week we turned it on, it left forty comments on a two-hundred-line PR. Rename this variable. Add a doc comment. Consider extracting this. This could be a const. Every one was defensible. Together they were sludge, and within three days the team was collapsing the bot's comments without reading them. That's the failure mode you have to design against: a reviewer people learn to ignore is worse than no reviewer, because now the real bugs are buried in noise.
Two things fixed it.
First, an explicit severity contract in the prompt. The model must classify every finding as blocker, warning, or nit, and — this is the key part — nits are dropped entirely unless the author opts in. We only surface blockers and warnings by default.
Classify each finding as blocker | warning | nit.- blocker: will cause incorrect behavior, data loss, or a crash.- warning: likely bug or a real maintainability trap.- nit: style/preference. DO NOT REPORT nits.If you are not at least 80% sure it is a real defect, do not report it.Output nothing when the diff is clean. Silence is a valid review.
"Silence is a valid review" mattered more than I expected. Models want to be helpful, and helpfulness leaks out as manufactured concerns. Explicitly permitting an empty review stopped it from inventing problems to look useful. If your prompt doesn't grant the model permission to say nothing, it will always find something, and that something is usually noise.
Second, I measured the false-positive rate by hand for two weeks. Every comment got a tag: true bug, real-but-trivial, or wrong. When wrong-plus-trivial crept past a third of comments, I tightened the prompt. The target I settled on: at least two out of three surfaced comments should be things a good human reviewer would also have flagged. Below that ratio, people stop trusting it, and trust is the whole product.
A couple of other levers helped once the severity contract was in place. Confidence-gating — that "80% sure or stay silent" line — does real work; without a floor, the model reports hunches as findings. And a hard cap on comment volume forces prioritization: if I tell it to surface at most the five most serious issues, it spends its budget on the async race instead of the variable name. A reviewer that can leave forty comments will; a reviewer that can leave five learns to triage.
This is where it earned my loyalty. Every engineer has a blind spot — a category of bug their brain skips because it pattern-matches the code as "fine." Mine is async ordering in Dart. I read the happy path, it looks right, and I miss the missing await that makes a write race a read.
Consider this. It compiles, it usually works, and it's wrong:
Future<void> saveAndRefresh(Profile p) async { _repo.save(p); // no await — fire and forget by accident final fresh = await _repo.fetch(p.id); state = fresh; // may read stale data before save lands}I have shipped this exact shape more than once. A human reviewer reading fast skips right over it — the code looks like it does the obvious thing. The bot, sweeping for unawaited futures as a hard rule, catches it every time. The fix is a one-character change (await _repo.save(p);), but the whole point is that the defect is invisible to fast human reading and trivially visible to an exhaustive machine sweep.
The same category shows up all over a real Flutter + Firebase app:
setState or state mutation after a widget may have unmounted — the classic "setState() called after dispose()" crash that only fires when a user navigates away mid-request.where clause that the security rules assume exists. It passes in dev where you're the only user and your reads are always scoped to your own uid, then fails or leaks under real access patterns.parseAmount in one file, forgot its twin. Copy-paste drift is invisible to the author, who only remembers touching one of the two.StreamSubscription or AnimationController created in initState and never disposed — a leak that never crashes and never shows up until you're profiling a memory graph weeks later.None of these need intelligence. They need a reviewer that runs the same boring checklist on every single line without tiring. That is exactly what a human is bad at and a machine is good at. Once I stopped treating the bot as a peer and started treating it as an exceptionally diligent linter with reading comprehension, it stopped disappointing me and started saving me.
If you want to find your own blind spot, do what I did: for a month, tag every escaped bug with a category. Mine clustered hard around async ordering and resource cleanup. Whatever your cluster is, that's the checklist you write into REVIEW.md first, because that's where the machine buys you the most.
It is not free and it is not universally good. There are places I explicitly disable it, because there its confidence becomes a liability.
Knowing when to switch it off is not a limitation of the tool. It's the skill of using it. A reviewer that's on for everything is a reviewer nobody calibrates their trust against. The teams that get value out of AI code review are the ones with an explicit "here's where we don't use it" list, not the ones who flip it on globally and hope.
I don't run tools on faith, so I tried to answer honestly: is this making the code better, or just making me feel productive? Vanity metrics like "comments generated" tell you nothing — a broken bot generates plenty.
What I actually tracked over a quarter:
The honest result: the bug reduction was real but modest. The bigger effect was where human attention went. My reviewers stopped playing linter and started doing the judgment work only they can do. I'd have kept it just for that.
If you're piloting this on your own team, pick one leading metric (false-positive rate) and one lagging metric (escaped defects) and ignore everything else for the first month. The leading metric tells you whether people will keep trusting the tool; the lagging one tells you whether it's actually protecting the codebase. Comment counts, time saved, "developer sentiment" surveys — all of it is downstream of those two.
Tools that get tolerated eventually get disabled. The difference between tolerate and trust came down to a few unglamorous choices.
Once the team watched it catch a real production-shaped bug — the unawaited write, in a PR two of us had already approved and were one click from merging — the argument was over. It wasn't replacing anyone. It was covering the exact gap human attention reliably leaves on a Friday afternoon.
REVIEW.md. A diff alone makes an AI reviewer blind, and that's the number-one reason teams find it shallow.