After looking at the fourth PR for our service from someone we’d never even talked to and seeing the code comments that looked like Christmas trees, I thought that we have to create some rules so we don’t have to even bother reviewing the AI-generated slop nobody bothered reviewing before sending it to us.

I posted a question in our team channel, and everyone agreed with my proposition of drafting an “away-team” guide. Ironically, I did use AI for that. However, I mostly used it for two things:

1. Formatting, so it looked pretty (just like I do with these newsletters).
2. Looking at other teams’ guides to see if I may have missed anything.

Most of the rules in the guide I just spoke into the terminal, and those are mostly my opinions, and most of these things I have hooks for set up on my local setup. By the end of the day, my team reviewed the draft and agreed with the proposed rules.

Where do we put the guide?

Then came the question of where to put it. Because by the time people may see it in our docs, it may already be too late. And we also shouldn’t be duplicating it in the docs for each of our services. So I put it on the wiki about our team and the services we own as a separate tab. But I couldn’t fully avoid duplication because discovery was still gonna be a problem.

Then I realized. If people use AI to produce these PRs, we should target agents and not people.

So I created one huge code review for all of our packages to add to their Agents.MD a line like this:

with the link to the guide.

Whether it’s going to be effective remains to be seen, but at least instead of us spending time reviewing PRs we didn’t ask for, if they or their agents decide to do it anyway, now we can just link the guide and ask them to follow it.

You might be wondering what kind of rules we’ve added. And whether we’re actually following them ourselves. I can’t speak for the whole team, but I’ve already mentioned that I have hooks set up for most of these things already.

Here’s roughly what it looks like.

Before you write any code

First, open a ticket. Explain what you want to change, why, and roughly how. The owning team decides whether to accept and prioritize the work, take it themselves, or point you toward a different fix.

A surprise PR gets closed unread. That might sound harsh, especially when someone has already done the work. But that’s exactly why we want the conversation first. Waiting for an answer can cost the contributor a day. Finding out afterward that the finished change doesn’t belong in the package wastes considerably more effort.

For anything beyond a trivial fix, the approach needs an explicit yes before implementation. A ticket comment is enough. Resolve the open design questions there. Spending fifteen minutes agreeing on an approach seems preferable to arguing about it underneath a completed implementation.

There’s also a boundary to respect. A contribution should extend the package for its existing purpose. Business logic specific to your use case belongs in your own repo. Finding somewhere you could add it doesn’t mean it belongs there.

Keep the change small enough to understand

​I’ve already mentioned that’s how I have my AI agents do it too. Our target is seven files or fewer, reviewable in under 30 minutes. One cohesive change, one commit. A refactor and a new feature go separately. If the pieces depend on each other, split them into stacked PRs.

A huge PR mixing several unrelated changes is one of the first things we’d send back. I don’t want the reviewer to spend the first half hour figuring out how many different things they’re being asked to approve.

Follow the package’s existing style, layering, naming, and error handling. Look at the nearest relevant pattern before introducing your own. Working code still has to fit the codebase it’s joining.

I personally have almost 0 tolerance for comments. Especially if they don’t add any value and look like Christmas trees. The only reason to ever have them is if they explain a non-obvious reason: an external gotcha, or a constraint elsewhere in the system. A comment describing what the next line does means it shouldn’t exist.

Tests need the same scrutiny. Parameterize near-identical cases. Skip tests for getters, constants, and other coverage padding. Test current behavior; checks that confirm a removed feature is gone (a favorite thing for Claude to add) don’t meet our bar. I want to know which regression a test would catch. Personally, if I am working on a bug fix, I ask the AI agent to write a failing test to repro the behavior first. I don’t make them do full TDD when developing features, though.

Do your part before asking for review

The PR description should explain what changed, why, and how it was tested. It needs to stand on its own, with no unresolved design questions left for the reviewer to settle in the comments.

Automated analyzers should already pass, and review bots should have zero open comments (yep, even here I have 0 tolerance for comments - another hook I use to enforce this).

For a behavior or UI change, show it running. Screenshots, a short video, a running-app link, or the actual request and response you exercised. Give the reviewer something concrete to check. I would not even bother looking at the code without it. While working on Oneiras, I found that the “before” and “after” screenshots attached to the PR were REALLY helpful.

Then comes the requirement I care about most: you can explain the change, live, without the AI. How does it work? How does it fit the architecture? What did you consider and reject?

The bar is that you could fix this code if every AI tool became unavailable tomorrow. You cannot just say “Claude wrote it,” and the reviewer doesn’t get to use that excuse either. Our team has to carry the pager, and if I am woken up at night by this change and can’t figure it out, someone better know what’s going on.

Merged doesn’t mean you disappear

This one Claude found for me from looking at other teams’ guides. The owning team takes ownership of the code, but an outside contributor still supports the handoff. Our default support window is three months.

That includes tracking the change through every deployment stage, updating the runbooks and documentation, and resolving security and accessibility findings before handoff. If an away-team change causes instability, the owning team can revert it without prior notice.

Yep, we don’t have SREs either. We’re the ones carrying a pager and getting asked questions by security folks.

Do these rules apply to us too?

In an ideal world, yes. I do trust my coworkers to do the right things because we all share the on-call burden. I do care a lot less about whether the tests are parameterized or not than whether they understand the change and its impact.

I am not perfect either. I did trust AI too much for one of the changes this week because I was burdened with other things and then had to revert changes across 3 different repos. The difference is that I take ownership and don’t pass it on to someone and forget it ever happened.

What do you require from the human before reviewing an AI-written PR?

Cheers!

Evgeny Urubkov (@codevev)