You can read a change you did not write, understand roughly what it says, and still have no idea whether it does what you asked for.
That gap is the whole difficulty. When you write code yourself, you carry the intent in your head while you type, so reading it back doubles as checking it. When AI writes it, the intent and the code came from different places — and reading the code tells you what it does, not whether that was the point.
So the useful question is not "is this good code?". It is: does this do the specific thing I asked for, and has it left everything else alone?
Test against what you accepted, not against the code
If you check the result by reading the implementation, you will tend to confirm whatever the implementation says. The code becomes both the answer and the mark scheme.
The way out is to have written down what should happen before you had a result to be pleased about. That description — plain sentences about what someone using the software should see — is your test. Everything else is commentary.
This is why testing gets much easier when the work was described properly in the first place. If your instruction was "add editing", there is nothing to test against. If it was "clicking Edit turns the row into a text box containing the current name, and Save replaces the row with the new name", the test writes itself. That side of it is covered in how to plan an app before you ask AI to build it.
A worked example
Take one small increment: adding an item to a list.
The accepted behaviour was:
When I type a name and press Add, the item appears at the bottom of the list straight away, and the input box clears.
Alongside it, three decisions had been settled: empty input is ignored, duplicate names are allowed, and existing items are not reordered.
The check is then mechanical, and takes under a minute:
- Type a name, press Add. Does it appear? At the bottom, not the top?
- Did the input box clear?
- Press Add with the box empty. Is nothing added?
- Add the same name twice. Are both there?
- Are the items that were already there still in the same order?
Five checks, drawn directly from the description. Notice that none of them requires you to understand the implementation, and that the last one — the items you did not touch — is the one that most often catches a real problem.
Notice also what is not being tested: whether the function is well named, whether the approach is elegant, whether it would scale. Those may matter later. They are not what this increment claimed to do.
Always test the thing you did not change
The check that earns its keep is the one on adjacent behaviour.
Changes made by AI often reach further than the request, because the model has the surrounding code in view and no strong sense of which parts are settled. So after any change, spend thirty seconds on the two or three things nearest to it that were working before. Can you still edit? Does the page still load with an empty list? Did the styling survive?
This is the cheapest possible insurance and it is almost universally skipped, because the change "obviously" only touched one area. It is worth knowing what to do when that assumption turns out to be wrong: when AI changes code you didn't ask it to touch.
What "it runs" does not tell you
A page that loads without an error has demonstrated exactly one thing: it did not crash.
It has not shown that the data is saved, that the right item was deleted, that the order is correct, that anything happens when the input is empty, or that the feature you built last week still works. Those are separate claims and each needs its own look.
The same applies to the AI's own summary of its work. Descriptions like complete, working, production-ready or fully tested are generated text, not test results. They may be accurate. They are not evidence, and treating them as evidence is how unverified changes get accepted — which is where drift begins.
Writing the checks down as you go is what turns "I had a look and it seemed fine" into something you can point at later. The test record is a structure for that.
How much checking is enough
Proportionate to the risk, which varies enormously.
A personal tool that formats some text needs very little. If it is wrong you will notice, and the cost is a minute of annoyance. Manual checks against the accepted behaviour are entirely reasonable.
Software that handles other people's personal data, takes payments, controls access, or does anything where being wrong has consequences beyond your own inconvenience, needs considerably more — automated tests that run every time, and, past a certain point, review by someone who genuinely understands the area. That is not a failure of method. Recognising where your own knowledge stops is part of doing this well.
Between those extremes, a reasonable rule: the harder it would be to notice a fault, the more effort the checking deserves. Visible faults are cheap. Silent ones — wrong figures, data quietly not saved, the wrong record updated — are the ones worth building a real test for.
When something fails
Write down what actually happened and what you expected instead, in that order, before asking for a fix. "Adding an item puts it at the top of the list; it should appear at the bottom" gives the AI a bounded problem. "The add function is broken" invites a rewrite.
Then re-run the same checks afterwards, including the adjacent ones. A correction is a change like any other, and it deserves the same scepticism as the change that caused the problem.
Would your test suite actually catch a regression?
Even a well-designed test suite can pass for the wrong reason: not because the code is correct, but because the tests never really exercised the behaviour that changed. Birgitta Böckeler, writing on Martin Fowler's site, raises this in a piece about signals that something has gone wrong in AI-assisted coding - what she calls sensors for coding agents - and treats the test suite as one of the more important sensors. The question worth asking of your own project is this: would this suite actually fail if a regression were introduced, or would it simply stay green?
Mutation testing is one way to answer that mechanically rather than by inspection. A mutation-testing tool alters the code in small, deliberate ways - flipping a comparison operator, shifting a boundary, skipping a line - and reruns the test suite against each altered version. If the suite still passes, that mutant "survived": something in the code changed and none of your tests noticed. A suite with a high proportion of surviving mutants is one that would probably let a genuine AI-introduced regression through as well, for the same reason - it isn't watching the behaviour it looks like it's watching.
This matters more, not less, when the code came from an AI agent rather than from you. You didn't write it line by line, so the test suite is doing more of the work of telling you when something you didn't ask for has changed. A suite that has never been checked for whether it would catch a deliberately introduced fault gives you agreement, not evidence.
You don't need to run mutation testing on everything, all the time. Running a mutation-testing tool occasionally against one piece of Behaviour you've defined carefully - the kind captured in the Behaviour Definition Template - and seeing what survives is enough to tell you whether that part of your test suite is a real check or just something that runs.