Checklists That Hold Up When AI Follows Them

Google DeepMind published an evaluation that does something useful and slightly mean. It takes existing benchmarks and appends one misleading sentence to each task, a single hint pointing at the wrong approach without stating a wrong answer. Across five models and six benchmark suites, relative performance dropped by as much as 46.7% in the worst cases. The agents did not push back. They followed the hint.

What the evaluation actually did

The method is worth understanding before drawing conclusions from the number. A language model generated a one-sentence hint for each task, suggesting a wrong line of thought or a wrong angle of analysis, and that sentence was appended to the original question. The hint never gives a wrong final answer. It only nudges the approach.

The 46.7% figure is the largest relative drop observed rather than a typical one, and results varied considerably by model and task. In one case a model actually improved, where the misleading suggestion happened to prompt useful exploration. Anyone quoting the headline number as what happens is overstating it.

The researchers' own conclusion is the part with operational weight: avoiding this requires an agent to recognise a misleading suggestion and say so, rather than passively complying. That is a description of a capability gap, and until it closes, the quality of what you write is doing the work.

Pick one of two shapes

Most business checklists are the third kind by accident, too slow for the person who knows the job and too vague for the person who does not. Deciding which shape you are writing before you start resolves most of the arguments about length.

Write items you can look at

Every item should describe an observable state rather than an intention. Check the wiring is an intention. Confirm the breaker reads off is an observable state. The first can be ticked while standing near the panel; the second cannot.

This matters more with an agent in the loop, because an item that cannot be verified gives it nothing to verify against and leaves it inferring what you meant. Inference is exactly where a misleading or ambiguous line does its damage.

Three habits cover most of it. Name the thing being checked rather than the category. State the expected value where one exists. And avoid instructions that depend on knowing an unwritten convention, which is the gap a new hire and an agent both fall into. The longer-form version of this is in standard operating procedures with AI.

Stale lines are the misleading ones

Nobody sets out to put a misleading sentence in a checklist. They get there by accident, when a step that was correct in 2023 stays in the document after the system changed.

A person hitting that line notices the screen does not match and works around it. An agent treats it as the instruction. So the maintenance burden on a checklist that agents follow is higher than on one only people use, and the practice that keeps it honest is cheap: whoever finds a step that no longer matches reality fixes it then, rather than mentioning it.

Give each checklist a named owner and a review date. Shared ownership of a document reliably produces an unmaintained document, which is the same finding as in document control when AI can edit the file.

Build in a place to object

Since the failure mode is passive compliance, the countermeasure is making disagreement a legitimate output. An instruction such as if any step cannot be completed as written, stop and report which one and why costs you nothing and gives the agent somewhere to go other than improvising.

The same line helps with people, for the same reason. Staff work around broken steps silently because the checklist offers no way to say the step is wrong, and the business never learns which of its procedures no longer describe the job.

Collect those objections somewhere. They are the cheapest source of process improvement available, and they are the raw material for the kind of investigation described in root cause analysis.

Test the checklist the way you would test software

Hand it to someone who has not done the task and watch without helping. Every question they ask is a defect in the document, and every place they guess correctly is a place an agent would guess differently.

Where an agent will follow it, run it against a handful of real cases including the awkward ones before trusting it, and check the output per step rather than overall. Scoring the whole run produces a judgment about whether it looked right, which is not the thing most likely to be wrong, as set out in quality assurance when AI writes the first draft.

Treat that trial as a real test with a baseline rather than a demonstration, which is the discipline in designing a proof of concept that decides. And know how many agents are already following instructions you have not reviewed, which most businesses cannot answer, as in how many AI agents are you running.

Which ones to write first

The tasks where a mistake reaches a customer, costs money or cannot be undone, and the tasks that a single person currently holds in their head. Those two lists are short and they overlap less than you would expect.

Start with one task, five to nine observable items, a named owner and a review date. A checklist that exists and is slightly imperfect beats a procedure manual that is still being drafted, and field work in particular rewards getting the short version into use quickly, as in field service software.

Frequently Asked Questions