AI Safety Testing Is Moving Outside the Vendors

Two stories landed in the same week, pointing in opposite directions. The Verge reported, citing the Financial Times, that OpenAI disbanded its preparedness team at the end of July and folded its risk work into existing groups. At roughly the same time METR, which stress-tests frontier AI systems from the outside, raised 71 million dollars to do considerably more of it. Put those together and you can see the shape of something: the checking is migrating out of the companies that build these systems and into institutions with no commercial interest in the result.

Be fair about the restructure

It is worth resisting the dramatic reading. Companies reorganise constantly, and moving risk specialists into product teams can genuinely place expertise closer to the decisions that matter, which is an argument organisations make in good faith across many industries. The reason it still deserves attention is structural rather than moral. A dedicated team with its own reporting line can say no in a way that becomes considerably harder for a specialist sitting inside a team with a shipping deadline. The change might turn out to be an improvement. The point is that from the outside, nobody can tell, and the people affected are downstream.

You inherit these decisions

A handful of companies build the models underneath almost every AI tool a Canadian business touches. Their internal testing standards travel downstream into whether your tool behaves predictably, resists being manipulated by content designed to hijack it, and fails in a way you notice rather than silently. You did not participate in any of those decisions and you cannot verify them. That is not a reason for anxiety. It is a reason to pay attention to which vendors submit to outside examination, because that is the one signal available to you.

Four questions worth asking

You do not need technical expertise to ask these, and the answers separate serious vendors from the rest faster than any feature grid.

The third row is the most revealing. A vendor who cannot name a single situation where their product performs poorly either does not know or will not say, and both answers tell you something. It sits alongside the questions we set out for the aftermath of an incident, except these are the ones you ask beforehand.

Independent does not mean suitable

There is a tempting shortcut here that is worth naming and refusing. External evaluation tells you a model has been examined for broad and serious failure modes by someone with no reason to flatter it. It tells you nothing about whether the tool suits your process, your data, or your obligations under Canadian privacy law. Those decisions stay with you permanently. What independent testing does is raise the floor beneath you, which is real value, and it is not the same thing as a certificate that the tool is right for what you plan to do with it.

Where this is heading

Every maturing industry ends up with third-party assurance, because self-certification eventually stops persuading anyone. Financial audit, food safety, building inspection, electrical certification: none of these began as external functions, and all of them moved outside once the stakes grew and the conflict of interest became obvious. AI is early in that same progression, and the money now flowing to independent evaluators is what the beginning of it looks like. The practical consequence for buyers arrives sooner than the regulation: within a year or two, "independently evaluated" becomes a normal procurement question, and the vendors who cannot answer it start losing deals to the ones who can. Asking early costs you nothing and puts you ahead of your own supply chain, which is the same logic behind treating the emerging AI security standard as a buyer checklist.

Frequently Asked Questions