2 Million Test Prompts Show Where Your AI Data Can Leak
When you type something into an AI tool, how many companies read it before the answer comes back? A security researcher spent five months finding out, by hiding two million trackable secrets in prompts and waiting to see which ones somebody used.
What the researchers did
HD Moore, chief research officer at the security firm Dragos, published the results on 19 November. Over five months, from July to November 2026, his team sent about two million prompts to widely used AI services. Each prompt contained a secret that existed nowhere else: a unique web address or a login credential created only for that prompt.
Security people call these canaries. Nobody has a legitimate reason to open that address or try that credential. If someone does, the prompt was read by somebody other than the service it was sent to, and because each secret went into exactly one prompt, the researchers could tell which request had leaked.
What they found
| Finding | What Dragos reports |
|---|---|
| Providers compromised | Evidence that data from at least three AI providers was exposed. Seeded credentials were used in attempts to get into the test systems |
| Resellers reading prompts | Multiple resellers on an AI inference marketplace could view and keep customer prompts through standard logging, without exploiting any flaw |
| Privacy settings not honoured | Multiple cases where providers appeared not to follow customer privacy settings or their own published policies |
On the third finding, Dragos gives one detailed example. An AI lab in China fetched the secret addresses from more than half of the prompts sent to its models, including roughly two-thirds of the requests marked no data collection or zero data retention. The visit typically came two or three days after the request.
Dragos also reports prompts sent to US-based providers ending up with organisations in China, and addresses being opened months after a single request.
Why the chain matters
A prompt is encrypted while it travels. It is decrypted at each stop so that the stop can do its job. A router reads it to decide which provider should answer. That provider may be a reseller passing it to another. In the words of the report, every hop can see the entire prompt and response.
A small business can be on such a chain without having chosen one. The AI feature inside a piece of software, or a low-cost tool offering many models under one login, may send prompts through a router and on to whichever provider is cheapest that minute. We described that layer in the AI gateway. It has real benefits. It also adds companies that handle your text.
What the study does not show
The secrets were test data. No real customer records were involved in the experiment.
This is one firm's research, published two days ago. We are not aware of an independent replication.
The counts are not rates. At least three providers and multiple resellers tells you the problem exists, not how common it is.
The parts of the report we read do not say that prompts sent directly to the large model makers leaked. The findings concern the wider network of routers, resellers and smaller providers.
What to do about it
Keep credentials out of prompts. The secrets that were stolen and used in this study were login credentials. Passwords, API keys and access tokens do not belong in a chat window or an automation step that passes through an AI model.
Find out your chain. For each AI tool you use, ask the vendor three things. Which companies process our prompts? Are prompts logged, and where? If we turn on a no-retention setting, does every company in the chain honour it, or only the first?
Prefer fewer stops. Moore's advice, as reported by Ars Technica, is that organisations which cannot host models themselves should buy directly from a first-tier provider, so there is one contract and one privacy policy to read. He adds that this reduces the chance of a leak and does not remove it.
If you use a router, choose the providers. The same report advises against accepting a default pool of providers and resellers. Pick the upstream providers yourself.
Treat a privacy setting as a request. A retention flag is honoured, or not, by each company that receives the prompt. Past the first company you are relying on someone else's contract.
Remember who stays responsible. Under PIPEDA, a business remains accountable for personal information it hands to another company for processing. Sending a customer's details through a chain you cannot name is hard to square with that. See a privacy impact assessment and the buyer checklist for AI security.
Check what you promised others. If client material is covered by a confidentiality agreement, the chain matters twice, as set out in whether pasting into AI breaks an NDA. An approved-tools list in your AI policy is where the answer should live.
Frequently Asked Questions
What was the canary study?
Research published on 19 November 2026 by HD Moore, chief research officer at the security firm Dragos. Over five months the team sent about two million prompts to AI services, each containing a unique web address or credential, and monitored whether anyone later used them.
What did it find?
Dragos reports three things: evidence that data from at least three AI providers was exposed, multiple resellers on an AI inference marketplace that could view and keep customer prompts through standard logging, and multiple cases where providers appeared not to honour privacy settings or their published policies.
Does this mean ChatGPT, Claude or Gemini leak prompts?
The parts of the report we read do not say that prompts sent directly to the large model makers leaked. The findings concern the wider network of routers, resellers and smaller providers that a prompt can pass through. The researcher still notes that buying direct reduces the risk without removing it.
What is zero data retention, and did it work?
It is a setting or contract term under which a provider says it will not keep prompts. Dragos reports cases where it was not honoured, including one AI lab that fetched the planted addresses from roughly two-thirds of requests marked no data collection or zero data retention.
What is an AI router or reseller?
A router is a service that receives a prompt and forwards it to one of several model providers, chosen by rules such as price or speed. A reseller sells access to models that someone else runs. Each one in the chain decrypts the prompt to handle it, so each one can read it.
What should a small business do?
Keep passwords and keys out of prompts. Ask each AI vendor which companies process your prompts and whether they are logged. Prefer buying directly from a major provider on a business plan. If you use a router, choose the upstream providers yourself, and record approved tools in your AI policy.
Know who reads your prompts
We trace where your AI requests go, tool by tool, and help you set a short approved list that fits your privacy obligations.
Related Articles
How AI Can Strengthen (Not Weaken) Your Enterprise Data Security Posture
Document Control When AI Can Edit the File
Data Loss Prevention When Staff Use AI Every Day
Ajan leads the ChatGPT.ca team: 200+ custom GPT builds and automation projects for 50+ businesses across 20+ industries. Based in Markham, Ontario. PIPEDA-compliant solutions.