Only 2.6% of AI Agent Tools Finish a Whole Task

A number travelled fast this week in the form "only 2.6% of AI agent tools actually work." The research it comes from does not say that, and the thing it does say is more useful if you are the one signing a purchase order. Cohere Labs published the Agentic Task Ecosystem dataset on September 3, 2026: 696,291 published tools across 123,069 public MCP servers, each matched against the 923 occupations in O*NET.

What Cohere Labs actually built

The team aggregated seven public directories of AI tools, collected in May 2026, into a single catalogue and released it on Hugging Face as Parquet files covering tool descriptions, occupation titles, SOC codes and quality ratings. Every tool is matched to tasks in O*NET, the U.S. government's occupational database, and each match carries its own quality score. Cohere calls it the largest open dataset of its kind, and the claim looks fair at nearly 700,000 tools.

MCP is the Model Context Protocol, the open standard that lets an AI system connect to software and data: query a database, edit a file, schedule a meeting. It has become the common plumbing between assistants and business systems, which is what makes a census of public MCP servers a reasonable proxy for what agent builders have shipped. We covered the standard itself in MCP, the quiet standard plugging AI into your tools.

Where the 2.6 percent comes from

The researchers applied a strict test: a tool qualifies only if it completes a recorded work task from start to finish, rather than informing a person doing one or handling a single step inside a process a human coordinates. About one tool in forty clears that bar. Those tools cover 1,380 task statements, roughly 15 percent of the software-performable work catalogued in O*NET.

So the 97 percent that did not qualify are not broken. Most of them assist. A tool that pulls the right three documents for a person reviewing a claim is doing something valuable and is not doing the whole task, so it sits outside the count by design. Cohere is direct about the limits: the dataset says nothing about reliability, deployment, or long-horizon completion. It also notes the count can only miss tools rather than overcount them, so 2.6 percent should be read as a floor.

None of that makes the finding less interesting. It makes it a different finding, about the shape of what has been built rather than the quality of it.

Three patterns a buyer should care about

Among the tools that group into recognisable categories, Cohere reports three dominant patterns: existing work represented at a finer grain than occupational databases record, infrastructure for running agents themselves, and a small amount of genuinely new work, most of which is the work of managing agents.

A large share of what you find when you go shopping is plumbing. Components for building agents, not agents that do a job. That matters for expectations: a business owner browsing an MCP directory expecting to find something that handles their scheduling is mostly looking at parts, and parts need an integrator.

The second pattern is the coverage gap. Cohere found 419 of the 923 occupations with no agentic tool activity of any kind, and observes that what gets built follows what can be built. Supply is tracking ease of construction, not business value. If your work sits in one of those gaps, the absence says something about how hard the plumbing is, not about how much the automation would be worth. We pulled the workforce side of this apart in will AI take my job.

Four questions for the vendor

The completes-versus-assists distinction is the most useful thing to take from this research, because it maps directly onto a sales conversation.

1. Does this complete the task or assist with it? Both are legitimate products. Only one of them lets you change how the work is staffed, and the pricing rarely distinguishes between them.

2. Who performs the last step? Many tools do 90 percent of a task and hand back for the approval, the send, or the sign-off. That is often the right design, and it means the human is still in the loop on every instance.

3. What happens when the input is unusual? The invoice in a format nobody has seen, the customer who replies with a question instead of a yes. Ask for a walkthrough of a case that went wrong and what a person had to do about it. A demo shows you the happy path.

4. What fraction of real cases hit the exception path? If it is 30 percent, your saving is a third of what the pitch implied, because a person is still reading every third case and now also has to work out why the agent stopped.

Assistive tools earn their keep

Tools that help a person work faster have paid back consistently in the businesses we see, and they carry far less operational risk than tools that act alone. The problem arises when an assistive tool is bought on a completion promise, and the team is planned as though the task has gone away.

There is evidence on the completion side worth holding onto. A benchmark that asked AI agents to run a simulated shop end to end found they did poorly at exactly the long-horizon coordination that a whole-task claim requires, which we covered in a benchmark asked AI agents to run a shop. Cohere's dataset counts what developers published. That benchmark tested what agents did. Both point the same direction on unsupervised completion.

Where you do buy a completing tool, the access controls matter more, because nobody is reading each output. Give the agent its own account, scope it to the systems the task needs, gate anything irreversible, and log what it did. The framework is in three boxes for an AI agent.

Read the caveats before quoting the number

This is a supply-side map. It records what developers published on public servers, and it is not evidence that any employer adopted any of it. Private MCP servers built inside companies are invisible to the count, and there are a lot of them. The collection happened in May 2026 in a market that moves monthly. And O*NET is U.S. occupational data, so Canadian job definitions differ in the details even where the task-level picture carries over.

One more caution, given the ecosystem being catalogued. A public directory of nearly 700,000 tools is also an attack surface, and installing an MCP server from an unfamiliar publisher gives it access to whatever you connect it to. We wrote about that in malicious AI skills and the supply chain.

What to do with this

Take the task you most want automated and decide which of the two products you are actually shopping for. If an assistive tool would give you most of the benefit, buy that, because it is cheaper, available now, and does not need an approval workflow built around it. If only a completing tool will do, budget for the exception path and the integration, and expect the timeline to be longer than the demo suggested.

Then ask the four questions above before the second meeting. Vendors who have thought carefully about where their product hands back to a person will answer them easily, and that answer tells you more about the product than any benchmark in the deck.

Frequently Asked Questions