AI Just Got Fast Enough to Sit Inside a Conversation

OpenAI hit 750 tokens per second running GPT-5.6 Sol on Cerebras hardware. That is roughly 550 words a second, faster than you can read and far faster than anyone speaks. It sounds like a spec-sheet number, the kind of thing only engineers care about. It is not. Speed is the quietest and most underrated variable in AI, because it does not change what the model says. It changes where the model can be, and a whole category of product only works below a certain amount of waiting.

Where the pause is fatal

Think about the difference between two situations. You ask AI to summarize a long document and it takes three seconds; you do not notice, because you were not going to read it faster anyway. Now put that same three-second gap in the middle of a phone call. The caller hears silence, assumes the line dropped, starts talking again, and the whole interaction gets clumsy. Identical model, identical quality, completely different outcome. That is the point: for anything that has to keep pace with a person, latency is not a comfort feature. It is the difference between a product working and not.

Where speed matters, and where it does not

This cuts both ways, which is useful when you are choosing what to pay for.

Speed is decisive	Speed barely matters
Voice agents answering calls	Overnight research and reports
Live suggestions while someone works	Batch document processing
Interactive tools a customer explores	Scheduled data cleanup
Anything a person waits on in real time	Anything reviewed the next morning

For the right-hand column, paying a premium for speed is money wasted; buy accuracy instead. For the left, speed is the whole product. This is exactly the distinction that determines whether AI voice agents on customer calls feel natural or robotic, and whether real-time coaching during calls is useful or distracting.

The ideas worth reopening

Almost every business that experimented with AI has one or two concepts they tried and quietly dropped, and in my experience the reason is rarely that the output was wrong. It is that the thing felt slow, awkward, or embarrassing in front of a customer. A phone assistant that paused too long. A live tool that made people wait. A chat widget that felt sluggish enough that visitors gave up. Those judgments were correct at the time and may simply be out of date. Before commissioning anything new, go back and re-test what you already rejected.

Measure it yourself

One caution: headline throughput figures come from specialist hardware under ideal conditions, and what you experience through a normal API on a normal day will be different. So do not buy from a spec sheet. Time a realistic task, the actual prompt with your actual context, on the tier you would actually pay for, and see what the delay feels like in the setting where it will live. That is the same evidence-over-marketing habit we recommend for evaluating AI beyond benchmarks, and it applies just as much to speed as to quality.

Why this keeps mattering

As AI moves out of the chat window and into live interactions, phone calls, in-product help, tools customers touch directly, latency stops being a technical detail and becomes part of the customer experience. It is the same shift that makes AI holding a live expert conversation plausible at all. The businesses that notice early get to build things their competitors still believe are impossible, for the simple reason that they last checked a year ago.

Frequently Asked Questions

What does 750 tokens per second actually mean?

A token is roughly three quarters of a word, so 750 tokens per second is somewhere around 550 words per second, faster than anyone can read and far faster than anyone speaks. OpenAI hit that figure running GPT-5.6 Sol on Cerebras hardware, which is built specifically for this kind of throughput. In practical terms it means the model finishes a substantial answer in the time it currently takes to produce the first sentence. The output does not change; the waiting does. And the waiting turns out to be what determines whether certain products work at all.

Why does speed matter if the answer is the same?

Because latency decides where AI can live. A two-second pause is invisible when you are reading a document summary and unacceptable in the middle of a spoken conversation, where it reads as an awkward silence and the caller starts talking over it. The same is true for anything that has to keep pace with a person: live suggestions while someone types, instant answers as a customer browses, interactive tools that respond as you adjust them. Below a certain latency those experiences simply do not feel right, and above it they suddenly do.

What becomes possible that was not before?

Mostly things that must happen inside the flow of a live interaction. Voice agents that respond without the tell-tale pause that signals a machine. Live coaching that surfaces a suggestion while a call is still happening rather than in a summary afterward. Search and product tools that respond as fast as a customer can click, so exploring is fluid instead of a series of waits. Multi-step agent work that finishes while you are still on the task instead of arriving later. None of these are new ideas. They were just uncomfortable to use.

Is this available to a normal business yet?

Partly. The headline figures come from specialist hardware that most businesses will only ever touch through an API, and speeds like this typically arrive first at premium pricing before spreading. But the direction is unambiguous, and there are already meaningful differences in responsiveness between mainstream providers and tiers you can access today. The practical move is not to chase the record. It is to notice that if you dismissed a real-time AI idea a year ago because it felt sluggish, the constraint that killed it may no longer apply.

How should a Canadian business use this?

Revisit the ideas you shelved. Most businesses have one or two AI concepts they tried, found too slow or clunky, and quietly abandoned, usually something customer-facing where the pause was embarrassing. Those are now worth a second look. Beyond that, measure responsiveness deliberately when you evaluate tools: time a realistic task rather than trusting a spec sheet, and weigh speed most heavily for anything a customer will experience live. For work that happens in the background, speed barely matters and you should pay for accuracy instead.