OpenAI Graded Itself on Errands. Read the List.

OpenAI published LifeVal today, an evaluation that scores models on everyday practical tasks instead of coding problems and exams. Enrolling a child in kindergarten after moving. Checking when a car is due for service and booking the inspection. Ordering groceries within a budget. Unsubscribing from email lists. The scores are high, and the scores are the least useful part of it.

What it is and how it was built

OpenAI says it drew on existing time-use datasets and then ran a new demographically representative survey of 3,995 respondents, asking for each task how much people wanted help from a personal assistant. The resulting areas are caregiving, health, education, household administration, personal finances, shopping and accessing government services.

The methodology is better than most product benchmarks, and the framing is honest about intent: the company says it used this evaluation internally over the past year to track and drive improvement. That is a benchmark built to steer a roadmap, which is a reasonable thing to build and a specific thing to read it as.

Two caveats before the numbers

Best recorded is the phrase to slow down on. A best-of score tells you a task is achievable; it tells you nothing about the failure rate, and failure rate is the only number that decides whether you let software complete a transaction unsupervised.

The timing matters too. An independent index moved the majority of its weight onto private test sets last week precisely because public benchmarks are gameable, and a few days later a vendor published its own benchmark with its own model scoring 100% on several tasks. Both things can be done in good faith and they belong in different mental categories, which is the argument in the benchmark that went private.

Where those errands actually land

Read the task list as a business rather than as a consumer and the picture changes. Checking a service date and booking an inspection lands on a garage's booking system. Ordering groceries within a budget lands on a retailer's ordering flow. Enrolling a child lands on an enrolment form. Comparing providers lands on your pricing page. Unsubscribing lands on your mailing list.

Every one of those is a thing somebody will soon be doing to your business through software rather than in person. OpenAI says more than a billion people use ChatGPT weekly, which is the figure that turns a capability into a volume problem.

That makes the list a reasonable preview of the agent-driven traffic arriving at your intake over the next year, and the groundwork for it is covered in websites that talk to AI agents and customers ordering inside AI assistants.

What to check on your own side

Can your form be completed without reading the page around it? Instructions that live in prose beside a field, or requirements implied by a heading, are invisible to software filling the form on someone's behalf. Labels and validation messages have to carry the meaning.

Is your availability and pricing legible? A comparison task needs numbers it can read. Prices in images, availability that requires a phone call, and quotes gated behind a form are all ways to be absent from a comparison the customer is making anyway.

Does your unsubscribe work in one step? Scoring 100% on unsubscribing means agents will do it successfully at volume. A broken or multi-step unsubscribe becomes a complaint rather than a retained subscriber, and the engagement signal you lose was telling you something anyway, as in what your retention data already shows.

Can you tell agent traffic from human traffic? Most businesses cannot, which means they will not notice the shift until a pattern looks strange, and the logging work that makes it visible is in what customers will actually do themselves.

And on the inside

The same capability class is what you would use internally, so the caveat applies in both directions. A high best-recorded score on an errand is encouraging and it is not a reason to let an agent book, order or cancel on your behalf without a check.

Run your own small evaluation on your own real tasks, record the failures rather than the successes, and decide the approval rule from the failure rate. That is the method in how to evaluate AI yourself, framed as a real test in designing a proof of concept that decides.

Frequently Asked Questions