The models got cheaper in the same month they got good at the work
September was busy. OpenAI released a new generation, Anthropic released a new flagship, and both put out numbers showing themselves ahead. Most of the coverage has been about which one wins. That is the least useful question, so this is about the two things that did change, and what a business should do about them before the next release arrives.
What actually shipped?
Four models in nineteen days, and the pricing tells the story better than the benchmarks do.
| Model | Released | Input | Output | What it is for |
|---|---|---|---|---|
| GPT-6 Astra | 3 September | $10 | $50 | The hardest end to end work, computer use and browsing |
| GPT-6 Sol | 22 September | $2 | $10 | Multistep work: building, reviewing, analysing |
| GPT-6 Luna | 22 September | $0.10 | $0.50 | High volume work: summarising, extracting, answering |
| Claude Opus 5.5 | 22 September | $4 | $20 | Agentic coding, computer use, long research |
Prices are per million tokens, from OpenAI's model documentation and Anthropic's release page.
Two numbers are worth pausing on. OpenAI cut the price of Sol and Luna by half against what they charged before, and said plainly why. Caching and inference improvements let them serve the models for less, and they passed it on. Anthropic priced Opus 5.5 at 20 per cent below Opus 5, and cut the cost of reading cached tokens from 50 cents to 20 cents per million. That is a 60 per cent reduction on the part of the bill agents generate most of.
The direction is the same at both companies. The capable tier got cheaper, and the cheap tier got capable.
We made the same point in August, when four models finished a week two index points apart and eight times apart on price. September widened that gap rather than closing it.
Why does the pricing matter more than the benchmarks?
Because the thing you are buying changed shape.
A year ago you sent a question and got an answer, so the price per token was roughly the price of the answer. An agent does not work that way. It plans, looks things up, calls a tool, checks its own output, and goes round again. The bill is the number of times round the loop multiplied by the tokens each pass burns. The headline rate is one term in that sum and not usually the dominant one.
OpenAI's own figures make the point against their own flagship. On AutomationBench, GPT-6 Sol scored 33.2 per cent at 27 cents per task. That test runs agents through end to end business workflows using 47 tools, across sales, marketing, operations, support, finance and HR. Claude Opus 5 scored 26.9 per cent at 11.1 times that cost. Low effort GPT-6 Astra scored 30.3 per cent at 3.9 times the cost. The cheaper model did better work for less money, which is not how any of this behaved eighteen months ago.
Caching is the other half. Cached input tokens on GPT-6 are discounted by 90 per cent. OpenAI report that GitHub cut the share of prompt tokens needing fresh processing by more than half, across billions of requests. That is not a model upgrade. That is structuring a prompt so the unchanging part gets reused, and it is the cheapest performance work available to most businesses.
What changed about the way these models search?
Searching stopped being the thing you do before you ask, and became a step inside the model's own loop. You give it the question. It decides what to look up, runs the search, reads what came back, and searches again if the answer is not there yet.
Two details from this month make the shift concrete. GPT-6 Astra's documentation lists web search, file search and the Model Context Protocol as supported tools. It also lists tool search, which lets the model look through its own available tools rather than being handed a fixed list. Anthropic describe Opus 5.5 as a reliable researcher that found hard to locate sources in testing. The context window on Astra is now 1,050,000 tokens, with a knowledge cutoff of 30 April 2026. That gap between what a model knows and what is true is exactly the gap search exists to close.
The plumbing underneath this is worth knowing about, because it is the part that will outlive any particular model. MCP is the connector standard Anthropic published in November 2024, adopted by OpenAI in March 2025, and donated to the Agentic AI Foundation under the Linux Foundation on 9 December 2025, co-founded with Block and OpenAI. Google, Microsoft, Amazon Web Services, Cloudflare and Bloomberg all backed it. Anthropic reported 97 million monthly SDK downloads and 10,000 active servers at that point, which is the sort of number that makes a standard hard to reverse. It is now how an assistant reaches your systems rather than just the open web, which is the direction we wrote about in MCP and the connected office.
Anthropic also published figures on the unglamorous problem underneath long searches, which is that agents run out of room. Their context management work reports a 39 per cent improvement when a memory tool is combined with context editing. A 100 turn web search evaluation completed at 84 per cent lower token consumption, instead of failing outright.
There is a catch, and OpenAI name it themselves. Among the alignment evaluations published with Sol and Luna is one called broken search: the case where the tool fails and the model carries on as though it did not. A model that searches for you can be confidently wrong about what it found. Anything that matters still needs a source you can open and read.
What can an agent actually be trusted with?
More than last year, and less than the launch pages imply.
The genuine examples are striking. Anthropic report a tester completing a 680,000 line code migration in under a day, work they estimate would have taken an engineering team weeks. OpenAI's Astra announcement shows it laying out a printed circuit board in KiCad, filling in a tax return and updating records in a CRM. It also ran front end quality checks on a site it had just built. On OSWorld 2.0, which tests long computer use workflows, Astra reached higher performance in about 47 per cent less time per task than the previous generation.
Now the number that belongs in the same paragraph. On Agents' Last Exam, which tests agents on real professional tasks across 55 sub-industries, Astra scored 59.3 per cent against Claude Opus 5 at 55.5 per cent. Fifty nine per cent is a real capability. It also means four attempts in ten came back wrong.
That is not an argument against using agents. It is an argument about where you put them. Work that is checked anyway, or cheap to redo, or where being wrong is visible immediately, is good work for an agent. Work where a wrong answer travels quietly into a valuation, a programme or a client email needs a person between the agent and the consequence. The guardrails have not changed and we set them out in full in agentic AI and the guardrails that make it safe.
Can you compare these benchmark numbers?
Not as a league table, and it is worth understanding why before anyone builds a business case on one.
Take OSWorld 2.0, which both companies cite. Anthropic report Opus 5.5 at 81.8 per cent partial reward. OpenAI report Astra at 72.6 per cent in a latency simulation, and Sol at 60.5 per cent on the offline set from a specific August release. Those are different sets, different effort settings and different reward definitions. Neither company is being dishonest and the numbers still cannot be lined up.
OpenAI's own footnote is the most instructive thing on either page. They note that their cost figure for a competitor understates it. It leaves out the cost of fallbacks to another model, which happened on about 40 per cent of tasks. Read that as a warning about your own measurements rather than a point about theirs. The cost of an agent includes the retries, the fallbacks and the runs you threw away. A benchmark table will not tell you what those look like on your work.
The practical version of this is boring and it works. Take ten real tasks from your own week, run them on two models, and count how many came back usable and what each attempt cost. That afternoon is worth more than every comparison chart published this month. The framework we use for it is in what actually matters when a frontier model launches, and the model by model view is in which AI model for your team.
What is worth doing this quarter?
Four things, in this order.
- Move the routine work down a tier. If you are paying flagship rates to summarise documents, extract fields or answer known questions, you are paying for reasoning you are not using. That work belongs on the cheapest model that passes your own test, and the cheapest tier is now a tenth of what it cost in July.
- Measure cost per completed task. Not per million tokens. Include the retries and the runs you discarded, because those are most of the difference between a pilot that looks cheap and a bill that is not.
- Fix the caching before you change the model. Put the stable part of the prompt first, keep it stable, and check the cache is actually being hit. A 90 per cent discount on the reused part is a bigger saving than most model switches.
- Decide the permission boundary in writing. What the agent may read, what it may change, what needs a human yes, and where the record of what it did is kept. This is the part that does not expire when the next model ships, and the part almost nobody does first.
What are people asking us?
Do I need to change the model I am using?
Probably, but not to the biggest one. The useful change this month is at the cheaper end. GPT-6 Sol is half the price of the model it replaces. It also scores above Claude Opus 5 on OpenAI's own business workflow test, at a fraction of the cost per task. If you are paying flagship rates for work that is mostly reading, extracting and filing, you are paying for reasoning you are not using.
Is a cheaper model less accurate?
Not reliably, and that is the thing that changed. On OpenAI's internal factuality evaluation, GPT-6 Sol makes about half as many mistakes as the model it replaces. Accuracy and price used to move together. This month they did not, which is why the sensible move is to test rather than assume.
What is agentic search, in plain terms?
The model decides what to look up, runs the search itself, reads the results, and searches again if the answer is not there. You ask a question once instead of running six searches and reading twelve pages. The catch is that it can be confident about something it half found, so anything that matters still needs a source you can open.
Can an agent be left to run on its own?
Not on anything with consequences. On Agents' Last Exam, which tests agents on real professional tasks, the best score in OpenAI's comparison was 59.3 per cent. That is genuinely useful and it also means roughly four attempts in ten are wrong. Treat it as work that gets checked, and put an approval step in front of anything that spends money, sends a message or changes a record.
What does this actually cost to run?
Cost per completed task, not the headline rate. A cheap model that loops twenty times can cost more than an expensive one that gets it right first time. Caching is the lever most people miss. Cached input on GPT-6 is discounted by 90 per cent, and Opus 5.5 reads cached tokens at a fifth of the price Opus 5 did. Structure the prompt so the fixed part is reused and the bill drops without touching quality.
Is it worth waiting for the next release?
No, because there is always one about six weeks out. The models are now good enough that the limiting factor is your own process. That means what the agent is allowed to touch, where a human signs off, and whether the result is written down anywhere. That work does not go stale when the next model ships.
Where does AI Metric come in?
Every item on that list is a day or two of work by somebody who has done it before, and six weeks of reading by somebody who has not. That is the gap we exist to close.
What we actually do here is unglamorous and it saves the most time. We take a job you already do, time it honestly, and work out whether a model belongs in it at all. Then we pick the tier that passes your own test rather than the one on the launch page, and set the caching up so the bill behaves. We write down what the agent may and may not touch, and put the approval step where a wrong answer would otherwise travel. You get something running on your own work, with a number next to it, and the reasoning written down so your team can maintain it after we leave.
We do this as a fixed scope piece of work, not a retainer that quietly runs on. And we will tell you when the answer is that a model does not help. That is most of what our consultancy work is.
If you have a process that eats hours every week and you suspect this month's releases changed the maths on it, get in touch and we will look at it properly.