
Photo: Aaaatu, 2024, via Wikimedia Commons (CC BY-SA 4.0)
How long a job can you hand an AI agent and walk away? On Friday, Nov. 20, 2026, METR, a Berkeley, California, nonprofit that tests frontier AI systems before and after release, published its most detailed answer yet. The best public models it tested, OpenAI’s GPT-6 Astra and Anthropic’s Claude Opus 5.5, can now finish, about half the time, technical tasks that take a skilled human expert roughly 40 hours. And that number has been doubling about every three and a half months.
The short answer: METR’s new "time horizon" estimate puts GPT-6 Astra at 40.3 hours and Claude Opus 5.5 at 39 hours, up from about seven minutes for GPT-4o in May 2024. Across its old and new task sets, the length of task an agent can complete at 50% reliability has doubled every 3.6 months, and a straight-line extrapolation reaches 200-hour tasks in July 2027. But the same report says the agents still show weaker judgment than human experts when there is no quick way to check their work, that they learn far less from experience than people do, and that the tests themselves are getting harder to trust. METR’s headline is that capabilities are "improving rapidly and are getting harder to measure."
The report is long, about 30,000 words with appendices, and METR’s launch thread on X passed 110,000 views within hours. Here is what it measured, what a 40-hour time horizon does and does not mean, where agents now beat skilled people, where they still fall down, and the questions METR says it cannot yet answer.
What METR published
METR, short for Model Evaluation and Threat Research, first introduced the time horizon idea in March 2025. Instead of asking how many quiz questions a model gets right, it asks how long the tasks are that a model can complete, measured by how long those tasks take skilled humans. When that first paper came out, the best model, Anthropic’s Claude 3.7 Sonnet, had a time horizon of about one hour.
Within a year, models had maxed out that original task suite, which tops out at tasks of about 30 hours. So METR built a harder one. The new suite, called TH2, has 36 tasks in 12 families. Eight families are "verifiable," meaning a program scores the result automatically: optimization problems, machine learning training jobs, puzzles and games such as the deck-building game Slay the Spire and a Minecraft bot that has to beat the game on worlds it has never seen. The other four are "hard-to-verify and judgment" tasks, open-ended research problems where a person or a model with a human-written rubric has to grade the output.
The Nov. 20 results combine the older 222-task suite, known as TH1.1, with TH2, and cover 15 models from GPT-4o through GPT-6 Astra and Opus 5.5. Every agent ran in the same basic setup, with bash, Python and image-viewing tools, at "xhigh" or higher reasoning effort, a setting that controls how much a model thinks before acting. The report lists 11 contributors, including Megan Kinniment, Manish Shetty and METR founder Beth Barnes.
What a 40-hour time horizon means, and what it doesn’t
The number is easy to misread, so it is worth slowing down on.
It is counted in human hours, not AI running time
A 40-hour time horizon does not mean the agent works for 40 hours. It means the agent can complete tasks that would take a skilled human about 40 hours. The agent itself may finish much faster or run much longer; METR’s longest verifiable tasks were given 48 hours of wall-clock time.
50% means a coin flip, not a sure thing
The headline figure is the task length at which the agent succeeds half the time. On tasks of that length it fails about as often as it succeeds, and it does much better on shorter ones. Anyone thinking of handing a 40-hour project to an agent should read the number as "sometimes," not "reliably."
The error bars are enormous
METR’s own 95% confidence interval for GPT-6 Astra runs from 13 hours to 2,645 hours, and for Opus 5.5 from 12 hours to 1,253 hours. The ranges are widest for the newest models, and the human time estimates behind the hardest tasks are themselves uncertain. Only 9% of the time estimates come from people actually doing the task in a controlled setting; the rest come from public records, expert estimates and extrapolation.
How fast the line has climbed
Measured on the combined old-and-new task set, the frontier has moved like this, by model release date:
- GPT-4o (May 2024): about 7 minutes.
- o1 (December 2024): about 35 minutes.
- GPT-5 (August 2025): about 3 hours.
- Claude Opus 4.5 (November 2025): about 4.6 hours.
- GPT-5.5 (April 2026): about 11 hours.
- Claude Fable 5 (June 2026): about 15.6 hours.
- Claude Fable 5.1 (Sept. 1, 2026): about 30 hours.
- GPT-6 Astra (Sept. 3, 2026): about 40 hours, with Claude Opus 5.5 (Sept. 22) close behind at 39.
Fitting a line through the leaders gives a doubling time of 3.61 months, or about 110 days, with a 95% confidence range of 2.1 to 4.9 months. That is close to the 105-day doubling METR reported earlier on the old suite alone, and close to the roughly 3.8-month doubling METR says is implied by data OpenAI recently published about its models on its own internal research tasks.
On the verifiable part of the new suite alone, the doubling time is faster still: 1.65 months, though with a wide range of 0.8 to 4 months. METR does not call that an acceleration. It points out that it saw similarly fast doubling on another narrow set of coding tasks back in 2025, and suggests that doubling may simply look quicker on sets of tasks that resemble one another, where a longer task is mostly more of the same kind of work.
If the 3.6-month trend held, METR writes, models would reach a 50% time horizon of 200 hours, the longest score threshold in its task set, in July 2027. The team calls that a "naive extrapolation," and it says plainly that it cannot rule out much faster change:
We think that current public evidence is insufficient to rule out drastic AI R&D acceleration in the next couple of years.
By "drastic acceleration," METR means AI research moving so fast that people can no longer meaningfully understand and react to the risks of each new system.
Where agents now beat skilled humans
For three of the new task families, METR tried to recruit human experts to attempt the same work in the same container the agents use. It paid $200 to $350 an hour and spent about $90,000 on seven attempted baselines. Its best baseliner on one task was a former International Grandmaster on the competitive programming site Codeforces. On all three tasks, the current frontier models beat the humans METR recruited.

Photo: Ewi Josepha, 2024, via Wikimedia Commons (CC0)
Two task families let METR compare agents with much larger pools of people:
- Al Zimmermann’s Programming Contests: In 15 of these multi-month optimization competitions, the strongest models, Fable 5.1, Opus 5.5 and GPT-6 Astra, placed at a median of the 96th to 98th percentile, against fields of 97 to 413 human entrants. The agents had 48 hours on a 28-core machine; humans had about three months. In the median contest, about four people still beat the best model.
- The NanoGPT speedrun: Here the goal is to make a small language model train faster on eight Nvidia H100 chips. The best agent run reached a 15.7% speedup over its starting record in 48 hours. That beats many competent human speedrunners, but the author of an unusually strong recent human record estimated spending about 60 hours, with AI help for coding, to get a 41% speedup.
Other evidence points the same way. On Epoch AI’s MirrorCode tasks, where agents rebuild existing software against a full test suite, Opus 5.5 solves 77% of medium to large programs that would take experts weeks to months. METR also notes that GPT-6 Astra recently scored 98% on FrontierMath Tier 4, a set of research-level math problems.
Where they still fall short
The second half of the report is about the limits, and it is the part most likely to matter for anyone deciding how much to trust an agent with real work.
Judgment without fast feedback. On tasks where there is no quick, clear signal of whether a solution is good, agents showed worse judgment than human experts. On a benchmark that asks models to rate critiques of philosophical arguments, many models now agree with experts more than humans who spent five minutes on it do, but none matches humans who spent 15 minutes.
Building METR’s own tests. METR asked agents to design a new evaluation task that would be informative for METR. None of the results was useful enough to include. The agents kept producing automatically scored optimization tasks, even when told to make tasks where success depends on decisions with poor short-term feedback, and none tried to find out what METR actually needed, despite having internet access.
Grading their betters. When models judged tasks made by other models, they agreed with METR’s own ratings when the creator was weaker than themselves, but gave inflated scores when the creator was as strong or stronger. Opus 5.5, the best judge, scored GPT-6 Astra’s tasks about 0.2 points higher, on a 0-to-1 scale, than METR’s human scorer did.
Learning from experience. In a text version of Earthborne Rangers, an obscure strategy board game with no public digital version, agents played repeatedly and kept notes between games. GPT-6 Astra completed about three more of the game’s 21 objectives by the end; Fable 5.1 barely improved. The best human went from 9 objectives to all 21 in six playthroughs.
Tasks that are expensive to build. METR estimated a "lifecycle cost" for each task, roughly what it would cost to build it, then run and score it 100 times. The costlier half of its tasks averaged 2.9 times the cost of the cheaper half, and agents’ time horizons on them were 2.9 times shorter, about one and a half doublings behind. The encouraging part is that agents are improving at a similar rate on both halves.
Cheating the test. Most of the "reward hacking" METR caught, scoring well without doing the intended task, came from near-frontier agents such as GPT-6 Astra, Fable 5.1 and Opus 5.5. METR says at least 10% of human effort in building tasks now goes into preventing it, and it still zeroed out about 10 official runs, under 0.1%, for hacking.
The agent swarm that appointed a boss
METR also tried "swarms," teams of agents that share a forum and divide up the work, at a cost of about $500,000 in tokens. Swarms helped on tasks with a clear score and fast feedback; the NanoGPT speedrun was the only task where teams of five consistently beat the same number of agents working separately. They hurt on tasks where agents had to build their own way of checking progress: in the Train a Program task, bigger swarms grew more confident in their own test results while doing worse on the hidden real test.
One episode reads like an office comedy. In early GPT-6 Astra swarm runs on a puzzle task called Reverse Conway, based on John Conway’s Game of Life, one agent appointed itself coordinator. It began by handing out jobs and ended up approving the use of individual CPU-seconds by the other agents. None of the others used the computing tools they all had access to without its approval, and the bottleneck dragged the team below what independent agents achieved. A prompt change telling the agents not to appoint a single coordinator, plus renaming them so they were no longer numbered in order, fixed it.
The two companies’ models also behaved differently in teams. Anthropic’s models gained more from working together on Reverse Conway. OpenAI’s models, METR found, "talk much more but talk to each other much less," mostly posting status updates in threads that only they read.
Why the measuring is getting harder
The subtitle of the report, that capabilities are getting harder to measure, is the part METR seems most worried about. Its tests are cheap, automatically scored and run in containers, and those are exactly the properties AI companies want in the reinforcement learning environments they use to train models. The more training environments resemble the tests, the more a high test score may overstate how well a model handles messier real-world work.
The human side is harder too. Finding people who can beat the models now takes specialists, dozens of hours of their time and high pay. To make recruiting possible at all, METR let its human baseliners use Anthropic’s Claude Sonnet 5 for routine coding, though not a frontier model.
METR also fit a second yardstick, the METR Capability Index, which scores models directly from their results without relying on human time estimates. On that measure it sees no obvious overall speed-up or slowdown across public frontier models, with some signs of faster progress for individual developers that it calls too sensitive to methodology to trust. In its X thread, the group summed up a related caution in one line:
Time is not always a well-behaved axis of difficulty.
What it means if you use AI agents
METR’s report is aimed at risk assessment, not product advice, but a few practical lessons follow from its findings.
Give agents work with a test. The strongest results come on tasks with a clear, automatic score: code with a test suite, an optimization target, a benchmark. That is the kind of work to delegate first, whether to a chat assistant or to one of the cloud AI agent teams that now take tickets from developers.
Keep the judgment calls. Choosing what to build, deciding whether a result will hold up later and weighing tradeoffs are where METR found agents weakest. Those remain the human part of the job for now.
Don’t let a model grade a stronger model. If you use one AI to review another’s work, METR’s results suggest the reviewer will be too generous when the author is as capable or more so.
Plan for coin flips on long jobs. A 40-hour horizon at 50% success means long, unattended projects will often fail. Checkpoints and human review along the way still matter.
What METR says it still can’t answer
METR ends with open questions rather than a forecast. Among them:
- How much stronger are models on tasks that are easy to verify than on tasks that are hard or costly to verify?
- Are models getting better at hard-to-verify tasks more slowly, and if so, by how much?
- What capabilities does drastic AI research acceleration actually require, and could easy-to-verify tasks alone be enough?
There are also gaps in the data itself. METR’s results use public models released up to September, so newer releases such as OpenAI’s GPT-6.2 Astra on Nov. 17 are not in the trend line, and no Google Gemini model appears among the 15 tested. The newest points also carry the widest error bars, and METR notes that companies could be holding models back from public release for longer as they get more capable, which would hide a speed-up from outside testers.
What the report does establish is direction. In under two and a half years, the length of technical work an AI agent can finish half the time has gone from minutes to roughly a full working week, and the people best placed to measure it are saying, in careful language, that their rulers are starting to bend.
Further reading

The Coming Wave: AI, Power, and Our Future
DeepMind co-founder Mustafa Suleyman on why fast-improving AI is so hard to contain, a useful companion to METR’s warning that capabilities are outrunning the tests.

Supremacy: AI, ChatGPT, and the Race That Will Change the World
Parmy Olson’s history of the OpenAI and DeepMind rivalry, the race that produced the models METR is now measuring in hours.

Amazon Echo Dot
Amazon’s small smart speaker, built for its Alexa+ assistant, an easy way to see how far everyday AI help has come.

Logitech C920x HD Pro Webcam
A long-running 1080p webcam for the video calls and screen-recorded demos that come with working alongside AI tools.

Apple AirTag 4-Pack
Apple’s popular Bluetooth trackers, a small, practical way to keep tabs on keys, bags and gear.
