Research / Rankings
Rankings
Each test ranks the models by full cost per task: the model's price, the time a person spends checking its work and the mistakes that still get through. Each model is run the cheapest way its accuracy allows.
Ranked on the default assumptions: 10,000 statements a month, $55 an hour and 10% of fee errors becoming a real loss. Every model is ranked on the plain prompt, round 1, so all are compared on the same instructions. Models within 0.05 points of each other on fees wrong are treated as tied. To use your own, see the Test 1 page.
| Rank | Model | How it runs | Fees wrong | Full cost per statement | Cost a year |
|---|
How the ranking works
Each model is costed three ways: used as is, with spot checks on 1 task in 10, or with a person checking every task. A way is only allowed if the model's error rate is under that way's threshold, and the model is ranked on the cheapest way it's allowed. A model that's wrong too often for any of them is listed as not acceptable.
Full costs aren't added across tests, because a statement and a reconciliation aren't the same unit. Once two tests are published, an overall view will cost a typical firm's mix of tasks for a year. The full method is on the method page.