Test 1: Reading card acquiring statements
Published34 AI models read UK card acquiring statements and had to get every fee right. Each answer was scored in pounds and weighed against the model's price per statement.
Read a statement and list every fee, with its amount.
Invented UK acquiring statements with the right answer set in advance, plus faulty ones as traps.
34 through OpenRouter, on routes with zero data retention. The 7 finalists were also run with an expert prompt.
Fees wrong as a share of fee value, against cost per statement.
What I found
- 01With the expert prompt, all seven finalists got 0.07% of fees wrong or less.
- 02DeepSeek V4.1 Flash at half a cent a statement got close to GPT-5.5 at 25 cents.
- 039 of the 34 models weren't usable. They invented fee lines, missed the traps or gave broken answers.
What would they cost you a year?
Bars show the model cost a year. The hosting and the engineering to run it come on top. Gemini 3.8 Flash rises to about $6,890 a year at 10,000 statements a month when its introductory price ends on 31 December 2026.
Results
Showing 34 of 34 models.
Every model was run with a plain prompt. The seven finalists were run again with an expert prompt, which has fuller instructions. Cost a year uses the expert prompt for finalists and the plain prompt for the rest.
| # | Model | Result | Plain prompt | Expert prompt | Cost a year | ||
|---|---|---|---|---|---|---|---|
| Fees wrong | Cost per statement | Fees wrong | Cost per statement | ||||
How these models do on the other tests is on the leaderboard, and where each one is hosted is on Prices.