Research · Model tests · October 2026
Nine Skills for Testing AI Models in Payments and Finance Operations
Test 1 asked mostly for the two easiest skills, so it says little about which model to trust with harder work. Here is the full list, and where each test fits.
- A test only separates AI models if the task asks for something some of them can't do.
- Test 1 asked for extraction and classification. Two models got almost every fee right, and several cheap ones came close.
- The one part of Test 1 that needed judgement, spotting the two faulty statements, is where the best models pulled ahead.
- Test 2 adds reasoning across sources and long context. Later tests add rule application, calculation and judgement.
In Test 1, 34 AI models read UK card acquiring statements and had to list every fee with its amount and type. With the same plain prompt for every model, Grok 4.7 and Claude Opus 5.5 got almost every fee right, and models costing 1 to 3 cents a statement, such as Gemini 3.8 Flash and GLM 5.3, got under 0.5% of fees wrong.
That's a useful result for anyone reading statements at volume. It doesn't say much about which model to trust with harder work, because the task didn't ask for much. Reading a statement is mostly finding the lines and putting each one in the right box.
The nine skills
Payments and finance work asks a model for a mix of skills. I use nine to plan the tests, from the most mechanical to the ones that need the most judgement.
| Skill | What it means in payments work | Tested in |
|---|---|---|
| 1. Extraction | Finding the data in a document, such as the fee lines and amounts on a statement | Test 1, Test 2 |
| 2. Classification | Putting each item under the right type, such as a non-UK card surcharge rather than a penalty | Test 1 |
| 3. Rule application | Applying a contract term or scheme rule, such as checking a fee against the contracted rate | Later tests |
| 4. Reasoning across sources | Tying several documents together, such as a settlement file, the bank lines and the ledger | Test 2 |
| 5. Calculation set-up | Building the right calculation, such as the blended rate on a pricing deal | Later tests |
| 6. Judgement | Deciding what matters, such as which difference is a real error and which will clear by itself | Later tests |
| 7. Long context | Working across a lot of material at once, such as a month of files | Test 2 |
| 8. Tool use and planning | Looking things up step by step in other systems | Not planned for now |
| 9. Staying within authority | Not changing what it isn’t allowed to change | Not planned for now |
The last two matter most for AI agents, which work a case step by step in a firm's own systems. ScalePoint Research has dropped agent tests for now, so those two wait.
What Test 1 showed about the skills
Every usable model found the fee lines and read the amounts correctly, so extraction wasn't the problem. The mistakes were classification, mostly higher charges on non-UK and commercial cards filed as penalties. A payments analyst learns those calls on the job, and the models that made them correctly were the ones that stood out.
Two of the statements were faulty: one had a subtotal that didn't add up. Spotting that is a small piece of judgement, because the model has to check the statement's own sums rather than copy them out. Claude Opus 5.5, Grok 4.7, Gemini 3.8 Flash, GLM 5.3 and DeepSeek V4.1 Flash flagged both faulty statements. Mistral Large 3 and GPT-5.5 flagged neither.
What changes in Test 2
Test 2 gives each model a month of an invented merchant group's settlement files, bank lines, ledger and acquirer contract. The model has to find and value every difference. That adds reasoning across sources and long context. It also asks the model to leave timing differences alone, such as a card sale on 31 October that the acquirer settles on 2 November, which is a first step towards judgement.
Later tests move further down the list, to applying contract terms, setting up a pricing calculation and deciding which alerts are real.
What this means when choosing a model
A model that does well on a reading task has shown it can read. It hasn't shown it can apply a rule or make a call. Hence a business should test each model on the skills its own task needs, and a price gap measured on an easy task shouldn't be assumed to hold on a harder one.