What Changes When an AI Agent Works the Queue

A firm that wants an AI agent on one of its queues usually starts with a pilot. The agent works one queue, such as reconciliation breaks, for a few months alongside the team, and the firm then decides whether to roll it out. A pilot can go well and still stall at rollout if nobody has decided what the team does, who owns the agent, where the approvals sit and what gets reported each month.

This article covers those decisions and what to set up before the pilot starts. The example throughout is reconciliation breaks, and the same points apply to chargebacks, alert review and payment repairs. The reconciliation example shows how an agent works a single case, and the process map shows which other processes suit an agent.

Analysts become reviewers

Today an analyst opens a break, pulls the settlement file, searches the ledger, works out the cause and books the fix. With an agent, the analyst opens a break that's already been traced, valued and explained, with a drafted journal waiting. The job becomes checking the explanation against the records, approving, editing or rejecting it, and taking the cases the agent escalates.

That's a different skill. Reviewing well means spotting a plausible explanation that's wrong, which is harder than it sounds when most of them are right. People who know the process best make the best reviewers, so the analysts doing the work today are the right people to review it.

There's also a new job: owning the agent. Someone has to maintain its instructions, decide which tools it gets, keep a set of test cases and look at why it got things wrong. ScalePoint Research does the same work for its own model tests. For each test it keeps the instructions and a set of cases with known answers, and it publishes why each model got things wrong on the Where models fail page. In a firm, that role belongs in the finance or operations team, with a named owner. Left with IT, the agent drifts away from how the process actually runs.

Approval points are a design decision

Every agent needs a line between what it may finish on its own and what a person approves. The firm should set that line before the pilot starts, based on three things:

• Amount. Below a threshold the firm sets, the agent's fix might go straight through. Above it, a person approves.
• Type of action. Some actions should always need a person whatever the amount, such as writing off a balance, sending a claim to a provider or contacting a customer.
• Sampling. Whatever goes through without a person gets sampled, for example a fixed share each week, and the results are reported.

Start strict and loosen the thresholds as the numbers come in. The record of what was sampled and what reviewers found is also what you'll show the second line and the auditors when you ask to loosen them.

The three lines of defence still apply

Payments firms already run three lines of defence, and an agent fits into them without a new framework.

• First line: the operations or finance team owns the agent and its results, the same way it owns the process today.
• Second line: risk and compliance set the limits, meaning what the agent may read, what it may change and where approvals sit. They also test it before go-live and review the sampling.
• Third line: internal audit checks that the controls work, using the agent's audit trail of every lookup, result and draft.

The rules that sit behind this differ by market.

• UK. The FCA has said it won't add AI-specific rules and will rely on its existing frameworks, including the Consumer Duty and the Senior Managers and Certification Regime (SM&CR). In practice, the senior manager responsible for the function stays accountable for what the agent does.
• EU. DORA has applied since January 2025 and covers ICT risk, including software and AI bought from vendors, so an agent product falls under the firm's ICT third-party risk rules. The EU AI Act classes few operations agents as high-risk. Consumer credit scoring is the main exception in payments and finance, and those requirements are due to apply from December 2027.
• US. The Federal Reserve and the OCC replaced their model risk guidance, SR 11-7, in April 2026. The new guidance leaves generative and agentic AI out of scope for now, so banks apply their existing controls and third-party risk guidance to agents, and stay accountable for the results.

In each market, what changes is what the accountable senior manager needs to see each month, which is the next point.

The numbers you report change

Hours spent and cases closed no longer show how the process is doing. These are the numbers to put in front of the leadership team and the senior manager each month:

• Cost per case: model fees plus reviewer time, per case closed.
• Money got wrong: the value of fixes approved that turned out to be wrong, found through sampling and later corrections.
• Override rate: how often reviewers change or reject the agent's draft. A rising rate is the earliest sign that something has drifted.
• Escalation rate: the share of cases the agent hands to a person.
• Time to close: from break to booked fix.

Set the baseline before the pilot. Without today's cost per case and time to close, nobody can say whether the agent improved anything.

Plan for changing model or vendor

Models change every few months, and vendors reprice. The protection is to keep three things in-house whichever way you buy or build: the instructions that describe how your process works, the list of tools and data the agent may use, and a test set of past cases with known answers. With those, you can rerun the test set on a new model or a new vendor and compare the results in money before you switch. Without them, every change means rebuilding the instructions and tests from scratch, which weakens your position at renewal.

ScalePoint Research builds and maintains test sets like this for its public model tests, and runs private evaluations that help a firm design, run and keep up its own.

What happens to headcount

Be careful promising cuts. The time an agent frees up shows first as backlog cleared and volume absorbed without hiring. Whether that becomes a smaller team is a decision to make in advance, and to tell the team about before the pilot starts. If people think the pilot is about cutting their jobs, that worry will shape how they review the agent's work.

What to put in place before the pilot

• A named owner for the agent in the finance or operations team.
• Approval points by amount and by type of action, agreed with the second line.
• Today's baseline: cost per case, time to close and the error rate.
• A test set of past cases with known answers, kept by you.
• A clear message to the team about what their work becomes.