Help

Glossary

The words we use, in plain English.

Accuracy
The share of tasks a model got fully right, out of the tasks we gave it.
Fully right
Every field in the answer matches the answer key. One wrong pay line or invoice status makes the task wrong.
Fresh task
A task built by code from a new random seed for this run, so no model can have seen it before.
Answer key
The exact correct answer, worked out by code before any model replies.
95% range
The span a score could reasonably move within by chance, given the number of tasks. More tasks, narrower range.
Drift
A change in how well a model does the same kind of work over time, for example after a provider update.
Model retirement
When a provider switches a model off and asks customers to move to a newer one.
Task family
A kind of work, such as payroll, bank reconciliation or claims settlement, with its own rules and checker.
Signed results file
A file of results with a digital signature, so anyone can check it was not changed after we published it.
Alarm
Our alert when accuracy falls by a set number of points and a statistical test says the drop is unlikely to be chance.