Help
Glossary
The words we use, in plain English.
- Accuracy
- The share of tasks a model got fully right, out of the tasks we gave it.
- Fully right
- Every field in the answer matches the answer key. One wrong pay line or invoice status makes the task wrong.
- Fresh task
- A task built by code from a new random seed for this run, so no model can have seen it before.
- Answer key
- The exact correct answer, worked out by code before any model replies.
- 95% range
- The span a score could reasonably move within by chance, given the number of tasks. More tasks, narrower range.
- Drift
- A change in how well a model does the same kind of work over time, for example after a provider update.
- Model retirement
- When a provider switches a model off and asks customers to move to a newer one.
- Task family
- A kind of work, such as payroll, bank reconciliation or claims settlement, with its own rules and checker.
- Signed results file
- A file of results with a digital signature, so anyone can check it was not changed after we published it.
- Alarm
- Our alert when accuracy falls by a set number of points and a statistical test says the drop is unlikely to be chance.