For AI teams · early access
TruthForge: fresh tests for AI agents, with exact answers
New business tasks every run, the right answer planted inside, and no AI judge. Find out what your agent really gets wrong.
What you get
- Deliverables
- A pack of fresh tasks in your kind of work, with exact answer keys and a checker, or we run them on your agent and report.
- Timing
- Free sample pack in about 5 working days. Custom families are quoted in writing.
- What we need
- What your agent does. For a private agent: an HTTPS endpoint and a test token that we generate for you.
- Example
- Sample report (PDF, labelled SAMPLE).
Why fresh tests
Public benchmarks leak into training data, and AI judges can be biased. So a score can look good while real work still goes wrong. TruthForge builds a brand-new task each time and knows the exact answer before any model sees it.
Six kinds of real business work
- Invoices against bank lines, with planted mismatches and duplicates.
- Staff rosters at the lowest legal cost, with skills, rest rules and overtime.
- Long company reports with dated facts, later corrections and look-alike distractors.
- A support agent inside a small fake shop, marked on the final state of its database.
- Contracts against company policy, after amendments and defined terms.
- Delivery vans and routes at the lowest cost, with capacity and chilled stops.
Each family has a difficulty dial, and every task gets a record of its answer made before the test runs.
What we measured
Answer keys: 3,000 of 3,000 confirmed by separate checking code. Repeats: 0 in 60,000 tasks. Difficulty: every model we tested scored lower as the level went up. For example, one open model went from 92% at level 1 to 31% at level 5.
It also found real failures in strong models: one top model solved 0 of 4 of our hardest rosters, and another reported a tool result it never ran. Test: TruthForge build test, 14 November 2026, on our own setup with small samples, so read the numbers as direction, not decimals.
New: see the public results on the AI business-work scoreboard.
Preview: a DEMO of the Model Change Alarm, which tells you when your AI model quietly gets worse at your work.
Ask for a free sample pack
Tell us what your agent does. We will send 20 fresh tasks with exact marking. Free, and no obligation.
Or email hello@avanciq.com.
Questions
How is a task 'fresh'?
Code builds every task from a new random seed, so no model can have seen it before. In our test, 60,000 generated tasks had no repeats.
Who marks the answers?
Nobody guesses. The code that builds the task knows the exact answer, and a second, separate checker confirms it. In our test, 3,000 of 3,000 answers matched.
Can we use our own kind of work?
Yes. The six task families are a start. We can build new families that look like your own workflows.
Is this a public benchmark?
No. Your tasks stay private to you, so they cannot leak into training data. We plan a separate public leaderboard on a sealed batch.