01
Reports
A report is one comparison: one pool of models, one test setup, one set of run dates. Compare models within a report; results from different benchmark versions are not directly comparable.
- 02 23 Oct 2026 · 7 models · 18 tasks × 3 runs Hermes Agent × Nous Portal free models · v0.6.0 Can a free model run your personal staff? Beam 1.0 501B leads at 81%, but 5 models are too close to tell apart, and every task was passed at least once. Read report →
- 01 15 Oct 2026 · 9 models · 18 tasks × 3 runs Hermes Agent × Nous Portal free models Can a free Nous Portal model run your personal staff? MissingNo not free as of 21 Oct 2026 leads at 41%, but 5 models are too close to tell apart, and 6 tasks beat every model. Read report →
- 03 Next Another model pool Each report compares one pool of models, such as leading paid models or models run on your own computer, on the same tasks.
02
How it works
The same rules apply in every report, so a model is judged on the work rather than on luck with tools.
- Fixed tasks, invented data. Two tasks per role, built from made-up people, numbers, logs and files.
- Offline. No web and no tools. Everything the model needs is in the task.
- Checked, not judged by feel. Automatic checks recalculate totals, match IDs and look for planted problems.
- Instant fails. Serious mistakes, such as claiming an action was taken, fail the run.
- Three runs each. Reports show how often a model passes every time, not just once.
- Honest uncertainty. Scores carry ranges, and models that cannot be told apart are shown as tied.