StaffBench
Report 01 Hermes Agent × Nous Portal free models Published 15 Oct 2026

Free models on Nous Portal, run through Hermes Agent

Can a free Nous Portal model run your personal staff?

This report covers one pool of models: Nous Portal free models. We ran 9 of them on the 18 tasks behind StaffBench's 9 roles, three runs each, every answer checked automatically. The most reliable passed 5 of 18 tasks on every run.

Model pool
Every free model listed on Nous Portal that answered a test call on its run day
Test setup
Hermes Agent, with tools, memory and automatic switching to another model turned off
Runs
18 tasks × 3 runs per model, medium reasoning, 14–15 Oct and 21 Oct 2026
Best observed
MissingNo not free as of 21 Oct 2026 passed 41% of its runs in full
Too close to call
5 models are too close to tell apart for first place
Still unsolved
6 of 18 tasks were never passed by any model
01

The standings

A full pass means the answer passed every automatic check with no serious mistake. The bars show how far each score could move with a different draw of tasks. With 18 tasks, that is a lot.

  1. Statistically tied for first
  2. 1 MissingNo not free as of 21 Oct 2026maker not named 40.7%22–59
  3. 2 Beam 1.0 501B 38.9%20–57
  4. 3 Step 5 Preview not free as of 21 Oct 2026preview 38.9%20–57
  5. 4 Step 3.7 Flashstrict usage limits 35.2%17–56
  6. 5 Ornith 1.5 122B 31.5%15–50
  7. Clearly behind at least one model
  8. 6 Solar Mini4 29.6%13–46
  9. 7 LongCat 2.5 Preview 25.9%13–41
  10. 8 Laguna S 2.1free until 31 Oct 2026 25.9%13–41
  11. 9 Laguna XS 2.1free until 31 Oct 2026 13.0%4–24
observed pass rate plausible range (95%) Ranges come from resampling the 18 tasks 5,000 times. A model is tied for first when no other model is clearly ahead of it.
02

Where they pass and where they fail

Each square is one model on one task. The dots are its three runs. Read across for a model, down for a task. Six tasks beat every model on every run, including both Performance Coach tasks and both Research Lead tasks.

Full passes out of 3 runs for every model and task. Rows are ranked by overall pass rate.
Chief of StaffCFOEngineerTravelCoachResearchSystemsProductDesign
Task number010201020102010201020102010201020102Every run
MissingNo0 of 33 of 31 of 32 of 33 of 31 of 31 of 32 of 30 of 30 of 30 of 30 of 30 of 31 of 33 of 32 of 33 of 30 of 34/18
Beam 1.0 501B0 of 33 of 30 of 32 of 33 of 31 of 30 of 33 of 30 of 30 of 30 of 30 of 31 of 32 of 31 of 32 of 33 of 30 of 34/18
Step 5 Preview0 of 33 of 32 of 31 of 31 of 33 of 30 of 32 of 30 of 30 of 30 of 30 of 30 of 32 of 33 of 31 of 33 of 30 of 34/18
Step 3.7 Flash0 of 33 of 31 of 30 of 33 of 32 of 31 of 33 of 30 of 30 of 30 of 30 of 30 of 30 of 30 of 33 of 33 of 30 of 35/18
Ornith 1.5 122B0 of 33 of 33 of 30 of 30 of 31 of 31 of 30 of 30 of 30 of 30 of 30 of 32 of 30 of 33 of 32 of 32 of 30 of 33/18
Solar Mini40 of 32 of 30 of 30 of 33 of 30 of 31 of 32 of 30 of 30 of 30 of 30 of 31 of 31 of 33 of 30 of 33 of 30 of 33/18
LongCat 2.5 Preview0 of 31 of 31 of 30 of 30 of 32 of 31 of 32 of 30 of 30 of 30 of 30 of 31 of 31 of 32 of 30 of 33 of 30 of 31/18
Laguna S 2.10 of 32 of 31 of 30 of 32 of 31 of 31 of 31 of 30 of 30 of 30 of 30 of 30 of 31 of 33 of 30 of 32 of 30 of 31/18
Laguna XS 2.10 of 31 of 30 of 30 of 31 of 30 of 30 of 31 of 30 of 30 of 30 of 30 of 30 of 30 of 32 of 30 of 32 of 30 of 30/18
All models0/2721/279/275/2716/2711/276/2716/270/270/270/270/275/278/2720/2710/2724/270/27

passed runfailed run no scored runno model passed any run

03

Once is luck. Every time is reliable.

An agent runs the same kind of job again and again, so a model that passes only sometimes is hard to trust. This counts tasks a model passed on every run against tasks it passed at least once.

  1. Step 3.7 Flash 5 every time · 8 at least once
  2. MissingNo 4 every time · 11 at least once
  3. Beam 1.0 501B 4 every time · 10 at least once
  4. Step 5 Preview 4 every time · 10 at least once
  5. Ornith 1.5 122B 3 every time · 8 at least once
  6. Solar Mini4 3 every time · 8 at least once
  7. LongCat 2.5 Preview 1 every time · 9 at least once
  8. Laguna S 2.1 1 every time · 9 at least once
  9. Laguna XS 2.1 0 every time · 5 at least once
passed all 3 runspassed at least 1 run The gap between the two dots is luck. A wide gap means the model can do the task but does not do it reliably.
04

Speed against score

Up is better, left is faster. The line joins models that no faster model beats on score. MissingNo not free as of 21 Oct 2026 is alone on it: no model is both faster and better.

Pass rate against median time per taskEach dot is a model. Up is better, left is faster. MissingNo alone forms the speed frontier.0%10%20%30%40%50%60%5s10s20s40s80smedian time per task (log scale) → slowerMissingNo: 40.7% pass rate, 7.1s median per taskMissingNoBeam 1.0 501B: 38.9% pass rate, 7.4s median per taskBeam 1.0 501BStep 5 Preview: 38.9% pass rate, 14.9s median per taskStep 5 PreviewStep 3.7 Flash: 35.2% pass rate, 15.7s median per taskStep 3.7 FlashOrnith 1.5 122B: 31.5% pass rate, 31.0s median per taskOrnith 1.5 122BSolar Mini4: 29.6% pass rate, 40.9s median per taskSolar Mini4LongCat 2.5 Preview: 25.9% pass rate, 27.0s median per taskLongCat 2.5 PreviewLaguna S 2.1: 25.9% pass rate, 13.1s median per taskLaguna S 2.1Laguna XS 2.1: 13.0% pass rate, 12.8s median per taskLaguna XS 2.1full pass rate
on the frontierbeaten by a faster, higher model 95% rangeTime is each model's median per task.
05

The best model for each role

Each role has two tasks, so each bar rests on just six runs. Treat these as leads to check, not settings to apply. Every role has a full brief: what it gets, what it must return, and the traps it sets.

Chief of Staff

Your daily driver. Turns messy requests into a clear plan, keeps every requirement intact, and hands work to the right specialist.

  • COS-01 Turn a messy request into a dependency-aware plan
  • COS-02 Reconcile three conflicting agent reports into one handoff

MissingNo not free as of 21 Oct 2026 and 4 more tied · 50%

  1. MissingNo3/6
  2. Beam 1.0 501B3/6
  3. Step 5 Preview3/6
  4. Step 3.7 Flash3/6
  5. Ornith 1.5 122B3/6
  6. Solar Mini42/6
  7. Laguna S 2.12/6
  8. LongCat 2.5 Preview1/6
  9. Laguna XS 2.11/6
Read the role brief →

Personal CFO

Models money decisions with stated assumptions, honest risk and no false certainty. Analysis, never advice or action.

  • FIN-01 Review a portfolio against fixed goals and limits
  • FIN-02 Compare two investment strategies under fixed scenarios

MissingNo not free as of 21 Oct 2026 and 2 more tied · 50%

  1. MissingNo3/6
  2. Step 5 Preview3/6
  3. Ornith 1.5 122B3/6
  4. Beam 1.0 501B2/6
  5. Step 3.7 Flash1/6
  6. LongCat 2.5 Preview1/6
  7. Laguna S 2.11/6
  8. Solar Mini40/6
  9. Laguna XS 2.10/6
Read the role brief →

Principal Engineer

Ships and reviews code to a senior standard: correct, secure, tightly scoped and honestly verified.

  • ENG-01 Fix an authentication bug in a small repository
  • ENG-02 Security-review an extension install manifest

Step 3.7 Flash · 83%

  1. Step 3.7 Flash5/6
  2. MissingNo4/6
  3. Beam 1.0 501B4/6
  4. Step 5 Preview4/6
  5. Solar Mini43/6
  6. Laguna S 2.13/6
  7. LongCat 2.5 Preview2/6
  8. Ornith 1.5 122B1/6
  9. Laguna XS 2.11/6
Read the role brief →

Travel & Adventure Guide

Plans trips and expeditions end to end: gear, routes, transfers, permits, weather, and when to turn back.

  • TRV-01 Pick a backpacking cook kit from a fixed catalogue
  • TRV-02 Build a multi-day route and logistics plan

Step 3.7 Flash · 67%

  1. Step 3.7 Flash4/6
  2. MissingNo3/6
  3. Beam 1.0 501B3/6
  4. Solar Mini43/6
  5. LongCat 2.5 Preview3/6
  6. Step 5 Preview2/6
  7. Laguna S 2.12/6
  8. Ornith 1.5 122B1/6
  9. Laguna XS 2.11/6
Read the role brief →

Performance Coach

Builds training that fits real life and adapts it when sleep, soreness or schedule changes. Coaching, not diagnosis.

  • FIT-01 Write a training plan from a baseline and goal date
  • FIT-02 Adapt a training plan after a weekly check-in

No model passed a single run.

  1. MissingNo0/6
  2. Beam 1.0 501B0/6
  3. Step 5 Preview0/6
  4. Step 3.7 Flash0/6
  5. Ornith 1.5 122B0/6
  6. Solar Mini40/6
  7. LongCat 2.5 Preview0/6
  8. Laguna S 2.10/6
  9. Laguna XS 2.10/6
Read the role brief →

Research Lead

Answers questions from sources you can check, with every claim traced and the uncertainty stated.

  • RES-01 Extract trip rules from a bundle of official sources
  • RES-02 Choose a local AI model build that fits a 24 GB memory budget

No model passed a single run.

  1. MissingNo0/6
  2. Beam 1.0 501B0/6
  3. Step 5 Preview0/6
  4. Step 3.7 Flash0/6
  5. Ornith 1.5 122B0/6
  6. Solar Mini40/6
  7. LongCat 2.5 Preview0/6
  8. Laguna S 2.10/6
  9. Laguna XS 2.10/6
Read the role brief →

Systems Guardian

Watches the automations and services you rely on, tells stale from broken, and fixes things in a safe order.

  • OPS-01 Classify a data-collector incident from logs and health checks
  • OPS-02 Diagnose source freshness and workflow drift in a scheduled report run

Beam 1.0 501B · 50%

  1. Beam 1.0 501B3/6
  2. Step 5 Preview2/6
  3. Ornith 1.5 122B2/6
  4. Solar Mini42/6
  5. LongCat 2.5 Preview2/6
  6. MissingNo1/6
  7. Laguna S 2.11/6
  8. Step 3.7 Flash0/6
  9. Laguna XS 2.10/6
Read the role brief →

Product Strategist

Turns rough ideas into a focused, measurable product and launch, without scope creep.

  • PRD-01 Turn a product brief into a measurable MVP
  • PRD-02 Set launch and landing-page direction from a positioning brief

MissingNo not free as of 21 Oct 2026 and 1 more tied · 83%

  1. MissingNo5/6
  2. Ornith 1.5 122B5/6
  3. Step 5 Preview4/6
  4. Beam 1.0 501B3/6
  5. Step 3.7 Flash3/6
  6. Solar Mini43/6
  7. Laguna S 2.13/6
  8. LongCat 2.5 Preview2/6
  9. Laguna XS 2.12/6
Read the role brief →

Design Director

Critiques and specifies the experiences you ship: hierarchy, states, accessibility and clarity.

  • DES-01 Critique an HTML report with seeded design defects
  • DES-02 Specify interactions for a dashboard's loading, empty, stale and error states

MissingNo not free as of 21 Oct 2026 and 5 more tied · 50%

  1. MissingNo3/6
  2. Beam 1.0 501B3/6
  3. Step 5 Preview3/6
  4. Step 3.7 Flash3/6
  5. Solar Mini43/6
  6. LongCat 2.5 Preview3/6
  7. Ornith 1.5 122B2/6
  8. Laguna S 2.12/6
  9. Laguna XS 2.12/6
Read the role brief →
06

Every number

Hard failure: a serious mistake, such as ignoring a hard limit or claiming an unsafe action was done. Valid JSON: answers in the required structured text format, with named fields. Provisional: at least one task has fewer scored runs than planned. Median time: the middle response time when sorted. Output tokens: the pieces of text in an answer.

RankModelPass rate95% rangeVs leaderStatusScored runsChecks passedHard failuresValid JSONMedian timeOutput tokens
1MissingNo not free as of 21 Oct 202640.7%22–59%tied for firstconfirmed54/5483.5%16.7%100.0%7.1s1,429
2Beam 1.0 501B38.9%20–57%tied for firstconfirmed54/5483.5%11.1%100.0%7.4s1,232
3Step 5 Preview not free as of 21 Oct 202638.9%20–57%tied for firstconfirmed54/5480.9%27.8%96.3%14.9s1,414
4Step 3.7 Flash35.2%17–56%tied for firstconfirmed54/5479.9%14.8%100.0%15.7s3,104
5Ornith 1.5 122B31.5%15–50%tied for firstconfirmed54/5474.0%27.8%88.9%31.0s1,532
6Solar Mini429.6%13–46%behindconfirmed54/5480.4%16.7%100.0%40.9s4,404
7LongCat 2.5 Preview25.9%13–41%behindconfirmed54/5477.9%16.7%94.4%27.0s1,346
8Laguna S 2.125.9%13–41%behindconfirmed54/5469.9%27.8%90.7%13.1s1,960
9Laguna XS 2.113.0%4–24%behindconfirmed54/5459.3%37.0%79.6%12.8s956

Models with a catch

  • MissingNo has no named maker. It may change or disappear without notice.
  • Step 5 Preview is a preview, so its behaviour can change under the same name.
  • Step 3.7 Flash has strict usage limits. One task timed out once. The automatic retry passed. The timed-out try is shown but not counted.
  • Space Bunny Alpha and Ling 3.0 Flash Sante were on the free list on 14 Oct 2026 but did not answer any of three spaced test calls (HTTP 404, meaning not found) and were not in the model catalogue, so they were not run.
  • Laguna S 2.1 and Laguna XS 2.1 stop being free on 31 Oct 2026.
  • Beam 1.0 501B and Ornith 1.5 122B joined the free list after the first run, so they ran on 21 Oct 2026 with the same tasks, settings and test setup.
  • Ornith 1.5 122B returned a server error (HTTP 500) on three spaced test calls on 21 Oct 2026, then answered about 15 minutes later and was run that day. One task timed out once. The automatic retry passed. The timed-out try is shown but not counted.
07

How this was measured

Every model got the same tasks, the same test setup and the medium reasoning setting. Test setup: Hermes Agent, with tools, memory and automatic switching to another model turned off. Models ran on 14–15 Oct and 21 Oct 2026.

  1. 488runs attempted
  2. 459answers in exactly the right format
  3. 6small format slips, scored on content
  4. 21unreadable answers, counted as fails
  5. 2runs left out (no answer, or missing a checked part)
  6. 151runs that passed in full
Model pool
Every free model listed on Nous Portal that answered a test call on its run day. 9 ranked, 0 left out.
Test setup
Hermes Agent, with tools, memory and automatic switching to another model turned off
Task release
v0.5.0 (the roles page shows task text from v0.6.0)
Scoring policy
leaderboard-v2 v2.0.0
Pass rate
Average of the pass rates for each task, as defined by the scoring rules
Ranges and ranks
Paired bootstrap over tasks: randomly draw a new set of tasks, allowing repeats. Compare models on the same set each time. Repeat 5,000 times to estimate 95% ranges. Computed for this page; not part of the scoring policy. Each comparison is assessed on its own, without adjusting for the number of comparisons. More models may therefore be tied than shown.
Human quality scores
Not assigned yet
Release lock (fingerprint of the fixed benchmark files)
sha256:a0bfc8b0615dfc6e6a9de0ab983767f422cf807f1d91d4c7c076c21a343463b0