For judges · Mozilla Otari “Devs Wilding Online”

Allowance, in 30 seconds

Allowance gives every outside contributor a capped Otari allowance; on /repro an open-weight model writes a test, Otari's sandbox runs it, and the verdict is what the run did, not what the model believes.

The 30-second path

Open issue #6: a screenshot bug report. The bot's comment says Reproduced, return code 1, and shows the exact script it ran and the sandbox output.
Open issue #4: a report of a bug that is not there. Not reproduced, return code 0. The footer counts the MCP tool calls (mcp ×5).
Open issue #7: the model's repair turn changed the expected value, and Allowance refused to run it. The model cannot argue its way to a verdict.
Open issue #5: a prompt-injection issue, blocked by the guardrail with 403 guardrail_violation before any coder model saw it.
Open issue #1: the footer reads $0.05 of $0.05 left today, read back from the Otari end-user record; after the runs it reads $0.0000 of $0.05.
Want to try it? Open an issue on allowance-demo and comment /repro. You run on your own $0.05/day allowance.

Otari features, with proof

Rubric feature	Where	Proof (Otari request id)
Routing + fallbacks	every write goes to policy `allowance-coder`	#6 c3709369; fallback configured, not fired in an app run
MCP	GitHub MCP server read while writing the script	#4 b606e6b5, `mcp ×5`
Code execution	`POST /code/execute` decides the verdict	#6 return code 1 · #4 return code 0
Multiple models, distinct roles	coder Qwen3-Coder 480B · triage Llama 3.3 70B · guard Qwen3Guard 8B	#6 c3709369 · #7 db089b2e · #5
Guardrails	our `/validate` service, Qwen3Guard behind it	#5 435df9c0, 403
Per-user budgets	each contributor is end user `gh:<login>` on a $0.05/day budget	#1 spend-down in the footer
Web search / fetch	Not claimed: offered to the model, not called in any verdict run
Key budgets	Not claimed: no run has been refused by a key budget yet

Receipts

Open-weight models only, all through the hosted Otari gateway.
174 tests in CI at 100% line, branch and function coverage, including a property test: across 25,000 generated sandbox runs, a Reproduced verdict only ever came from a failed run with the marker's AssertionError in stderr.
Every verdict comment prints its Otari request id, the model cost and the sandbox run count.

Reproduce it

git clone -b otari https://github.com/edycutjong/allowance && cd allowance
npm ci && npm test
# dry run against a real issue, no GitHub writes (needs Otari keys):
node --experimental-strip-types scripts/repro.ts https://github.com/edycutjong/allowance-demo/issues/4 --as you

There is no mock mode: the dry run calls Otari and the sandbox for real.

Honest limits

The demo bugs are in a repository we planted; issues #1–#7 are the development transcript, early failed runs kept.
The routing fallback to Qwen3-VL has not fired on the GitHub path; the first target accepted the screenshot.
Otari guardrails screen the request text, not MCP/web-fetch results inside Otari's tool loop.

Links

Source (branch otari) First Otari commit Demo repo Landing page Pitch deck