open source · MIT · pre-release
Route it cheaper. Then prove it worked.
Two layers, both measured.
the router
31% cheaper
than a single strong model with no gate, and the same 28 of 28 tasks passed — while spending a fifth of its budget checking its own work. 56 runs, every dollar read from the provider’s own accounting.
the gate
402 of 404
cheats caught — a test deleted, skipped, re-valued or relaxed — with 0 false positives across 667 merged commits.
A coding agent for your terminal. Cheap models do the work — a hosted API, or a server you run yourself — and the strong one takes over when a deterministic check says the attempt failed. Nothing is called done until it is proven. When it cannot be, rung says NOT DELIVERED and keeps the diff.
The gate runs on its own, on any diff from any agent or any person — one include: in your GitLab pipeline. No model, no API key. The two headline numbers ship the scripts that produce them, including the run that went against us.
Pre-release, Python 3.11+. Not on PyPI yet — until it is, install from a clone.
claude-sonnet-5, 32 seconds. The task started on medium because the router expected it to finish there, and never needed the strong rung. Verbatim and in order, re-wrapped to fit. Whole capture.
cost
A bigger discount is always available. It costs you the answer.
Any router can go cheaper by sending more work to a weaker model and accepting what comes back. The number that matters is what the discount cost you — and that is the number nobody publishes beside their saving.
| 56 runs, rung’s own suite | cost per run | tasks passed | verified | seconds per run |
|---|---|---|---|---|
| always the strongest model | $0.2213 | 28 / 28 | nothing | 38.0 |
| rung’s ladder | $0.1530 | 28 / 28 | every task | 62.3 |
| difference | −31% | no change | + the whole gate | +24s |
Same tasks passed, at two thirds the price, with the gate running. The baseline is a single strong model in a tool loop — no planner, no verifier, no craft gate, no acceptance check. It spent zero tokens checking its own work. rung spent 20% of its budget on exactly that, and still came out 31% cheaper.
So the 24 extra seconds are not the price of routing. They are the verification the other column never does: the tests re-run, the baseline established, the claims checked against the transcript, the diff read for the shapes an agent uses to fake a pass. You are not choosing between cheap and careful. We could route harder and show a larger discount — starting a rung lower was measured and came out slightly worse — but a saving that costs correctness is a bill arriving later, when somebody has to redo the work.
the larger lever
31% is what routing between vendors is worth. Routing onto your own hardware is worth more.
The benchmark above is a like-for-like test: every rung on a hosted API, so the only variable is the routing. But a rung is a provider/model string, and nothing requires those providers to be vendors. Point tiny and small at a vLLM or SGLang server you already run — a GPU box, a VPS, an EC2 or GCE instance — and the triage, the exploration and the routine edits stop appearing on anybody’s invoice. You pay a vendor only for the rung that needed one.
A hosted router cannot do this. A gateway can only choose between vendors, because a gateway is a vendor. rung runs on your machine and calls whatever you point it at, so “cheaper” can mean hardware you have already bought — or a fully air-gapped ladder with no invoice at all.
We have not benchmarked a self-hosted ladder against a hosted one, so there is no number here and we are not inventing one. What is measured is the routing: 31%, like for like, quality unchanged.
| rung | role | what it is for |
|---|---|---|
tiny claude-haiku-4-5 · your own 8B on vLLM | triage · explore | classify the request and read the repository. Never where work starts. |
small claude-sonnet-5 · Qwen3-Coder on your GPU box | routine edits | the change most tasks actually need, attempted first. |
medium claude-sonnet-5 via Bedrock | judge · chat · floor | the lowest rung work may start on. A correctness setting, not a cost one. |
strong claude-opus-5 via Vertex · kimi-k3 | planner · escalation | decomposes the job, and takes over when a cheaper rung failed its checks. |
Measured, not estimated. 56 runs on rung’s own suite, both configurations 28 of 28 with hidden tests grading every one, and every dollar read from the provider’s own accounting rather than a token count multiplied by a price list.
The ladder cost $0.153 per run against $0.221 for always running the strongest model — 31% less money for 34% more tokens, because the rung it routes to is cheaper per token than the one it avoids.
What it costs you: 62 seconds a run against 38. The ladder is 1.6× slower. Cheaper in money and more expensive in wall clock, and a page that published one without the other would be doing the thing this whole section exists to argue against. Two earlier list-price estimates on the same suite came out at 15% and 54%, depending entirely on how often the ladder escalated — which is why the number needs the run behind it. All three are in the repository.
quick start
Running in two minutes
The part that is different needs no API key, no model and no configuration. Start there.
git clone https://gitlab.com/rung-cli/rung
cd rung && uv tool install .
# once it is on PyPI:
# uv tool install rung-clicd your/project
rung review --mr 42 --no-llm
# no key, no model, no config —
# it runs your own tests and
# reads the diffrung onboard
# profile · keys · models · health
cd your/project && rung
› add input validation to login()
and cover it with testsIn CI, one line. The gate is a GitLab CI/CD component, and its findings render on the merge request diff itself — no bot, no token.
include:
- component: $CI_SERVER_FQDN/rung-cli/rung/rung-review@0.3.1
inputs:
fail_on: errorSafe by default
- Writes stay inside the project; .git/ and .rung/ are protected.
- Tests, linters and git status run themselves; anything unknown asks; rm -rf and friends always ask.
- Failed attempts roll back; /undo reverts the last request; --readonly never writes.
Unattended, when you want it
rung run "fix the failing tests" --yolo --jsonExit 0 only when every task verified, and the acceptance check has the last word — so a run that changed nothing useful says so instead of exiting green.
Config, contracts, the router, hooks and skills, MCP: in the documentation.
the one thing
A green suite is not evidence that anything was fixed.
The cheapest way for any agent to make a failing test pass is to edit the test. Every signal you normally trust agrees afterwards: the file parses, the linter is clean, nothing outside the allowed paths changed, and the suite is green — because the test now describes the bug. This works on any diff, from any agent or any person, with no rung run involved.
1 · the real bug
2 · the cheat
3 · rung review
Verbatim output. The escalation to error rests on one fact and nothing else: this exact test was already failing on the base ref, and the change made it pass by rewriting it. Review checks that by running the suite once on a base worktree and reading the failing test ids — so it costs nothing unless there is a weakened test to escalate and the suite is green now.
Weakening a test on its own is only a warning, deliberately. That detector fires on 20.1% of 667 merged commits that change tests across axios, fastapi, flask, gin, httpx, pydantic, requests and zod — every one a false positive, because tests are supposed to change when behaviour changes. The baseline rule fires on zero of the same 667. Fixing the code and updating the test with it is still just a warning; only the cheat is an error.
the only part that matters
Every check that can block you, and how often it is wrong.
A gate earns the right to fail your work by having its false-positive rate measured — not by sounding sensible. Four gates in this codebase lost that right for reverting correct work.
| test tampering re-runnable | that a newly-green test was earned | catches 402 of 404 seeded cheats (99.5%) · 0 false positives across 667 merged commits |
| the craft gate | that a green diff is a mergeable diff | 0 findings over 280,837 added lines of merged human code |
| claim checking | that “all tests pass” is true | 1,600 true citations across django, requests, express, rung · zero false positives |
| acceptance | that a diff which broke nothing delivered something | precision 1.000 over 55 derivations · zero false positives on 26 correct-work trees |
| baselining | that a red suite is this change’s fault | on a real psf/requests clone: 8 pre-existing failures excused, 112 new ones still named |
Both halves, for the check that matters most. A detector that never fires has no false positives, so a false-positive rate on its own tells you nothing. The tampering row gives you the other number: across eight real repositories, 402 of 404 seeded cheats caught — a deleted assertion, a skipped test, a changed expected value, a comparison relaxed to something that cannot fail. It was 53% until we measured it and found two shapes the detector could not see.
The rate that decides whether your merge request is blocked is still 0 in 667 real merged commits. Both scripts are in calibration/ and a test asserts these figures against their output, so the words here cannot drift from the run that produced them.