A vendor's own numbers, and a benchmark anyone can re-run.
HydraFusion is GitHub's cost-cutting router inside Copilot CLI. By GitHub's own figures it cut cost on all three of its benchmarks and held Opus 5 quality on one of the three. That's a real result, and the fact that Microsoft shipped it tells you the category matters. OmnisBench does the other half of the job: it measures how much routing efficiency is genuinely there, on fresh tasks the models can't have memorised, with every number re-gradable offline. Apache-2.0.
How they compare
| Dimension | HydraFusion (GitHub Copilot) | OmnisBench |
|---|---|---|
| Type | Cost-cutting routing feature inside Copilot CLI | Open-source benchmark you run yourself |
| Numbers come from | GitHub, on GitHub's own benchmarks | Anyone, re-graded offline from published responses |
| Works with | GitHub Copilot only | Any model pool you configure, tool-agnostic |
| Tasks | TerminalBench 2.1, DeepSWE, CheckpointBench | Fresh, contamination-controlled tasks |
| Independently verifiable | Published result, not re-runnable by you | Yes, omnisbench verify re-grades offline, zero API calls |
| License | Copilot product feature | Apache-2.0 |
Cheaper everywhere, quality held in one of three
Here are GitHub's own numbers, and they're honest about the trade-off. Cost fell on all three benchmarks. Opus 5 quality held on one.
| Benchmark | Cost vs Opus 5 | Quality vs Opus 5 |
|---|---|---|
| TerminalBench 2.1 | 67% lower | up 4.9 points |
| DeepSWE | 36% lower | down 1.5 points |
| CheckpointBench | 65% lower | down 0.1 points |
Cheaper is the easy half to prove. The hard half is what it costs you in quality, and on two of three it cost a little. That's the number a benchmark should pin down on tasks a model hasn't already seen, which is exactly what OmnisBench is for.
A vendor grading its own homework, and a check anyone can run
GitHub ran HydraFusion against its own chosen benchmarks and reported the result. That's normal, and the figures look reasonable. But a router's whole value is a cost-quality trade-off, and the honest way to know where that lands is a test the vendor doesn't own. OmnisBench publishes every model response for every task in results.json, and omnisbench verify re-runs the graders against those responses and re-derives the leaderboard offline, with zero API calls. Change one stored answer and it fails. You can check a number yourself in minutes rather than take it on trust.
HydraFusion stops at the Copilot boundary
HydraFusion only works inside GitHub Copilot CLI, billed per token on a Copilot plan. If your developers use Claude Code, Cursor, Codex, or hit the Anthropic and OpenAI APIs directly, it can't see them. OmnisBench measures routing efficiency independently of any one tool. OmnisRouter then acts on it across all of them, with a real local spend cap and a kill switch, and OmnisVigil turns the whole fleet's spend into a number a finance team can read. HydraFusion has no budgets, caps or per-team attribution. It lowers a request's price. It doesn't tell you who spent what.
Pick based on the job
HydraFusion
Your team lives entirely inside GitHub Copilot, you want a lower per-request cost with no setup, and you're happy to take GitHub's word on the trade-off.
OmnisBench
You want to know how much routing efficiency is actually available, on fresh tasks, with a number you can re-grade yourself, whatever tool the routing lives in.
OmnisBench's fresh split
On tasks the models can't have memorised, ideal routing hit 93.3% task success, above the frontier model itself, at about 60% lower cost, and every figure re-grades offline. We're putting HydraFusion through the same open test and will publish where it lands.
See the full leaderboard on the homepage →
Reproduce it on GitHub →
Read the open-source LLM routing benchmark overview →