HN.zip

Show HN: Smart model routing directly in Claude, Codex and Cursor

We built a model router that plugs into coding agents (e.g. Claude Code, Codex, Cursor, etc.) and intelligently sends requests to the best model to serve them. Here's a quick demo of running it locally: https://www.youtube.com/watch?v=isKhAyivtfM.At Weave, we write ~all our code with AI, and it's been getting more expensive. This came to a head when Opus 4.7 was released and, thanks to its tokenizer changes, our costs shot up. We knew we didn't need Opus for everything but we didn't want to lose out on the intelligence for the cases where you really need it. So we decided to build a model router to handle this for us.The Weave Router acts as an Anthropic/OpenAI endpoint specifically for coding agents. It looks at every inference request and intelligently (more on that in a sec) decides what model to send it to, handling all the translations required along the way. So it can use faster/cheaper models (e.g. DeepSeek v4, GLM 5.2, Kimi K2.6) when possible, and frontier models (Opus 4.8 & GPT 5.5 (& Fable whenever it's back)) when necessary.How do we know what model to route to? We trained an RL model on tens of thousands (so far!) of agent traces. We reward the routing model when it selects an LLM that successfully completes the given task.Here's an example: if you ask the router to plan a complex change, it will (probably) route that request to Opus 4.8. Subagents exploring the codebase to gather context will be routed to more suitable models (e.g. DeepSeek V4 Flash). Then when you have the plan ready to implement, it will be (most likely) be handed to a quicker model (e.g. GLM 5.2) to carry it out.We've been using this internally for the last month or so. We've saved 40% on tokens vs. what we otherwise would have paid, with no noticeable differences in quality or velocity.The router is source-available under Elastic License 2.0, so you can self-host it. Or if you prefer, you can also use our hosted version: weaverouter.com.I'll be here to answer any questions you may have!

37 points by adchurch - 11 comments

11 Comments

g00k [3 hidden]5 mins ago
Man, I'm not so sure if I'd use something like this because the way I prompt already changes based upon what model I am using. I'm not convinced it would route to the right model based on my diction or whatever.
stpedgwdgfhgdd [3 hidden]5 mins ago
The thing I do not get with these routers is that you will have more cache misses (5min ttl). And if there is one thing i’ve learned; using the cache is crucial.

How does this router translate to $$$ when developing?

adchurch [3 hidden]5 mins ago
You're right and that's why we built the router to be cache aware! Once it starts using one model, the threshold to switch to another model will be higher because the additional cost of the cache miss needs to be worth the cost savings or quality increase.

This is the key thing that other routers we've seen miss: they're stateless so for a coding agent use case you end up spending more money due to all the cache misses.

spqw [3 hidden]5 mins ago
This + making sure common requests are saved as reusable skills and scripts would probably save a large part of my token usage

As prices increase we will see more of these tools to optimise and make the best use of token budget

debarshri [3 hidden]5 mins ago
It is funny. We are building something similar.
_pdp_ [3 hidden]5 mins ago
Cool.. but I still don't get how this is going to save money. It seems to me that it might actually burn more money just because the whole system now seems to be coming from different LLMs.

Also, small LLMs are prone to stop before completion, throw errors and produce loops. Is this factored in the design of the tool? I am not sure.

edit: spellcheck

adchurch [3 hidden]5 mins ago
It saves money because some agent sessions can be entirely handled by a smaller model (also relevant: subagents use fresh context windows so a subagent with a simple task can be routed to a smaller model even if the main agent needs a frontier model).

Totally right about small LLMs btw, that's why we trained this on real agent sessions where we forced it to use different models. If the routing model sees small models can't handle a certain type of task then they won't be assigned. (Also as a fallback we have some guardrails that will have a bigger model come in to "rescue" a smaller model if it gets stuck)

arendtio [3 hidden]5 mins ago
What is the difference from Cursors 'auto' mode?
adchurch [3 hidden]5 mins ago
Fun fact: Cursor's "auto" mode is just Composer (or at least it was last time I checked). So it's different in the sense that it actually does route to more than 1 model
ai_slop_hater [3 hidden]5 mins ago
Isn't this more expensive than always using the same model, since, as I understand, by routing to different models you give up on cache?
adchurch [3 hidden]5 mins ago
If you statelessly route each new request: yes it does end up being more expensive!

So our routing is cache-aware. It will have a much higher threshold to switch from one model to another if there's already some cache for the first model. Experimentally this solves the problem (like I said we've saved 40% ourselves vs. what we would have otherwise paid).