Category narrowing to this degree really has diluted the term "SOTA" quite a lot
You mean it's state of the art for this subset of benchmarks you're targeting? No? Well surely it's state of the art for models of a certain size category? Huh, what about models that are available for local inference? O, state of the art for open models of a certain size made in the US specifically. Uh. Neat!
visarga [3 hidden]5 mins ago
No, the term is well defined. It means being on the Pareto curve of performance vs size, such as: https://tinyurl.com/26sw35fw
wmf [3 hidden]5 mins ago
Congratulations I guess but it's a pretty bad look to not even mention those other open weight models.
karmakaze [3 hidden]5 mins ago
What's funnier is that after cherry picking models for comparison, they further cherry picked Inking for size comparison instead of better and smaller Beam 1.0
The bar chart also shows Nemotron 3 Ultra instead of smaller and better Nemotron 3.5 Super.
tedivm [3 hidden]5 mins ago
We've always been at war with eurasia.
Seriously though, it's just easier to pretend that China doesn't exist when they're stomping us this badly.
flatline [3 hidden]5 mins ago
Right? These are not likely to be meaningful points of comparison to anyone but the authors.
wmf [3 hidden]5 mins ago
Maybe Palantir will use it or something. There probably is a market for US models but I don't know how large it will be.
Onawa [3 hidden]5 mins ago
US Gov won't let Chinese models be hosted on any internal infra, not even supposed to use them for local AI. So there is absolutely a market for US-made or non-Chinese models there.
syntaxing [3 hidden]5 mins ago
When a post doesn’t mention Qwen for comparison, you know the performance isn’t going to be great.
ethan-boneh [3 hidden]5 mins ago
I mean Qwen 3.8 27B hits 73% on TB2.1 and 42.2% on DeepSWE....
2001zhaozhao [3 hidden]5 mins ago
This model scores 80.5 on Terminal Bench 2.1
For reference Qwen 3.8 Flash Next (which is bigger than this model but probably cheaper to run due to MoE) scores 84.3, and Qwen 3.8-27B scores 73.0.
I think this model could be competitive, depending on how much it was benchmaxxed compared to Qwen?
dofm [3 hidden]5 mins ago
So we have a Recursive and a Ricursive, both making LLMs?
wmf [3 hidden]5 mins ago
Time to start Rickursive, the lab that will never give you up and never let you down.
throwaw12 [3 hidden]5 mins ago
we need RichUrsive
giwook [3 hidden]5 mins ago
There seems to be some recursion going on here.
xyzrt [3 hidden]5 mins ago
One making self-improving AI via AI-chip co-design, the other relying only on software for self-improving AI.
dofm [3 hidden]5 mins ago
I hope they never have to call each other about anything. It could get confusing if neither knows who is the base case.
jszymborski [3 hidden]5 mins ago
Would love a 27B model. Wonder if we'll get a distillation.
You mean it's state of the art for this subset of benchmarks you're targeting? No? Well surely it's state of the art for models of a certain size category? Huh, what about models that are available for local inference? O, state of the art for open models of a certain size made in the US specifically. Uh. Neat!
The bar chart also shows Nemotron 3 Ultra instead of smaller and better Nemotron 3.5 Super.
Seriously though, it's just easier to pretend that China doesn't exist when they're stomping us this badly.
For reference Qwen 3.8 Flash Next (which is bigger than this model but probably cheaper to run due to MoE) scores 84.3, and Qwen 3.8-27B scores 73.0.
I think this model could be competitive, depending on how much it was benchmaxxed compared to Qwen?