I like that they're being up front about this. Benchmarks are a metric that can be gamed. They give you an approximate estimate of where things land, but you really need to test for your use case.
pietz [3 hidden]5 mins ago
Or, you know, it's an easy excuse because they're already behind on many of them.
jwpapi [3 hidden]5 mins ago
Or they use it as marketing wording to get liked by devs.., which is most likely given we know that’s typesafes usp
tyushk [3 hidden]5 mins ago
Cool to see another model with support for bounding boxes on images. Really wish to see segmentation in a future release; Gemini 3.5 Flash-Lite is mandatory in my use case as it's the best model for semantic image segmentation in my experience.
moonu [3 hidden]5 mins ago
Curious why you use Gemini 3.5 Flash Lite instead of SAM/Moondream? Any specific workflows where it outperforms?
tyushk [3 hidden]5 mins ago
Using it as part of an agentic CAD workflow, where the model needs to be able to selectively segment from parts of a user's image as it works on a project. Using OpenSCAD so being a general purpose model is very useful to me.
I used the model and it was good enough for email classification.
I was wondering if I'm supposed to be doing multiple runs for topic classification with grouping related topics i.e. if its sport,economy or tech related news and then another run for soccer,basketball or something else
or batching them like the News article topic classification example in a single run.
120bits [3 hidden]5 mins ago
Slightly unrelated, but I really like the website styling. Looks like an improved version of Linux From Scratch dos.
lukeify [3 hidden]5 mins ago
Swiss Style/Typographic Style.
hnav [3 hidden]5 mins ago
too bad it's just LARPing, underneath the covers it's the typical bloated Framer site.
yipinwong [3 hidden]5 mins ago
Sounds like an incremental change with context size change, new "box" type, and composable images.
As it's minor change did not expect much, but having a higher context helps for sure. I will own the errors for passing bigger context.
therealpygon [3 hidden]5 mins ago
Well, they do need to try to compete with SubQ’s longer context and image processing that has been coming out ahead in benchmarks, plus the thousand self-hosted options that now exist.
jofzar [3 hidden]5 mins ago
Interesting way to release this, I'm not sure how to say this, but this has a real "enterprise"/old school taste to it verses how all the other models/AI companies are releasing things.
Very refreshing.
kypro [3 hidden]5 mins ago
I'm aware this is an unproductive comment, but why are all their releases so cringe? It's like they're written by a Reddit mod or something...
> For the builders AND dreamers
> Enter the danger zone at your own risk.
> Size does matter, apparently
> We’re trying something new out here: we’re going to treat you guys like adults
> Lots of people love to ask what kind of sci-fi we’re releasing next
That said, their docs are great and don't seem to include all the Reddit cringe.
shrinks99 [3 hidden]5 mins ago
I walked away with _mostly_ the opposite impression. They have a specific tone of voice! I think that's good.
I've yet to see any company match the copy writing skills of Valve though.
hexasquid [3 hidden]5 mins ago
I'm curious what people are building with these models
gopalv [3 hidden]5 mins ago
You can turn a lot of process and workflow systems into deep decision trees.
My team hasn't finished writing their blog posts yet, but the customer demos [2] went out the moment TypeSafe took our call and sorted all the paperwork out.
There's a tiny section at the bottom of the demo which explains how it actually works (or the github repo is the code).
We have a single decision API[1] over all similar models, so we run evals across all the "System One" api products & definitely don't trust any benchmark that you haven't run. All standardized benchmarks have been benchmaxxed already.
PS: just in case it wasn't clear, the demo is a game where we play to calibrate the human vs machine decision making (or just do it in auto-pilot mode).
Automatic tagging of images is what I'm doing with it. I'm using a modified Gemma model, which ingests images (and audio, for that matter).
Rudybega [3 hidden]5 mins ago
I use it as a drop in replacement for LLMs I had previously doing filtering and classification in a pipeline. It's loads cheaper and faster with roughly equivalent performance on my own internal tests. The confidence scores (which are also reasonably well calibrated on my data) are a super nice bonus.
vanuatu [3 hidden]5 mins ago
^ Same here. so much faster and cheaper than using an LLM to return structured data
ygouzerh [3 hidden]5 mins ago
I found an example online, I cannot find the github again, but it was interesting: an instant dynamic generation of React component, you type what you want to build using natural language, and it directly add one to the page.
If I remember well, in the background, it does the multi-choice question: "What is the component requested ? --> Calendar, Slider, ..."
it's actually more usefull than it looks. it is not much useful on its own but it can play a big role in a complex workflow or custom harness.
i used luna 5.6 as the ranker/filter to remove useless items from upstream rag(mixing embedding, regex etc) and it did a good job but took 5-10 seconds each time. now less than 1 second and no observable precision loss. hope it helps
ASalazarMX [3 hidden]5 mins ago
Judging from the tilte, I guess dreams.
matt3210 [3 hidden]5 mins ago
How does JEV prevent distillation?
himata4113 [3 hidden]5 mins ago
They don't. Generally we already have trillions tokens worth of classifier datasets so there's no reason to create synthetic data that will be worse than the data it was trained on.
nico [3 hidden]5 mins ago
I don't think they can. But they probably monitor usage and test for patterns to then block/ban accounts
benaduggan [3 hidden]5 mins ago
So they held back a bunch of features because they knew they had basically no moat, and now they're trickling out a bit more functionality to try to convince people they are better than others and move fast? And using Apple advert slogans to show how cool they are?
aryabakh [3 hidden]5 mins ago
With the speed they're adding features shows you're correct.
Videos will be next.
I like that they're being up front about this. Benchmarks are a metric that can be gamed. They give you an approximate estimate of where things land, but you really need to test for your use case.
As it's minor change did not expect much, but having a higher context helps for sure. I will own the errors for passing bigger context.
Very refreshing.
> For the builders AND dreamers
> Enter the danger zone at your own risk.
> Size does matter, apparently
> We’re trying something new out here: we’re going to treat you guys like adults
> Lots of people love to ask what kind of sci-fi we’re releasing next
That said, their docs are great and don't seem to include all the Reddit cringe.
I've yet to see any company match the copy writing skills of Valve though.
My team hasn't finished writing their blog posts yet, but the customer demos [2] went out the moment TypeSafe took our call and sorted all the paperwork out.
There's a tiny section at the bottom of the demo which explains how it actually works (or the github repo is the code).
We have a single decision API[1] over all similar models, so we run evals across all the "System One" api products & definitely don't trust any benchmark that you haven't run. All standardized benchmarks have been benchmaxxed already.
PS: just in case it wasn't clear, the demo is a game where we play to calibrate the human vs machine decision making (or just do it in auto-pilot mode).
[1] - https://github.com/prasanthj/duckdb-dual-cognition
[2] - https://demos.isotopes.ai/demo/apps/chain-command
If I remember well, in the background, it does the multi-choice question: "What is the component requested ? --> Calendar, Slider, ..."
i used luna 5.6 as the ranker/filter to remove useless items from upstream rag(mixing embedding, regex etc) and it did a good job but took 5-10 seconds each time. now less than 1 second and no observable precision loss. hope it helps
I guess their cheeky "Warning: AI Benchmarks, don't trust them" is their way out of that one.