Low messes up the bicycle frame, but medium/high/xhigh/max all get the bicycle frame right.

The max one took 5 minutes 9 seconds and cost 3.3826 cents. The cheapest one (low) cost 0.0936 cents and took 7 seconds.

The most recent release of my llm-anthropic plugin queries the Anthropic model listing API directly, so I didn't have to upgrade the plugin to add support for this model:

  llm install llm-anthropic -U                                
  llm anthropic refresh
  llm -m claude-haiku-5.5 'prompt goes here'

Even more so, because in a lot of their benchmarks they use the max models. I honestly think I'd rather these labs use their xhigh models as the default for benchmarking instead since I don't think the average person is even using max.

> "What's up with the pelican?"

Well you see in the early days of LLMs we wanted a fun way to test new models, and there was this blog, ...

The pelicans all start to look the same after a while.

But seeing the comparison to other models by class, family, or historical progression gives an excellent frame of reference.

Here's the Haiku 4.5 pelican from a year ago - it sucked in comparison to Haiku 5.5: https://simonwillison.net/2025/Oct/15/claude-haiku-45/

Here's a Blender model I had Claude Opus 5.5 create: https://tools.simonwillison.net/blender-viewer?url=https%3A%...

And here's some animated pixel art by Opus 5.5: https://tools.simonwillison.net/kakapo-party

And some Monkey Island style music (Opus can compose music too): https://tools.simonwillison.net/scrimshaw-jukebox

Anthropic's models do all of this by outputting code. GPT-6 Astra has similar capabilities - I got this Blender model using that: https://tools.simonwillison.net/blender-viewer?url=https%3A%...

They are still not great at SVG. I just asked Opus and Fable to add a background to an SVG and the results were, well, not great.

This creates Sierra AGI-style adventure game scenes painted live from simple Turtle-esque drawing instructions so you can basically provide it an empty canvas and then position text labels on the canvas where you want certain things (tavern, oak tree, etc) and it will generate a custom script for rendering them in a EGA graphics style.

https://kq-styles.specr.net

Please don't judge me too harshly for this particular poop video. But here is an example of something 100% generated with claude prompts only.

https://www.youtube.com/watch?v=2EqMplbt0gU

    Input
    $0.10 / MTok for prompts up to 100,000 tokens
    $0.50 / MTok for prompts over 100,000 tokens

    Output 
    $0.50 / MTok for prompts up to 100,000 tokens
    $2.50 / MTok for prompts over 100,000 tokens

In both cases, still much cheaper than Haiku 4.5's $1 input / $5 output and these prices better compete with GPT-6 Luna. ($0.10 input / $0.50 output, but with no token threshold [EDIT: the threshold for Luna is apparently 272k])

For a while Anthropic has lacked a cost effective “cheap” LLM for summarisation, compacting, RAG helpers, etc.

These ‘ephemeral’ workloads are often under 100k tokens, or can be structured to be under 100k.

In some coding benchmarks, Haiku 5.5 beats Sonnet 5! (Especially implementation; do a well defined Jira ticket; etc), it’s really impressive how much intelligence per dollar has grown in just a few short months.

Open weights models giving a distant salute from afar

You can test with Anthropic's count_tokens endpoint or with https://crates.io/crates/tokwc

So, it is might be even worse.

Neither encode nor decode are linear in compute, so providers need to price for average expected length.

This is just getting closer to the true cost of generating tokens.

For this application 100K token input is plenty.

Of course Anthropic and OpenAI, both at $0.10/M, are still 2.5x the cost of Jev's $0.04/M.

For other tasks like summaries (another suggested usage) it's good to see Haiku and Luna now competing against each other on cost.

I'd love to know how the business automation market breaks down by volume of call type though - hard to imagine that decision making (e.g. branching, triage) isn't a very large part of it, greater than these other suggested Haiku use cases.

From OpenAI's website: Prompts with more than 272K input tokens are priced at 2x input and cache rates and 1.5x output for the full request.

Fixed.

The benchmark in the article showed it as lower per completed task than luna, but I guess we'll find out how representative that is. Anthropic has generally been fairly honest in their benchmarking though.

With that said, the real reason to use Haiku is that it's faster than all of these models. OpenRouter is showing an average so far of 93 tokens/sec, and AA got at least 137 in each of their benchmarks. So it might be valuable for speed at lower thinking levels. (At higher thinking levels, it's likely going to take longer to produce results than Sol on low/medium.)

https://artificialanalysis.ai/models/releases/comparisons/cl...

I will try the new Haiku, but it would be worthwhile if Haiku could take sane instructions and do all file editing for Opus / Sonnet / Fable then it would be worth using.

Your vibes don't appear to be supported by facts. From the announcement:

>> Claude Haiku 5.5 is priced 90% lower than Claude Haiku 4.5 for requests up to 100,000 tokens, and 50% lower for requests over 100,000 tokens. On Haiku 4.5, 90% of requests fell into the former category.

I'm asking to learn for a similar project, not to discount anything you're saying.

There are plenty of workflows like translations where you'd easily be under the cap.

And Opus 5.5 is really good.

This is a very big benefit for me. I can now ship actual ai enhanced features behind my subscription without paying extra or fully relying on on-device models. I do worry that this is to soften the blow for user-unfriendly changes

https://support.claude.com/en/articles/15036540-use-the-clau...

Being forced through the non-OSS Claude Code with all of its quirks and issues is... such an exhausting use of force by Anthropic.

To the extent that you _can_ choose to disable telemetry and training on your traces in CC, it's not all that obvious what they gain by crippling your ability to use the subscription with other – better – tools.

It's also remarkable that it's coincident with OpenAI adding "Sign in with OpenAI", so that you can use your tokens with other tools.

You might be right and they will change this in the future, but that's speculative

This text has replaced the entirety of the page called "Use the Claude Agent SDK with your Claude plan."

What more do you need?

https://code.claude.com/docs/en/headless

So, as written, yes.

This is more to encourage people to try out adding AI into their product, which is a totally different flow and experience from using AI to build the product.

That's 5-15 minutes of work at most. Not exactly the type of lock-in the parent is implying.

Of course, they don't do this out of pure kindness, but I really struggle to see a negative for subscribers already using a Claude Max subscription, especially given changing to another model is essentially frictionless via OpenRouter.

Compared with "Sign in via OpenAI" which they just announced, this is far less lock-in for anyone hosting services but less interesting for users of said services. With Anthropics approach, you can just use the allowance on your users however you see fit along with any other models and once it's used up, you can still just decide not to use their models for the remainder. With users bringing their tokens meanwhile, there is less flexibility in terms of switching for you, though might be cheaper for users.

Both interesting, each approaching this from a very different direction, each having their own trade-offs. On the OpenAI front, will be interesting whether developers can set specific temp, reasoning budgets, etc. for such "provided tokens" or whether OpenAI exposes that only via the actual API.

[0] https://www.anthropic.com/legal/credit-terms

Anthropic isn't even close to being this useful.

Biggest loss is that Ant models look like they are genuinely better.

This changes on a weekly basis, I ended up with subscriptions to most of the providers (except for X.ai).

> Claude Haiku 5.5 is priced 90% lower than Claude Haiku 4.5 for requests up to 100,000 tokens, and 50% lower for requests over 100,000 tokens

Haiku and Luna now have the exact same price up to 100,000 tokens. Luna is now cheaper for anything after 100,000 tokens, even after Luna's own price increases at 270,000 it's still less than Haiku.

So it sounds like they've directly addressed that problem. Their self-reported benchmarks are all higher than Luna too.

Good to know that is going back to being an actual option from perf/price perspective.

Haiku 5.5: https://html.non.io/lcars-haiku-5.5/

Opus 5.5 for comparison: https://html.non.io/lcars-opus-5.5

Designs it was building from: https://diffui.ai/app/canvas/5093e689-1e74-4f26-b632-2a4500f...

One interesting thing is it took a look at the job at hand, and immediately delegated it to Opus 5.5. It at least knows what it isn't good at. Very fast though, and likely best used for small subagent tasks / tightly scoped work.

Considering the price, no model comes close to being as good as this. However, it did take an extremely long time.

TIME 19m COST $0.16 https://jonclegg.github.io/pacman-bakeoff/#claude-haiku-5-5

All results: https://jonclegg.github.io/pacman-bakeoff/

Hasn't this always been the case with Haiku?

It's meant to be a good test, not a good design.

Anthropic has really been doing all the right things in the past few weeks, while OpenAI continues to fumble the bag.

https://support.claude.com/en/articles/15036540-use-the-clau...

That's not a good sign for Conductor...

Wait what? This has gotten their blessing?

  - https://blog.chrislewis.au/using-coding-agents-to-decompile-nintendo-64-games/
  - https://blog.chrislewis.au/the-long-tail-of-llm-assisted-decompilation/

I could now one-shot a new game, yeah.

Like could total war become a browser game?

They often ship the original assets in a somewhat brazen disregard for basic copyright law even when the games are still for sale on places like GOG though.

As such, I do not need to even reach for Haiku, and 4.5 was so inaccurate that it often cost more to do so in the past. Sonnet 5.5/low has been good for this kind of thing, and i didn't even touch thinking tokens or any of that. Opus 5.5 low for questions/repros, medium for implementation, basically never reaching for anything above that anymore. 5.5 has been great, so I'll try Haiku, but don't see myself going out of my way to integrate it.

Both evals and Human pairwise tests for our use case are giving Haiku 4.5 first place in pretty much all tests.

No we'll try understand if we need to change our prompts to match performance ...

edit: maybe this will help: https://platform.claude.com/docs/en/build-with-claude/prompt...

You'll know when the trend stops -> when the intelligence differential between smaller models like 7B starts to grow instead of shrink from 32B models -> that means 7B is getting about as smart as it can get. Then, 32B will follow next, then 70B, etc etc.

We haven't yet seen that at any size AFAIK.

I wouldn't be surprised if less than 1B param equivalent of our brain deals with solving math and writing computer programs and physics and all the things we tend to associate with "intelligence" - especially if you ultra optimized for that, I doubt our brain works like that.

Dealing with the real world, I highly highly doubt it.

Given that humans learn to talk while having encountered a measly number of word instances, and, given enough time, we should always be able to improve on the lottery that is biology, it does seems fairly likely.

> The cost of achieving a given level of AI performance has fallen about 47% per quarter since 2023, or 13× per year. [0]

0. https://epoch.ai/publications/the-plunging-price-of-thought

AI gets cheaper, people use it everywhere. Google searches, for example. Now we want to crack math problems and spend weeks with unreleased models.

If you used GPT-2, it'd be incredibly cheap. You basically can't use it for anything and it's simple to serve.

serious bit: if you think about how these smaller models work, at the end of the day it seems that they are now capable of forgetting useless information because they're able to derive it in reasoning allowing models to become smaller at the cost of requiring more reasoning tokens to solve a task.

Opus 5.5 runs 117 tps average on Openrouter, so it must be at least 10-20 tps slower for them to mention. IDK why they mention this as it does not help for marketing though. https://openrouter.ai/anthropic/claude-opus-5.5

This section makes the reader think: why would I not pick Sonnet 5.5 instead of Haiku 5.5?

They’re definitely planning to make the subscriptions API based so they can charge you full price.

Was a big fan of Haiku 4.5, though understand why for most Sonnet was the far better option back then.

This is kind of nuts

I also have an "ask" script that I use daily to ask simple stuff, it can access websearch and webfetch, it's more than enough to parse logs, ask for commands, quick research on the internet, small stuff. https://github.com/mariocesar/dotfiles/blob/main/common/.loc...

I use haiku for things that needs to be quick, have really clear instructions.

Of course it’s not as big, and hence falls-off quicker. I’d consider the 100k a “promotional price” to match Luna’s token pricing while delivering noticeably more intelligence.

used <10% of my 5hr limit on a $100 codex plan.

Planning on doing flash analyses of PRs that impact evals in some way, and then post comments on GitHub whenever there’s flaws in them

( https://evalship.com )

I've been using Luna, but I'll probably switch to Haiku.

I ask:

> how many r's in diminished

It answers:

> Diminished has 1 r.

This is more expensive, but it also looks like it's better enough that it's far more useful.

I also won't be surprised if you look at cost per completed task + wall clock time that it comes out ahead for the majority of what you'd want to actually use it for.

Luna will still be a great option for doing non-engineering tasks super cheaply.

For prompts over 100k tokens it's 5 times more expensive - $0.50 in, $2.50 out.

Did anyone read this? We get free API credits on some plans now

She said she was using Haiku 4.5 because she was advised to be careful with the spending.

I hate that model so much lol.

(sadly Mistral Large 4 isn't up to par - but Mistral serves GLM at 130 tps!)