The LLMs themselves are capable of doing some aspects of chip design as evinced by the K3 press release.

Furthermore, the frontier models are "good enough" for a wide swathe of tasks and will soon hit that threshold for a good amount of software engineering (if not already). Does anyone think we need a Mythos level model to plan a road trip, or give someone tips on making a cake recipe?

A Fable 5 model running at 9,000 tokens/s on an ASIC rather than 150 tokens/s on electricity chugging Nvidia GPUs, or even giant SRAM Cerebras or Groq chips could be good enough to meet the majority of demand.

Furthermore, if you're an enterprise the risk of data exfiltration and feeding data to a potential competitor like OpenAI or Anthropic is greatly reduced if you could shift to on-prem ASIC deployments. A handful of chips could cover a wide variety of use cases and cover them more securely. There are a lot of corporate use-cases for LLMs that are not frontier math research or coding.

So I guess it depends on how much the latest-greatest model motivates people, and my read on the current churn is that developers are extremely unloyal to brand at this point and will jump to whoever has the best model. And as long as the best model is running on programmable GPUs, that will be the dominant form.

Custom AI for things like facial recognition in cameras has existed for decades, before LLMs were a thing. I don't see that getting replaced. And on-device conversational intelligence might go that route as well, we'll have to wait and see. It's a lot of silicon to dedicated to a static non-changing thing. My money would be on programmable TPU-like things (Apple's NPU kind of stuff). It just seems more flexible to have an array of compute that you can load different models into, so you can update it, etc.

What about all the things you don't currently use an LLM for?

If a specialized chip can run a model 100 times faster, you can suddenly use it for a lot of things at sub-second latency. You can write "make white transparent and add a red outline to x.png" instead of the corresponding imagemagick invocation and perceive little to no latency difference. You can hook it up to your browser and have it yank out all advertisements live, or tell it to highlight anything that might interest you, again, live. There's probably thousands of latent use cases nobody has thought of that would be enabled by a truly fast LLM, even a mediocre one.

I don't think an on-device model needs to change much; it's already quite general in its capabilities.

AI: "I'm sorry, a security guardrail prevents me from performing this operation."

Your company will decide what makes economical sense.

If you can have only one AI processor in your laptop (because they're big and expensive), it's going to be a GPU. This AI processor needs to inference LLMs, audio processing, image generation, video generation, etc. This is on top of normal graphics processing requirements such as video games, playing videos, decoding, encoding, etc.

At the enterprise level, I can see some ASICs working once the market fully matures and improvements in architectures slow down drastically while demand for inference increases drastically. How far are we from this world? Maybe 5-10 years? It seems like model architectures are still changing rapidly and labs want fast experimentation that programmable GPUs offer.

GPUs will still dominate in general - just like how CPUs still dominate despite ASICs.

They are fast, but they're still programmable accelerators, not a model burned into the gates.

If they designed this right, it means that once they have a model, so long as they keep the hyperparameters fixed they can change the weights much faster than it takes to spin up a completely new chip, essentially at a cost of doing a minor revision.

It takes about 18 months to go through the design, verification, and manufacturing process if you move at breakneck pace. Design could probably be sped up.

About 18 months ago the top model was GPT-4o. Not great by today's standards, but still good enough for many tasks (certainly a big chunk of chatbot queries). The current SOTA covers far more use cases, but importantly at a level that surpasses many thresholds of utility.

Maybe, but what is the shelf-life of that 18 month decision? Barely good enough today, when it launches, starts to get worse and worse every month going forward. You have to recuperate that investment on your depreciating asset.

And you are competing against anyone with the foresight to use a TPU instead, and the benefit from any new paper that finds how to distill, quantize or whatever better so their solution gets algorithmic boosts while you are locked in.

One could also imagine hybrid models, where part of the model is burned into ASICs and part of the model exists in VRAM/HBM2 so it can be updated.

I don't have enough low-level knowledge to evaluate the technical or economic feasibility of the above ideas, however.

Sometimes you need speed; sometimes you need quality. There are very different use cases for each. For my own workflows, I sometimes want something very simple done ASAP; other workflows need "subjective" reasoning and careful crafting of responses.

If you let a model spin too much, you get worse results... that's a fact, even for the best models. Nothing you've stated since actually refutes that and acting like a paid bot doesn't mean anything.

I mean, if you want to felate Xi Jinping, have at it... that's all on you bro. I don't have a horse in this race.

Yesterday there's a news on a breakthrough for probabilistic computer with 1 million p-bits [1].

Since LLM is stochastic in nature, this type of new computer can be much better than ASIC for processing LLM data.

[1] Biggest Probabilistic Computer Turns Noise into Answers:

https://news.ycombinator.com/item?id=48971938

This is not like a bayes model or something were it's distributions all the way down.

Many AI uses are not that volatile. I had a 20 minute conversation today with some company's AI phone assistant. It was extremely good and would have been very helpful if any of the dozen people it tried to route me to would have picked up their phone. That AI won't need to be upgraded for a very long time. There is no reason for it to have a cloud brain except to force a recurring revenue for the company selling it.

Hardcore gamers are constantly throwing down insane money on the latest hardware. The rest of us can get by for a couple years with whatever we bought when the last one broke. Yeah, it's not the latest, but it gets the job done. I wonder if AI has not already reached the point where a gen 10 CPU--uh, I mean a v3 AI model--will get the job done for the next year. If I really need the up-to-the-second latest abilities for a minute, I can fallback to a cloud brain @ 1M tokens/$. Why pay a monthly lease on a 5-year plan for a 4-door Ford Ranger as your daily commuter? Buy a Clio and rent an F-250 twice a year when you need the hauling/towing capabilities.

Assuming such work was happening.

[1] https://finance.yahoo.com/technology/ai/articles/google-plan...

Imagine if Anthropic could give effectively unlimited access to Sonnet, for $20. Wouldn’t that be an appealing option for many users? I know I’d make a lot of use of it for agentic tasks, office work, summarization, etc; when right now I’d save quota for more important tasks.

It is a bit like saying "why would you hire someone with a doctorate when you could get unlimited high school grads". How appealing that sounds depends on your needs.

The ability levels of the cheap models are encroaching on the abilities of the frontier models faster than frontier models are expanding their abilities. If we haven't already, we will very soon reach a "good enough" state where having the "best" model matters less and less and less.

By analogy, if you buy a new computer, do you get the absolute fastest CPU available? Maybe, depending on your workload. But if you're 90% of the population, you get the cheapest one that has enough power to meet your expected workload, which is mid-range, not top of the line.

Many of the responses seem to say "but there is a market for cheap models".

There is a world where we have cheap models and we don't have ASICs powering them. Things like TPUs and NPUs, which are programmable, are likely to fill that role. They are optimized for inference while also allowing different (and updated) models to run on them.

Given two companies competing on the cheap end. First company goes TPU, second goes ASIC: who wins? My bet is on TPU since they can update their model, even if their hardware is slightly more expensive and slightly slower, since the optionality of new models beats the performance gap. That may not hold forever but given the pace and volatility of the current LLM market, I believe it will hold for some time.

Models like Kimi 3, GLM 5.2, or even Fable 5 for that matter are reasonable to burn to.ASIC because they are over the threshold of "good enough to be generally useful", something that will continue to be true in the future.

Most people do not need the latest model, they need a sufficient model. If I had Fable 5 on an ASIC, I imagine I would use that and ignore paying API rates for Fable 6.

Which is why my original post mentioned the volatility in models. We aren't just doing research on frontier, there is a huge amount of research on quantization, distillation, etc. that is changing the landscape at the low-end almost as much as it is changing on the frontier.

And it is also why I mention revealed preference. What feels sufficient / "good enough" today is a moving target. This isn't just a question of what you want, it is a question of what is economically viable for the entity that will be designing, manufacturing and marketing this ASIC of which you speak.

This is me personally. The calculus is different for other people. But I suspect Kimi 3 is pretty darn close to that tipping point for an awful lot of people.

The managers of the firm will.

They dont care that its faster unless it translates into the financials. They want lower costs, higher revenues - explain how it fits brudda.

There’s absolutely a place for a lifestyle subscription to a sonnet model that you could just use everywhere all the time.

  A Fable 5 model running at 9,000 tokens/s on an ASIC rather than 150 tokens/s on electricity chugging Nvidia GPUs, or even giant SRAM Cerebras or Groq chips could be good enough to meet the majority of demand.

pretty stupid statement lmao

The incremental unlock of capability by ever increasing frontier model sizes will eventually reach diminishing returns.

I would argue tnference speed increases would actually unlock a different kind of more meaningful value for a wider audience.

640k was enough ... in 1981 ... almost fifty years later is 50,000 lower than a standard off the shelf PC now

I get this reference!

Yes. The latest OpenAI and Anthropic models are terrible at planning roadtrips.

This is something I try to use them for frequently. They constantly get things completely wrong.

I’d say that about half of the stops they suggest fail to follow whatever filters I’ve asked for.

And this is one of the big things that seems to be missed in these discussions: There is no longer a universal linear trend of LLMs being 'better' each iteration. They are becoming more specialized, and ones that approach problems from a different angle (like Fable/Mythos) can appear breakthrough when first released, but we don't appear to be on a path that actually leads to general purpose hyper intelligence.

It was nonsense to draw in investment and justify an inflated valuation.

lmao

This is starting to look at a lot like Intel vs Arm from the last era.

The Fable & Mythos are starting to look like a giant Xeon, while the smaller lighter models are starting to look like a lot of tiny ARM chips which sip on power instead.

The risk is the same as what Intel had. There is a group who are pushing them to go bigger and with a resource no limit approach, who have a lot of dollars to push you that way.

Follow them and they lead you to a pile of money, but then you risk something like Apple Silicon happening.

Something which got better because of efficiency & continuous improvement, not neutered due to it.

The problem was that Intel stopped making things faster and better, and shifted to wearables and mobile chips instead of investing in their hard-tech strategy that had worked for decades.

Any value will come from the largest models, and those largest models are unlikely to ever run on consumer hardware within their window of relevancy.

Small models are good enough depending on your task. That's the point. A model you can run on your phone or laptop is an incredibly useful tool for a lot of problems even though it isn't the "best" theoretically possible model.

You can run a Moroccan chicken fragment on the cheap in homage to what you cannot run

There is already custom hardware see cerebras.

GPUs have a lot of slack there is at least one lab that had a (small 8b) model generate almost 3000 tokens per second on a MI300X for a talk, instead of the typical software stack that did maybe 100ish tokens per second.

High bandwidth flash storage is in the works, i.e hard drives with TBs of storage and over 1 TB per second of read speeds. Meaning that in a couple of years you may be able to buy a card with 40-90GBs of HBM and 4TB of HBF and run a 3T model locally at a reasonable speed for 10-20k as opposed to a cool mil.

It'll obviously be China, and they won't need the bleeding edge of lithography tech to make it happen. Every Chinese smartphone will have something like Sonnet 5, along your car (well, not those of us in the US, but we'll look longingly at pictures of them while we drive whatever the government decides we're allowed to drive in Fortress America).

Give it ten years and your smart litterbox from Temu will be running its own local model.

The comparison I've seen elsewhere is the old console systems with separate cartridges for games... I wouldn't want to be regularly swapping them, but if they came out in a form factor that didn't require me to shell out multiple-4 digit figures in upgrades just to use the next model, it'd easily be worth it for me.

I've already got a home lab, and it's specced to last, minus the GPU. I picked up a separate system for local llm experimentation, but I'm not likely to be upgrading it again. The value add is incredibly small compared to the cost. The real benefits are data privacy and never worrying about rate limits, and there's a price point beyond which an incremental improvement to the model doesn't justify upgrading the system.

As of today, that appears to be Google!

https://news.ycombinator.com/item?id=48986351

https://news.ycombinator.com/item?id=48464958

Ultimately however, that just means that models will become dirt-cheap. The money will be made with applications built on top of the models.

    guess you are arguing that the models _now_ will 
    be durably useful enough to commit the time to 
    creating the ASIC?

Subjectively, it's hard to say if people will pay for "a model from 18 months ago, but REALLY FAST AND CHEAP"

The speed difference suggests some use cases that might narrow the performance gap. With a > 50x performance delta you have some headroom to play with.

You can do many many fast iterations of ye olde "Ralph loops." You can also jack up the reasoning/effort level. And you could probably do some combination of both, while still running really fast ie 10x the number of iterations at 2x reasoning/effort.

So I think a hypothetical "50x faster Opus 4.8, but burned into ASICs" could be pretty competitive against the frontier models from 2027, 2028, 2029, and maybe beyond?

What's fascinating is that we are pushing the state of the art of hardware at this point.

I don't understand how this works when the models are evolving so fast that your burned ASIC is outdated (or at least not top of the line) in a few weeks.

To me, it's like imagining if Sonnet 3 was burned into an ASIC 8 years ago and then never changed. It would still be revolutionary, and today we would have an entire ecosystem of tools and services built around it, likely surpassing some of our current workflows.

The frontier is a different beast, but it would likely mean competing on price.

I think that's the problem at the moment. A much better LLM that doesn't use my battery is 20ms and <10mb of data away.

Sure, an ASIC model will always be behind (12 months? 18 months?), and a hosted flagship will be significantly better. But as time marches on, would I use a flagship model from last year if it came on a PCI card and cost $1000? Without a shadow of a doubt.

Cerebras tries to get around this by keeping everything on cache SRAM as much as possible, which it burns directly to the chip wafer itself and physically places that SRAM directly next to the tiny compute unit that does the actual math.

An ideal setup (not sure how easy this is to achieve in practice), is the burn the weights of the model directly to the chip as a sort of ROM, the actual math operations as actual digital circuits, and have SRAM, or even something akin to naked registers to directly compute off inference batch data. Cuts out 2-3 layers of abstraction and indirection.

You still need some memory for the context, in flight answers, ... but not for the model weights and for the output of the intermediate layers. I found taalas demo here: https://chatjimmy.ai/

Nobody needs "web services run from China" to use Chinese open weight models.

Scaling this up to 2.8 Trillion (350X increase), will certainly be challenging.

If I was younger and had the right background, I'd love to dive into attempting somethign like this

Last I saw they posted Deepseek R1 numbers in Feb of this year.

The challenge is rolling out a new one every 7-8 weeks as the weights change & cheap enough for a hyper scaler to afford to buy one and save enough on power over the next 8 weeks as a payoff.

Likewise, for compute, is the ASIC somehow going to beat a systolic array? You can't have one circuit per weight: the die area and electrical fan-out would be insane. I'm not seeing how an ASIC specialized for a specific model would actually help much.

I mean, sure, we can build more specific accelerators, e.g. for softmax, but these work fine in the context of a programmable pipeline.

Yes, there are more exotic things out there, like optical matrix multiplication systems. Those are different. But above, aren't you talking about just doing conventional digital linear algebra, but with a model-specific set of circuits?

I can accept the idea of specializing a circuit for a specific model shape, but I'm not seeing a need to specialize a circuit for the weights inside the shape.

----

The Board Departure: Mike Krieger, Anthropic’s CPO and a co-founder of Instagram, sat on Figma’s board of directors. He resigned on April 14, just days before news of Claude Design broke. This sparked speculation over conflict of interest and the use of proprietary product strategy information.

Betrayal of Partnership: The launch aggravated the tech industry because Figma relied on Anthropic's models to power its own AI features, and even announced a joint "Code to Canvas" integration. Reports indicate Figma was blindsided by the depth and scope of Claude Design.

Market Reaction: The "SaaSpocalypse" thesis—fears that major AI foundation models will rapidly build application layers and cannibalize their own SaaS partners—was realized when the news broke. Figma’s stock saw an immediate 7% drop upon the announcement.

----

I would suggest to people using LLMs: you should be cautious about giving these companies data or relying on them. If you're building an AI startup, there's a very good chance they could decide to directly compete with you if your idea has traction. You're also at their mercy for API pricing etc.

These companies haven't got the attention span to work on one thing for that long.

The driving constraint if you want to take a spot in B2C is the speed at which consumers replace tech, not the speed at which tech can be developed.

Wrong question. The real question is "Who and what will they buy with the gold?"

The golden rule is just the beginning, and no, there's nothing positive for the rest of us.

But the big money has been skipping Leg Day for ages, and that lack of a foundation will bite us all. Which sucks.

Then, by the time GPT 5.5 came out you could already generate a flawless PDF and Word doc resume with the same formatting as your base resume. It really is amazing.

Trying to compete with these labs is a bad idea, you might work hard on your thin wrapper and then they come out and have their chatbot or harness do it way better than you.

But I also think this is really cool. It's a new era in software. Build something worthwhile and you won't need to worry about a lab doing the same thing with their chatbot.

They think whatever you are doing is cute and would very much like to ensure their models can do it even better in the future, but competing? Not even worth the time to think about

LLM generated code is not copyrightable, so even if they do "steal" it - I don't think there's legal grounds to do anything about it.

It can't be stolen. You don't really own it.

You can try to lock it in a safe and hope no one ever gets a hold of it. You can lie and say you didn't use an LLM, but Anthropic and OpenAI et al probably have logs to disprove that.

But, if push comes to shove, you don't own it...

Someone tried, as a bit of a stunt, to register a work for copyright with generative AI as the sole creator/author. That registration was rejected. This is quite different from a person using generative AI as a tool to create a work.

People have copyright in photos and videos they create, even if they used a camera. Same with images and code, even if they used an LLM.

I'm not your lawyer, but to the extent you have copyright in works you create, the fact that you used a tool doesn't diminish the copyright.

The clearest part from the page:

> Before the proliferation of generative AI, U.S. courts did not extend copyright protection to various nonhuman authors, holding that a monkey who took photos of himself lacked standing to sue under the Copyright Act; that human authorship was required to copyright a book purportedly inspired by celestial beings; and that a living garden could not be copyrighted. The U.S. Copyright Office has also long maintained that copyrighted works must be "created by a human being" and therefore refused to register works that are "produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author."

I think the long story short is if you're incorporting LLM generated content into your own work, then your own work is copyrighted but if it's purely AI created and you never put any of your own work into it, it's not (so: think the slop cannons people are pointing at social media for example -- probably not copyrighted (although I don't know why you'd want to put your name on it anyway)). Granted, I don't know how you'd ever prove that something was purely LLM generated so it's kind of a technicality for coding, but for image and music I think it applies pretty clearly.

As an aside, I think socially maintaining that copyrights are for humans only is extremely important. It might be the most important legal matter around this. I see nothing good of allowing AI labs that have strip-mined the internet to claim a copyright on the things they regurgitate, and I don't think spammers deserve copyright protection either. The human-only requirement is a very good one.

Also, a lot of people using LLMs are using it on code bases that were written before LLMs were common for coding. So in that context, you really are allowing them to train on your code for free.

> It can't be stolen. You don't really own it.

At least in the UK computer generated code has been copyright-able for decades [1]

> (3)In the case of a literary, dramatic, musical or artistic work which is computer-generated, the author shall be taken to be the person by whom the arrangements necessary for the creation of the work are undertaken.

And the ToS of the major/minor AI providers as far as i can tell have no opinion on what you do with the code in terms of ownership.

[1] https://www.legislation.gov.uk/ukpga/1988/48/section/9

[citation needed]

I've seen cases where it was determined that the LLM itself doesn't hold copyright. But where are you getting the idea that nobody holds copyright?

For one, on the margin people are willing to pay a lot for slightly better models. I know personally the value the LLM adds to my workflow is considerably more than the $200/m I pay the frontier labs. I have no interest in optimizing that to get it slightly lower. There are a very vocal minority that optimizes this or companies whose LLM expense is marginal, but I think that's the minority (correct me if I'm wrong, curious what their customer base looks like)

Also the actual LLM is a tiny portion of the value added. Anyone that tried to build agentic solutions from LLM apis quickly realizes that a huge value is the Claude Code / Codex harness. There are open source implementations like OpenCode but they're not nearly as good.

Think about it another way. Consider how much money Microsoft spends on maintaining Excel. There are open source alternatives that have >90% of the functionality, they'll even work w/ Excel files and generate them. Google sheets is probably 99% and available to everyone and better in a lot of regards. But the immense value spreadsheet software produces workers above the $100 or whatever a year makes it so that there is a real moat and no one bothers exploring alternatives.

Enterprise is how Claude/OAI make their money. Enterprise companies absolutely optimize around costs for infra, it's like an entire job category.

> Also the actual LLM is a tiny portion of the value added. Anyone that tried to build agentic solutions from LLM apis quickly realizes that a huge value is the Claude Code / Codex harness. There are open source implementations like OpenCode but they're not nearly as good.

Claude and OAI are not valued at $1T because of their harnesses

> Think about it another way. Consider how much money Microsoft spends on maintaining Excel. There are open source alternatives that have >90% of the functionality, they'll even work w/ Excel files and generate them. Google sheets is probably 99% and available to everyone and better in a lot of regards. But the immense value spreadsheet software produces workers above the $100 or whatever a year makes it so that there is a real moat and no one bothers exploring alternatives.

Enterprise excel is like $50 a month, the closed source labs charge orders of magnitude more than that per user per month for enterprise, and want to charge more if they can to maintain their valuation. I spent $20 in an afternoon yesterday using my work plan for one workflow for reference, I probably spend at least $50 a day and I don't even do that much coding anymore.

I'm genuinely not sure where the balancing point even is.

And cost of acquiring learning via users feedback versus additional learning data value for a given company will dictate whether such company will give up or will continue subsidized usage, despite being unprofitable on the paper, but probably really valuable as a long term strategy for GTM and product development and increased learning rates.

They're valued at that because they add a lot of value and people pay for the product. The product is more than the LLM. If you want argue the value of the harness vs LLM but flippant remark adds nothing.

> Enterprise excel is like $50 a month, the closed source labs charge orders of magnitude more than that per user per month for enterprise, and want to charge more if they can to maintain their valuation. I spent $20 in an afternoon yesterday using my work plan for one workflow for reference, I probably spend at least $50 a day and I don't even do that much coding anymore.

I want to know what percent of their revenue is this API spend compared to just a Claude/Codex max plan that's $200/m and very generous.

I can't imagine why anyone would need almost 7x that.

But $200/day is very common in the business. The monthly plans are just a trial version of what's going to come for all of us. That's why I am very carefully looking into the open models today.

This is a very rich / developed country privilege perspective.

Where I live, it's not unusual for a monthly wage to be around $200. Of course developer wages are much higher, maybe as much as $1000 a month, but $200 is still a huge chunk of that so it doesn't really matter how much "value" you get out of it if you're no longer able to pay rent or buy decent food.

Even in developed countries, $200 a month is out of reach for all kinds of people who would benefit from it (students without rich families, entrepreneurs, etc.).

Like Codex https://github.com/openai/codex

That margin is getting smaller and smaller. I would have been with you a week ago; paying for Fable was worth it compared to all other models. But with K3, the difference has shrunk to the point where, for me, it's not worth the cost anymore.

In other words, it may be worth paying five times as much to get 10% better real-world outcomes for a lot of people, but a lot fewer people will pay five times as much to get 2% better outcomes.

> a huge value is the Claude Code / Codex harness

For me, it's the opposite. Having to use Claude Code instead of the harness I prefer is a point against Anthropic, not for it.

On our side we use Claude/GPT/Kimi (it replaced Antigravity) for development. But we build our systems around a cheaper denominator (Deepseek previously, recently we added GPT 5.6 which have good prices as well). We offer BYOK for Claude but its def not an option to build something on top of it (for us).

Excel has network effects. If N people are using excel, the incremental N+1'th user is forced to use it or risk not being able to open documents from the N users. Excel is also sticky, in the sense that a user used to Excel UI is hesitant to switch to a new UI. Tokens dont have network effects. If N people use fable, the N+1'th user could use any other model, and have no impact.

But once you have an AI-powered application in production, why wouldn't you go with far cheaper and capable enough models? That's just optimising a business process like any other. You'd use cheaper providers whether Chinese, or other on-prem models.

> Think about it another way. Consider how much money Microsoft spends on maintaining Excel. There are open source alternatives that have >90% of the functionality, they'll even work w/ Excel files and generate them.

Sure but were these open source alternative as pervasive and widespread and talked about when Microsoft was bundling Excel with the OS? Or were there millions of dollars, if not billions of dollars being spent by these competitors? The answer is No. It is not the same scenario.

While I understand the logic of harnesses - those are not full proof. It is trivial to setup K3 or Qwen to work with Claude Code/Codex by intercepting requests and routing to K3/Qwen. You have tools like CCSwitch which can do that for you.

The only thing remains is - people willing to pay a lot for slightly better model. That is true but price sensitivity is also a thing. The mania case for Anthropic/OpenAI/SpaceX/Google is that they will capture large part of the enterprise market. But in many cases outside coding AI capabilities will be resold.

That is instead of buying AI for lets say HR functions you pay for embedded AI in your HR software. And the HR software company will have incentives to raise their own margins. If their provide similar looking experience using Qwen/K3 why will they buy Anthropic/OpenAI. That is the real risk here. While as an individual you can continue to pay top dollar.

If you're building an optimizing compiler, a CUDA kernel, a database, a high performance concurrent data structure with tricky locking, etc. etc. it's still not really close. Sol 5.6 on high just slays e.g. GLM 5.2 for this kind of work for me.

I'm sure K3 is fine for these things too, but I can't afford it at its API rates compared to a Codex coding plan.

For now.

https://github.com/rdaum/mica/tree/main/crates/relation-kern...

https://github.com/rdaum/mica/tree/main/crates/vm

https://github.com/rdaum/pagebox/blob/main/crates/wal/src/wa...

I have CUDA work here somewhere too but I have the repository private right now

The hype cycles are shortening, perhaps we really are reaching some kind of plateau this time (famous last words)

People use these models for diff things.

Its quite possible for the things they are used for, people do not see much of a difference.

Do you hold stock in Anthropic?

> Unnecessarily aggressive

> First ever comment said "Further releases of Chinese models that demonstrate the gap is not growing substantially is a huge problem. The spending will be called into question."

Yeah I think you have an agenda

I wish.

Chinese labs have come up with a bunch of genuine innovations: GRPO, auxiliary loss free MoE load balancing, MLA, muon optimizer, and a bunch of other ones. The Deepseek papers are really well written, this isn’t just sneaking a peek at a peer.

The problems are inherently harder now too, partially because they take longer, so your training pipeline is waiting for long completions.

Also there probably is some “distillation” (technically pseudo-labeling, which is common in ML). But I wouldn’t put too much weight on it because that was true 18 months ago as well.

Not to mention that US companies models constantly distill each other as Musk was forced to admit under oath. This whole narrative has just been a massive cope.

> This whole narrative has just been a massive cope.

So wait, US AI companies all use distillation because... it's not effective and it's all just cope? Or is distillation really powerful and they all do it, which Musk was forced to admit under oath? But when China does distillation it isn't powerful and they don't need to do it, but they do it anyway because it's fun?

Either it's powerful and everyone, including the Chinese labs, use it as a way to rapidly catch-up against the SOTA models, or it's a red herring and the huge amounts of energy spent to protect and enable distillation is all just wasted money. Which is it?

Both sides have extremely smart people. One side has more $$$ and exclusive access to the best chips. For progress to converge without a corresponding breakthrough suggests there's something else at work.

The US enjoyed an early advantage due to excessive money being poured into AI which led to the current bubble, and access to the hardware that was needed to train these models initially.

At this point, neither of these factors actually matter that much. The naive approach of simply making models bigger has hit a wall, and now you need ingenuity in figuring out better architecture for them. Precisely because Chinese companies have had to deal with more limited resources, they put a lot more effort into researching different kinds of optimizing techniques. And of course, China is also catching up in chip making, and Huawei clusters are already competitive with Nvidia for training. So, that gap is closing as well.

The big difference is that an absolutely insane amount of money has been spent in the US, while China managed to do this on a fraction of the budget. The AI Investment Surge graph here puts things in perspective. https://hai.stanford.edu/news/inside-the-ai-index-12-takeawa...

But OAI and Anthropic are trying to cash in ahead of their IPO window. I think that window is pretty much gone now.

I don’t think one should pay much attention to them.

My understanding is that the labs ran out of freely available data to train on a while ago, and now primarily rely on human data vendors such as Surge and Mercor to source their data.

Hm. How is OpenAI not a “model-only” provider just like Anthropic? Seems like they are vulnerable in the same way.

I could easily see OpenAI become irrelevant in 2 years if they stumble at all and don’t keep up with the other frontier models.

Google's quality has gone to shit, but internet searching is still "googling", both in verbiage and in the actual service people use. It's not an impenetrable moat, but OpenAI would have to stumble pretty hard to lose all of that edge.

That said I think it is pretty easy to make a case that these would-be differentiators are either currently underwhelming or completely unproven (as in the case of hardware).

Having a good model is one thing, being able to serve that model at good speeds and match demand is another. See Anthropic ~6months ago. Or Moonshot, they've already suspended subscriptions to their coding plans, because they can't meet demand.

Lower bound of AI progress - plateau. Progess is slowing, focus is on serving a meaningful peak capability at the lowest possible price. There's been news today that Google is building a Gemini chip with weights baked into silicon. Considering a chip's lifetime of 2-3 years at minimum, and that a 2-3 year model today would be useless today, they're expecting they wont make a similar amount of progress in the next 3. Game is about selling at the lowest margin. Anthropic/OAI is cooked.

So their survival rests on the presumption that AI progress will fall between these two extremes.

That feels like a very generous framing. There's very little opportunity between the extremes that would paint a convincing outlook of survival for either company.

Perhaps if they were able to scale down their spending drastically they could survive, but that requires acknowledging their current valuations are BS. Doing so is a major risk, that will piss off all share holders. There's also the employees they would need to fire or reduce salary. The shift of focus internally to sustainability would be a major challenge.

We need SETI@home for Open Weight models yesterday...

I would imagine it would look something like this:

Ground breaking/novel research -> SWE -> day to day assistant conversations -> chat support bot...

A related question is how much they're dependent on the APIs of Anthropic and OpenAI to achieve their results - whether through distillation or other uses.

If these models are derivative of Anthropic/OpenAI I would expect performance to be more narrow and progress to be limited.

Its value per unit of currency spent.

Financials will ultimately drive decision making.

We are already seeing that more intelligence does not correlate with more revenue, for the firm purchasing tokens.

If I was OAI/Anthropic I'd be brown and yellow in the boxers.

One of them heading to IPO and the other opting to not show their books should tell you everything

Ramp the number up to 85% if that doesn’t work

If it still doesn’t work, go nuclear and target 100% job losses language

Something i cant find any reliable data for, but would help for a sense of scale: How much use before its equal to training?

I.e. assuming you have the training data and setup, and we only care for compute - How hours of using eg Kimi K3 / Fable, for it to equal the compute required to train it?

they used ~14.8 trillion tokens with about 2.66 million GPU hours. 14.8 * 3 = 44.4 t inference tokens.

obviously, this is back of the envelope math, but at 100t/s you would need like ~14k years. scale this to >100k GPUs and your in the hours to a couple days range.

I was with you up until this point. I don’t think OpenAI has any more of a substantial product moat than Anthropic; if anything, the Claude / mythos etc brand is a valuable asset that OpenAI lacks.

Yes, many of the elite HN engineer always online types have come to prefer Codex, and but if you actually talk to regular engineers in industry, agentic coding is simply still synonymous with Claude Code.

And for the non-engineering uses, Claude is so much more pleasant of a conversational companion than any of the GPT line, and I suspect is this baked deeply into the model, otherwise OpenAI would have closed this gap by now.

I'm happy if Anthropic's business remains viable as one of several strong competitors. The company's safety-first ethos is driving them to increase refusals and deliberately-built-in ignorance with their newer models. In the long run, Anthropic may be best remembered for accelerating the development of software in general so that other people could build less timid tools.

Making the latter happen involves either reformulating work in ways less intelligent LLMs can work better with. Or condensing intelligence into smaller models.

Also, “smaller” can mean many different things. The cost is not in storing the weights on disk.

It is in the power required to do the inference with the “active parameters”.

There is increasingly more evidence that those two can be decoupled and more power to those who are pushing on that lever!

For example - What percentage of workflows require this new highly capable model? How much of it can be replaced with the software tooling around it? What I mean is if the software tooling can optimize the query over a few iterations does it get the same output as from a single shot high capability model query?

I'm not saying these other models are trash, just that I'm not quite ready to put them on equal footing to Anthropic or OpenAI models.

I think the future of LLM coding (and more) is probably routers that decide on a per-task basis which model to route to. Know when to use Fable/Opus and when to fall back to DeepSeek/Kimi/Qwen, even all the way to local models. Of course that's not in the frontier lab's interest but it feels like there is a lot of low-hanging fruit there. I hate that right now it's pretty much "just use the same model for planning/execution/etc" (without standing on your head).

Even their hail mary of being first to "AGI" will never happen because all it takes is China blockading Taiwan or Nvidia cutting them off to stop them from eating up a large chunk of the economy.

There is no scenario where the rest of the world will sit on their toes and let OpenAI or Anthropic monopolize "AI". Too many countries, large well capitalized players and partners / suppliers who could never let that happen.

Kimi K3 allows all existing players to restart at the frontier and keep competing with OpenAI/Ant. It also gives employees at these labs a better more lucrative path of starting new labs with fresh books and clean cap tables, building on top of K3 without needing to spend all the capex on pretraining their own models. Plenty of them already vested their stock and would have 0 problems raising 100s of millions of dollars for new labs, making them paper billionaires over night.

Even now on the $200 plan I use up my Fable credits in a single day and had to start using codex and openrouter for more usage because Fable burns $100s an hour when billed on usage.

It became an easy decision, even the $200/month by Anthropic sounds like a bad deal.

It's also already the case that in larger companies people do not have access to these plans as employees and must use API rates.

There's also the political angle. When Anthropic and OpenAI held back their top of the line models because of Bessent & Trump's bullshit, and threatened to deny unwashed foreigners like me access... I dropped my Codex plan and made do purely with GLM 5.2 for three weeks before OpenAI finally released 5.6 Sol. Feels inevitable that this will happen again.

Or, somebody will come up with a way to serve e.g. Kimi K3 or the new Qwen model in an extremely cheap way. Or DeepSeek releases a competitive model at their cut-throat rates. And then the cost argument just wins.

we'll get new quants, dspark speculators, distills and optimized kernels

as long as there are near frontier models available there will be inference providers selling them at or below cost of inference in attempt to get market share.

This is a very large model. Much larger (3x) than GLM. The resources to run it are very expensive.

If Anthropic doesn't make money because of distillation attacks, how would they convince investors to invest in them, such that it makes financial sense for Anthropic to train even bigger models?

Assuming it is preferable for everyone that we get better models in the future. Distillation attacks remove the financial incentive.

And if they are, the point remains that Anthropic has a brittle product advantage that users and investors should be cautious about.

If you put any faith in claims made by OpenAI's "Head of Strategic Futures," then it's hard to believe these models are just the result of distillation.

if they are cautious, what would make them invest in newer bigger models without the expected return? generosity?

So much money has been invested in Anthropic and OpenAI at this point that to declare it a loss and walk away could potentially destroy a lot of VC firms, and a non-trivial chunk of the US Economy.

do you know how ROIC is calculated?

Good luck hiding your huge sunk cost of bilions and billions in there bro. I wish people who had zero understanding of actual finance would never comment about it.

Moreover if they declare their existing assets are bunk, the valuation is marked down, especially after now pricing in failure risk - this is catastrophic for VC's.

Again, bro, just be quiet.

I think at least deepseek can easily take Anthropics to court and win a defamation case. The number of requests allegedly done from deepseek IP range is so small it is barely enough to run a few benchmarks.

I dont care if that company dies. Or if OpenAI dies. I dont mind investors investing into other things either.

To everyone praising free and open source software, if software companies don't make money, how are they and YOU going to get paid, PERIOD?

People keep pretending that Chinese companies only make second-class copies of American products until it's too late.

the vast majority of people do not care about like-for-like phones based purely on features.

Almost the same thing for twice the price, just for the pleasure of saying that you believe Apple was first?