I really want capable models local but it’s impossible for me to afford the hardware. If there is a magical way to run glm 5.3 without quantization at 80-100tps with a total spend of less than 10K I’d be happy but that’s impossible now.

Apple, to me, is the winner in the end of this AI chapter.

Distillation works: we can get models leaner, have better performance with lesser and lesser parameters, more efficient, running locally. Apple gets this and we'll see the result with M7 Max/ultra chips and I reckon this will be one of the titular apps when M7 line of chips are introduced.

I guess it explains the haste Anthropic and OpenAI are in to IPO, cash out and leave the the retail investor holding the bag.

I think the question to me is whether Apple can ship affordable consumer devices with enough memory to run sufficiently useful models sufficiently fast.

Right now 128GB MacBook is the top of the line and you could do Qwen 3.8 Flash or similar.

There is a rumor that the M7 chip can use 1.5TB of memory which would imply Apple is aiming for more memory on devices.[1]

Would that imply the entry-level MacBook will have 128GB and top of the line models could run larger reasoning models? Then the question is price and if any of this will translate to iPhone for on-phone models.

With how expensive memory is, IDK if they can pull this off but I hope they do

[1] https://www.tomshardware.com/tech-industry/semiconductors/ap...

Even if it's 2030 - that's a mere 3 years from now - it will take at least that long for the greater economy to figure out how to use LLMs. And probably a memory boom/bust cycle in that time too.

If Apple, in 2030 has a compute product that a worker can BYO(X) to work that has 2TB RAM, with an encrypted personalized context that can float between agents, then they will be in a very interesting place.

> 2030 - that's a mere 3 years from now

holy shit

Most likely Apple has been spending whatever they can to get in front of the memory market.

They definitely are, judging by their price increases.

Even so, DDR5 is so high-demand that it's likely that Apple can't outbid customers like Nvidia or OpenAI when they drive the margins they do.

No but they can pay for new manufacturing to be built. 2-3 lead time should be enough to bankroll the effort. Apple has to much money to sit around hoping that memory situation resolves itself. They were caught flat footed by the RAM market.

Cloud compute has been completely commoditized. Yes, local models will get better and better, but I think for the vast majority it will continue to be both cheaper and more productive to use cloud based models, far into the future, where bulk purchasing, oversubscription, and discounted industrial electric rates can be taken advantage of.

Where I am, it's cheaper to purchase multiple claude subscriptions than it is for the power bill from running a local model. Factor in the hardware costs, and you're looking at a break over a decade in the future, for comparable cloud models. Yes, there's privacy, etc, but now you're down into fractions of a percent of users who care about the government reading the memory of their computer.

And if you're on a mobile device, any halfway powerful local model is going to cut your battery life way down.

Related, I tried Juggernaut XL (image model) on my iPhone, and ate 2% battery per image!

Take a look at how qwen3.8 27B compares on benchmarks to much, much larger models. A while back I called it the crow of AI models: paradoxically strong given its tiny size, like the brains of crows and other intelligent birds.

It's not comparable to huge models of course, but given the power/size ratio here it shows that serious gains are on the table for models that can run fast on a high-end laptop.

Man, if all that makes Apple a "winner" then I'm dying to hear your assessment of Nvidia.

Factoring in the increased cost of memory, Apple's threatened service revenue and the stagnation of ecosystem products like the Mac and iPad, I think they have a really challenging future to navigate. The whole "sell people a B2C ecosystem" business model is more threatened than ever.

The contrast I'm drawing is running these models locally vs on the cloud and how that would be a selling point well received.

I don't think that's materially different from what you can do with Nvidia and AMD GPUs, besides the fact that you're limited to Thunderbolt interconnect.

Are you an apple user or investor.

If you are an investor, you should remember they are a fashion product more than a tech company.

If you are a user, you should remember they are a fashion company more than a tech company.

> Cluster for scale: “Connect two or more Thunderbolt 5-capable Mac computers and Harbor pools their unified memory into a single inference resource. Run models that exceed any single computer’s capacity, or serve more concurrent requests across your cluster.”

This seems like the main way it would differentiate itself over existing solutions like LMStudio etc.

"Harbor is currently in a private preview under Apple’s Claris brand"

There's a name I haven't heard in a long time! Claris (always an Apple subsidiary, since being founded in 1987) renamed themselves to FileMaker in 1998, but apparently they rebranded back to Claris in 2019.

Looks like they've mostly just released annual upgrades to FileMaker for the past decade, but they did add MCP to it last year: https://www.claris.com/blog/2025/why-mcp-matters-for-claris-...

If they keep Harbor under the Claris umbrella, that is a nudge that they are doing it for SMB market instead of the consumer market and it will likely be a onerous subscription to use it.

https://getlocalcode.app/ runs local AI models (larger ones of course, DeepSeek 4.1 Flash 1017) easy. You can download and try it now. The problem is elsewhere: you cannot find M5 Ultra that you can purchase and set it up today.

Deepseek has found safe harbor!

You can chat with the local model that ships with macos on your M1+ mac right now. There's a few servers on github that make it easy.

You can also "fm chat" in terminal

OMG after 25 years, Claris gets something beyond FileMaker (among the most legacy software still alive)? Claris would not have the engineering talent, so why would Apple use them as a vector?

Currently FileMaker is a really expensive but sophisticated developer deployment environment. They stayed pretty true to the original vision in the mid 80’s when they merged their DOS database and added a gui layer for the Macintosh. I started on version 2 and it was at its best at when it went relational in version 3. Since then they have been focusing on deployment of graphical data applications and being a better database.

https://en.wikipedia.org/wiki/FileMaker

I love that they are bringing Claris back for this. Moof!

I'm just learning that the Claris brand still exists.

But you’re still in for a ton of pain if you want to run them headless right?

They desperately need to fix that.

It was pretty fast to get it set up. It could be better for sure, as it's not a first class concept the same way that it is in linux, but an hour of google search ai mode will get you where you need to go.

How so?

Only took them 3 years for something that's inevitably worse than lm studio.

Users will praise it still. What's the phrase, Apple distortion field?

Hopefully it won't be another preinstalled bloatware app like Image Playground. macOS enshittification has been greatly accelerating, even giving Win 11 Home a run for its money.