HN.zip

Huawei releases MACaQ, its internal quantizer for local inference

119 points by - 26 comments
[3 hidden]5 mins ago
People are reproducing it and creating GGUFs https://www.reddit.com/r/LocalLLaMA/s/36atJ2nNjh
[3 hidden]5 mins ago
The top 3 repos sorted by stars on Huawei's github page are all for quantization. I am not an LLM scientist so I can't tell what the difference is between them, or why they need three different pipelines for model quantization.
[3 hidden]5 mins ago
https://huawei-csl.com/blog/macaq/

Why does some HN users seem to prefer GitHub page whenever available at expense of official site? I mean, Huawei is still sanctioned, anything they host third party outside China can disappear at any hour.

[3 hidden]5 mins ago
I prefer GitHub, because I can see the code and immediately know where to click. Product pages are always full of various marketing shots, videos, etc that just takes more time to parse through.
[3 hidden]5 mins ago
This is pretty much the reason that if I read about a new product, I generally just go to the Wikipedia page. I just want the no-bullshit version, no "Say hello to blah" video or marketing word salad.
[3 hidden]5 mins ago
In this particular case I think you'd be pleased with it https://huawei-csl.com/blog/macaq/

It is from their lab in Switzerland, after all.

[3 hidden]5 mins ago
Looks like many of their repos there have been around for a while. Also not sure why we would sanction free stuff from them
[3 hidden]5 mins ago
Well I'm not a lawyer but are you allowed to provide material support to sanctioned entities?
[3 hidden]5 mins ago
It’s a commerce department sanction not a treasury department sanction (which would be far more serious).
[3 hidden]5 mins ago
Did they release any model? Seems good for Strata
[3 hidden]5 mins ago
Qwen 3.8 27B has a macaq quant available
[3 hidden]5 mins ago
What a project name.
[3 hidden]5 mins ago
Probably intended to reference macaque monkeys.
[3 hidden]5 mins ago
I forgot my French, but isn't it incredibly rude?
[3 hidden]5 mins ago
Why?
[3 hidden]5 mins ago
I think the immature commenters are referring to the pronunciation being similar to "my cock"
[3 hidden]5 mins ago
Aaa, thanks I didn't think of that.
[3 hidden]5 mins ago
Congratulations on not being a total degenerate.
[3 hidden]5 mins ago
It's also a racial slur in certain languages.
[3 hidden]5 mins ago
It's also "Roach" in Hebrew (מקק)
[3 hidden]5 mins ago
What are those?
[3 hidden]5 mins ago
No, no, not that.
[3 hidden]5 mins ago
I'd assume the French know something about naming projects in the vein of Coq.
[3 hidden]5 mins ago
xD
[3 hidden]5 mins ago
I do not think that macaques would mind.
[3 hidden]5 mins ago
Just enough plausible deniability :P