HN.zip

Shapelearn Qwen 3.8 27B (13.1 GB VRAM)

46 points by syntaxing - 4 comments
_ache_ [3 hidden]5 mins ago
From my own test. It's not faster than the unsloth model.

Disclarer: I'm unsing Vulkan on an AMD GC.

sheo [3 hidden]5 mins ago
Mashimo [3 hidden]5 mins ago
In the comments it reads like bonsei falls apart on longer running tasks.
txrx0000 [3 hidden]5 mins ago
Not really. The largest IQ4_XS quant here is still worth it because Bonsai doesn't offer larger quants. They could beat it if they made a quaternary variant though, I don't know why they're stopping at ternary.