unsloth / Qwen3.8-27B-GGUF like 1.98k Follow Unsloth AI 31.8k
Introducing Unsloth Dynamic v3 Qwen3.8
#74
pinned
by danielhanchen \- opened about 1 hour ago
Unsloth AI org about 1 hour ago
Hey everyone! We’re releasing new Qwen3.8-27B GGUFs with 10% higher accuracy for the same size. This uses a new version of Dynamic v3.0
Unsloth Dynamic V3 outperforms others by >10% on Div-300, KLD & more benchmarks. We also release 1-bit quants that retain 77% accuracy. Run on 8GB RAM.
---
Some of you already saw we updated our quants a few hours ago. No, nothing was broken, nothing needed fixes (I don't know why people even said this since it's a complete fabricated story). This was purely an update to make them EVEN BETTER.
We do not train on the imatrix calibration dataset, and we do NOT use QAT or QAD. Everything is done through post-training quantization. Our imatrix file used is available for the community to test, evaluate, and use. We encourage researchers and developers to create variations and fine-tunes of Qwen3.8 using our Unsloth quants/imatrix. You can read our over fitting analysis as well.
Blog with all details and more benchmarks: <https://unsloth.ai/docs/basics/dynamic-3.0-ggufs>
GGUF: <https://huggingface.co/unsloth/Qwen3.8-27B-GGUF>
Enjoy! We also will be doing a new Unsloth Desktop update today: <https://github.com/unslothai/unsloth>
We had A LOT of updates and will be introducing auto compaction, allowing external APIs to do tool calling and more.
See translation
🔥
15
15
👍
10
10
🤗
2
2
+
danielhanchen pinned discussion about 1 hour ago
Thank you so much for your excellent quantization work. I saw that you create a new folder named `MTP`, does that mean that you have seperated the mtp from the model gguf? Now if we want mtp acceleration, must we download the seperate mtp.gguf? Could you please make a clear instruction on how to use mtp acceleration for this model?
See translation
What happened with non-dynamic quants files like `Qwen3.8-27B-Q4_K_M.gguf`? I see they were deleted.
See translation
This is great. Can you run TB 2.1 for some quants?
See translation
From the ud3 description: "We also removed the MTP module from smaller quants under UD-Q2\_K\_XL (8.37GB and lower) to converse around 500MB of disk space - you can use the Q4\_0 MTP separate module if needed"
See translation
•
works well in LM Studio for me, but for some reason when using llama.cpp it crashes during token generation. Any suggestions? using bare bone arguments`llama-server -fa on --fit on -c 200000 --model Qwen3.8-27B-UD-Q5_K_XL.gguf`
See translation
EditPreview
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Comment