My "4-bit" quant was 6.2 bits per weight — The Koleslaw Blog

koleslaw.ai · 2026-08-03 · by Long Nguyen ·generic
Nemotron's hidden dimension is 2688, which is not divisible by 256. That one fact quietly inflated a 4-bit quant by a third and cost a weekend.

My "4-bit" quant was 6.2 bits per weight — The Koleslaw Blog

Comments (0)

No comments yet