Top all-time · #11772
mobiusml.github.io
Company blog · mobiusml.github.ioaiprogramming
mobiusml.github.io is the company blog at mobiusml.github.io, writing about ai and programming. Since 2023, 6 different Hacker News readers have linked it 11 times, most often “Half-Quadratic Quantization of Large Machine Learning Models”. No feed was found, so read it on the site itself.
Too few sampled comments (1) to say how often HN recommends it.?
Why HN links it
HN readers link this blog for its 1-bit model and HQQ quantization posts. One commenter questions the Llama2-7B results and asks what "group-size of 8" means, citing the HQQ post.
Summarised by Claude from a sample of 1 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Most-linked posts?
- Half-Quadratic Quantization of Large Machine Learning Models3 people · 5 mentions · first linked 2023-12-07
What Hacker News says?
Really strong binary results. So strong it was fishy. I hope someone can explain my confusion below. > We compared the performance of the Llama2-7B model in three configurations: FP16 (full precision), HQQ (without fine-tuning), and HQQ+ (with adapter layers) using a group-size of 8. Interesting, what is "group-size of 8"? From their HQQ post ( https://mobiusml.github.io/hqq_blog/ ), it's the block size at which the…
Comment linking /hqq_blog/
on “Towards 1-bit Machine Learning Models”Submitted Towards 1-bit Machine Learning Models as a story · 351 points
HN discussionWe are releasing 2-bit and 4-bit quantized versions of Mixtral utilizing the HQQ method that we just published https://mobiusml.github.io/hqq_blog/ and https://github.com/mobiusml/hqq . The 2-bit version can run on a 24GB Titan RTX. In terms of perplexity scores on the wikitext2 dataset, the results are as follows: Mixtral: 26GB / 3.79 Llama2-70B: 26.37GB / 4.13
Comment linking /hqq_blog/
on “2-bit and 4-bit versions of Mixtral”Submitted Gemlite: Towards Building Custom Low-Bit Fused CUDA Kernels as a story · 47 points
HN discussionI'm sharing a blog post https://mobiusml.github.io/low-rank-llama2/ on our approach to pruning the Llama2 model by leveraging low-rank structures. In a nutshell, we've managed to reduce the model's parameter count by up to 50%, double the training speed, and increase inference speed by 1.25 times. For those interested in the technical details or looking to replicate our results, the code is openly available for comm…
Comment linking /low-rank-llama2/
on “Low-Rank Pruning of Llama2”Submitted Faster and Smaller Whisper: A Deep Dive into Quantization and Torch Compilation as a story · 3 points
HN discussionSharing our work on model quantization. - Blog: https://mobiusml.github.io/hqq_blog/ - Code: https://github.com/mobiusml/hqq - Models: https://huggingface.co/mobiuslabsgmbh/ No data calibration needed, extremely fast , works on both language and vision models! * Why does it matter? Quantization significantly reduces GPU memory requirements but degrades the quality of the models. Having faster and more accurate quant…
Comment linking /hqq_blog/
on “Half-Quadratic Quantization of Large Machine Learning Models”Submitted Aana SDK: open-source library for building multimodal AI applications as a story · 2 points
HN discussionSubmitted Low-Rank Pruning of Llama2 as a story · 2 points
HN discussionSubmitted Half-Quadratic Quantization of Large Machine Learning Models as a story · 1 points
HN discussionSubmitted Half-Quadratic Quantization of Large Machine Learning Models as a story · 1 points
HN discussion
