Top all-time · #10089
Keller Jordan
Personal blog · kellerjordan.github.io/postsaiprogramming
Keller Jordan's personal blog at kellerjordan.github.io covers ai and programming. Since 2024, 8 different Hacker News readers have linked it 9 times, most often “Muon: An optimizer for hidden layers in neural networks”. No feed was found, so read it on the site itself.
Too few sampled comments (5) to say how often HN recommends it.?
Why HN links it
HN commenters link this blog for its post on the Muon optimizer, cited as a reference for LLM training. Comments discuss optimizer benchmarking and the modded-nanoGPT project.
Summarised by Claude from a sample of 5 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Most-linked posts?
- Muon: An optimizer for hidden layers in neural networks7 people · 7 mentions · first linked 2024-12-10
What Hacker News says?
> nanochat is also inspired by modded-nanoGPT Nice synergy here, the lineage is: Karpathy's nano-GPT -> Keller Jordan's modded-nanoGPT (a speedrun of training nanoGPT) -> NanoChat modded-nanoGPT [1] is a great project, well worth checking out, it's all about massively speeding up the training of a small GPT model. Notably it uses the author's Muon optimizer [2], rather than AdamW, (for the linear layers). [1] https:…
Comment linking /posts/muon/
on “NanoChat – The best ChatGPT that $100 can buy”Submitted Muon: An optimizer for hidden layers in neural networks as a story · 4 points
HN discussion> Apologies if this came across the wrong way I was uncertain but your other statements made me think that sentiment was unintentional. I just want to push back against it because it is too common and misused even with good intentions. I hope you don't see this as me saying anything about your character. Honestly, impressions are that you do care. > It sounds like the generalisation of projected gradient decent to "…
Comment linking /posts/muon/
on “Modular Manifolds”Submitted Muon: An optimizer for hidden layers in neural networks as a story · 3 points
HN discussionIndeed. Especially when a lot of papers are just using cherry-picked results that show some improvements just so they can publish something, but their method doesn't work that well when it comes in contact with reality (e.g. see the deluge of papers which claim to have come up with an optimizer better than AdamW), and when the majority of people are not even properly benchmarking their new methods wrt to the time ov…
Comment linking /posts/muon/
on “Multi-Token Attention”Submitted Muon: An optimizer for hidden layers in neural networks as a story · 2 points
HN discussion> Performance of BDH-GPU and GPTXL versus model size on a translation task. [...] On the other hand, GPTXL [...] required Dropout [...] The model architecture follows GPT2 I love when a new architecture comes out, but come on, it's 2025, can we please stop comparing fancy new architectures to the antiquated GPT2? This makes the comparison practically, well, useless. Please pick something more modern! Even the at-thi…
Comment linking /posts/muon/
on “The Dragon Hatchling: The missing link between the transformer and brain models”Submitted Muon: An optimizer for hidden layers in neural networks (2024) as a story · 1 points
HN discussionFor people who want to know more about the Muon optimizer: https://kellerjordan.github.io/posts/muon/
Comment linking /posts/muon/
on “Muon Is Scalable for LLM Training”
