Top all-time · #15014
Jeremy Bernstein
Personal blog · jeremybernste.in/writingaimathprogramming
Jeremy Bernstein's personal blog at jeremybernste.in covers ai, math and programming. Since 2021, 5 different Hacker News readers have linked it 7 times, most often “Deriving Muon”. It is still writing: the last post was on 2025-03-07, and you can follow it by feed.
Too few sampled comments (2) to say how often HN recommends it.?
Why HN links it
The comments discuss neural network optimization, including derivative-free methods versus gradient-based training and lottery ticket sparse networks. They do not assess the blog's writing.
Summarised by Claude from a sample of 2 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Most-linked posts?
- Deriving Muon5 people · 5 mentions · first linked 2025-03-07
What Hacker News says?
Every few years, a derivative-free neural network optimization algorithm gets some hype. I'd bet my life savings that none of them ever make an impact. Derivative-free optimization can be useful for genuinely discontinuous objectives [1], but common neural network objectives are smooth and/or Lipschitz. The gradient is useful. Instead of trying random directions and hoping that one of them is an improvement, it tell…
Comment linking /writing/deriving-muon
on “Dust: Pretraining Transformers Without Backpropagation”Submitted Deriving Muon as a story · 7 points
HN discussionYes to this. Furthermore: - you can solve neural networks in analytic form with a hodge star approach* [0] - if you use a picture to set your initial weights for your nn, you can see visually how close or far your choice of optimizer is actually moving the weights - eg Adam looks like it barely changes things whereas Muon changes the weights much more to the point you cannot recognize the originals [1] *unfortunatel…
Comment linking /writing/deriving-muon
on “The Lottery Ticket Hypothesis: finding sparse trainable NNs with 90% less params”Submitted Machine learning is just statistics and quantifier reversal as a story · 4 points
HN discussionSubmitted Deriving Muon as a story · 2 points
HN discussionSubmitted Deriving Muon as a story · 2 points
HN discussionSubmitted Deriving Muon, a geometry inspired neural architecture as a story · 1 points
HN discussion