Top all-time · #3223
P
Peter Bloem
Personal blog · peterbloem.nl/blogdataaimath
Peter Bloem's personal blog at peterbloem.nl covers data, ai and math. Since 2019, 23 different Hacker News readers have linked it 28 times, most often “Transformers from Scratch (2019)”. It is still writing: the last post was on 2026-08-16, and you can follow it by feed.
28mentions?
23HN people?
2019–2026first – last linked?
Repository for personal website
Most-linked posts?
- /blog/transformers7 people · 8 mentions · first linked 2022-08-08
- Transformers from Scratch (2019)5 people · 6 mentions · first linked 2021-01-05
- Transformers from Scratch5 people · 5 mentions · first linked 2019-08-23
- The Singular Value Decomposition2 people · 2 mentions · first linked 2022-08-07
What Hacker News says?
Yes this seems like an early work in progress, compared to Jay's previous Transformer articles. In addition to your link, I've found a really good Transformer explanation here (backed by a Github repo w/ lively Issues talk): http://www.peterbloem.nl/blog/transformers Additionally, there's a paper on visualizing self-attention: https://arxiv.org/pdf/1904.02679.pdf
Comment linking /blog/transformers
on “How GPT3 Works – Visualizations and Animations”Submitted Transformers from Scratch as a story · 265 points
HN discussionThis is the best article I have read so far explaining the transformer architecture. The clear and intuitive explanation can’t be praised enough. Note that the teacher has a Machine Learning course with video lectures on youtube that he references throughout the article : http://www.peterbloem.nl/teaching/machine-learning
Comment linking /teaching/machine-learning
on “Transformers from Scratch”Submitted Transformers from Scratch (2019) as a story · 63 points
HN discussionThis is a great comprehensive deep dive, thank you for sharing! Interestingly, there was a blog post and video in 2019 by Peter Bloem [1] with the same title which I consider to be one of the very best quick intros to transformers. Rather than diving into all of the details right away, Peter focuses on the intuition and “peels the onion” gradually. [1] http://peterbloem.nl/blog/transformers Other notable good intros…
Comment linking /blog/transformers
on “Transformers from Scratch”Submitted The Singular Value Decomposition as a story · 25 points
HN discussionIt's interesting to see how human understanding differs when it comes to complex, yet clearly defined topics, like machine-learning/Transformers. For comparison, my understanding of Transformers, after going through Peter Bloem's "Transformers from scratch" [1], implementing/understanding the code and the actual flow of the mathematical quantities, my understanding is that: - Transformers consist of 3 main parts: 1.…
Comment linking /blog/transformers
on “Transformers from Scratch”Submitted A Friendly Introduction to Principal Component Analysis as a story · 21 points
HN discussionIn case you find Attention (and specially transformers) interesting I have some saved links introducing the topic: http://www.peterbloem.nl/blog/transformers https://nostalgebraist.tumblr.com/post/185326092369/the-tran... https://papers.nips.cc/paper/7181-attention-is-all-you-need.... http://jalammar.github.io/illustrated-transformer/ https://arxiv.org/pdf/1807.03819.pdf
Comment linking /blog/transformers
on “Attention Mechanism in Deep Learning”Submitted AI Coding Without the Vibes as a story · 2 points
HN discussionThere are millions of "Transformers Explained" blog posts by now. The one I got the most out of is "Transformers from Scratch" by Peter Bloem: http://peterbloem.nl/blog/transformers
Comment linking /blog/transformers
on “Transformers for software engineers”Submitted The Experience Feedback Loop as a story · 1 points
HN discussionHere you go: http://www.peterbloem.nl/blog/transformers
Comment linking /blog/transformers
on “Introduction to Bert”Submitted Unraveling Principal Component Analysis as a story · 1 points
HN discussionyou can catch up very quickly. great resource on transformers, which underpin the openai breakthroughs (the "T" in GPT-3 stands for transformer): http://peterbloem.nl/blog/transformers great reddit sub for tracking ml research and trends: https://www.reddit.com/r/machinelearning great email newsletter on ml from andrew ng: https://www.deeplearning.ai/thebatch/ also, someone must maintain old code or things break. th…
Comment linking /blog/transformers
on “Clip: Connecting Text and Images”Submitted What Design Can Teach Us as a story · 1 points
HN discussionIMHO this is a very good intuitive intro to transformers by Peter Bloem. It does a great job of starting with basic intuition and gradually adding in complexity. Once you read it you’ll have a good mental picture of how self attention works. Blogs allow this type of informal, relaxed style of explanation that academic papers can’t afford (page number submission limits etc ) http://peterbloem.nl/blog/transformers
Comment linking /blog/transformers
on “The potential of transformers in reinforcement learning”Submitted The Singular Value Decomposition as a story · 1 points
HN discussionI'm the author of https://jalammar.github.io/illustrated-transformer/ and have spent years since introducing people to Transformers and thinking of how best to communicate those concepts. I've found that different people need different kinds of introductions, and the thread here includes some often cited resources including: https://peterbloem.nl/blog/transformers https://e2eml.school/transformers.html I would also…
Comment linking /blog/transformers
on “Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper”Incidentally Peter Bloem has one of the best intros to transformer models. https://peterbloem.nl/blog/transformers
Comment linking /blog/transformers
on “The Singular Value Decomposition”not sure what 'mixing LLM models' entails but these are maybe some good starting points - karpathy - https://www.youtube.com/watch?v=kCc8FmEb1nY - https://towardsdatascience.com/beautifully-illustrated-nlp-m... - https://dzone.com/articles/a-deep-dive-into-the-transformer-... - https://peterbloem.nl/blog/transformers - http://nlp.seas.harvard.edu/2018/04/03/attention.html - https://lilianweng.github.io/posts/2023-01…
Comment linking /blog/transformers
on “Ask HN: Open source LLM for commercial use?”And perhaps things like: https://www.youtube.com/watch?v=VPRSBzXzavo and http://jalammar.github.io/illustrated-transformer/ and https://peterbloem.nl/blog/transformers See also: https://news.ycombinator.com/item?id=35990118
Comment linking /blog/transformers
on “Ask HN: Resources to brush up from 'Intro to ML' to current LLMs/generative AI?”Transformers from scratch: 1.) https://e2eml.school/transformers.html https://news.ycombinator.com/item?id=35697627 (46 comments, 1 month ago) https://news.ycombinator.com/item?id=29315107 (17 comments, 2 years ago) 2.) http://www.peterbloem.nl/blog/transformers https://news.ycombinator.com/item?id=20773992 (28 comments, 4 years ago) https://news.ycombinator.com/item?id=29280909 (9 comments, 2 years ago) Source: htt…
Comment linking /blog/transformers
on “CS25 Stanford Seminar – Introduction to Transformers with Andrej Karpathy”Tangential to this post but Peter Bloem’s Transformers from Scratch (2019) was one of the best early tutorials on transformers and self-attention. There have been many more tutorials since then but this was the one that solidified a mental picture for me. https://peterbloem.nl/blog/transformers
Comment linking /blog/transformers
on “AI Coding Without the Vibes”I recently stumbled across a truly excellent 5-part deep dive into PCA on Peter Bloem's blog, which I highly recommend to anyone who's interested. It assumes you have some level of linear algebra knowledge. https://peterbloem.nl/blog/pca
Comment linking /blog/pca
on “A tutorial on Principal Components Analysis [pdf]”besides everything that was mentioned here, what made it finally click for me early in my journey was running through this excellent tutorial by Peter Bloem multiple times https://peterbloem.nl/blog/transformers highly recommend
Comment linking /blog/transformers
on “Transformers from Scratch”It took me a long time to wrap my head around the whole "key/query/value" explanation (and to be honest I regularly forget which vector is which). I find the "weighted sum of vectors" explanation much simpler/more intuitive; this blog post is IMO the best on the subject: https://peterbloem.nl/blog/transformers
Comment linking /blog/transformers
on “The Transformer Family”For a while now, an answer I've seen is to start with "Attention Is All You Need", the original Transformers paper. It's still pretty good, but over the past year I've led a few working sessions on grokking transformer computational fundamentals and they've turned up some helpful later additions that simplify and clarify what's going on. You can quickly get overwhelmed by the million good resources out there so I'll…
Comment linking /blog/transformers
on “The Transformer Family”
