Top all-time · #19892
Shyam's Blog
Personal blog · shyam.blog/postsaiprogramming
Shyam's Blog is a personal blog at shyam.blog about ai and programming. Since 2024, 3 different Hacker News readers have linked it 3 times, most often “Beyond self-attention: How a small language model predicts the next token”. The last post was in 2024, but the archive is still widely read.
Too few sampled comments (1) to say how often HN recommends it.?
Why HN links it
One HN commenter links the blog's discussion of LLM internals, asking about inner-layer permutability and how attention and feed-forward displacements align. The post seems to cover how LLM layers relate internally.
Summarised by Claude from a sample of 1 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Most-linked posts?
- Beyond self-attention: How a small language model predicts the next token2 people · 2 mentions · first linked 2024-02-04
What Hacker News says?
They inner layer permutability is super interesting. Is that result published anywhere? That's consistent with this graph here, which seems to imply different layers are kind of working in very related latent spaces. If you skip to the graph here that shows the attention + feed forward displacements tending to align (after a 2d projection), is this something known/understood? Are the attention and feed forward displ…
Comment linking /posts/beyond-self-attention/
on “Ask HN: Do LLMs get "better" with more processing power and or time per request?”Submitted Beyond self-attention: How a small language model predicts the next token as a story · 474 points
HN discussionSubmitted Beyond Self-Attention: How a Small Language Model Predicts the Next Token as a story · 3 points
HN discussion