Top all-time · #1974
Neel Nanda
Personal blog · neelnanda.ioaiscience
Neel Nanda's personal blog at neelnanda.io covers ai and science. Since 2021, 40 different Hacker News readers have linked it 50 times, most often “Othello-GPT Has a Linear Emergent World Representation (2023)”. It is still writing: the last post was on 2025-08-19, and you can follow it by feed.
Most-linked posts?
- Othello-GPT Has a Linear Emergent World Representation (2023)11 people · 12 mentions · first linked 2023-04-04
- /mechanistic-interpretability4 people · 5 mentions · first linked 2024-01-03
- Intentionally Making Close Friends (2021)4 people · 4 mentions · first linked 2021-06-29
- On Slack: Having room to be excited (2020)3 people · 3 mentions · first linked 2021-02-04
- Concrete Steps to Get Started in Transformer Mechanistic Interpretability3 people · 3 mentions · first linked 2023-04-27
- /mechanistic-interpretability/glossary2 people · 2 mentions · first linked 2025-05-28
What Hacker News says?
I am building https://linklonk.com and I think it adds a social proof to content discovery while preserving anonymity. Here is how it works: - When you upvote an item, you connect stronger to users that also posted that item and to RSS feeds that posted this. For example, if you upvote "Post 43: Intentionally Making Close Friends — Neel Nanda"( https://linklonk.com/item/827619236936941568 ) then you will get connect…
Comment linking /blog?format=rss
on “RSS is wonderful”Submitted Intentionally Making Close Friends (2021) as a story · 330 points
HN discussion> LLMs can't think. They are generating tokens one at a time Huh? They are generating tokens one at a time - sure that's true. But who's shown that predicting tokens one at a time precludes thinking? It's been shown that the models plan ahead, i.e. think more than just one token forward. [1] How do you explain the world models that have been detected in LLMs? E.g. OthelloGPT [2] is just given sequences of games to t…
Comment linking /mechanistic-interpretability/othello
on “Thoughts on thinking”Submitted The goal of standards is to improve our future actions as a story · 260 points
HN discussion> This is speculation at best. Models are black boxes, even to those who make them. We can't discern a "meaningful internal representation" in a model, anymore than a human brain. They are not pure black boxes. They are too complex to decipher, but it doesn't mean we can't look at activations and get some very high level idea of what is going on. For world models specifically, the paper that first demonstrated that…
Comment linking /mechanistic-interpretability/othello
on “AI is different”Submitted The world is full of wasted motion as a story · 122 points
HN discussionI was not familiar with it, and that does look fascinating, thank you. If anyone else is interested, this guide "Concrete Steps to Get Started in Transformer Mechanistic Interpretability" on his site looks like a great place to start: https://www.neelnanda.io/mechanistic-interpretability/gettin...
Comment linking /mechanistic-interpretability/getting-started
on “Understand how transformers work by demystifying the math behind them”Submitted When you already know the answer as a story · 52 points
HN discussionYou might be interested, if you aren't already familiar, in some of the work going on in the mechanistic interpretability field. Neel Nanda has a lot of approachable work on the topic: https://www.neelnanda.io/mechanistic-interpretability
Comment linking /mechanistic-interpretability
on “Understand how transformers work by demystifying the math behind them”Submitted Socratic Persuasion: Giving Opinionated yet Truth-Seeking Advice as a story · 3 points
HN discussionI find this article a nice exploration of slack concept in personal lives https://www.neelnanda.io/blog/38-slack
Comment linking /blog/38-slack
on “Care Doesn't Scale”Submitted Good Research Takes Are Not Sufficient for Good Strategic Takes as a story · 3 points
HN discussionLLM based AIs have lots of "features" which are kind of synonymous with "concepts" - these can be anything from `the concept of an apostrophe in the word don't`, to `"George Wash" is usually followed by "ington" in the context of early American History`. Inside of the LLMs neural network, these are mapped to some circuitry-in-software-esque paths. We don't really have a good way of understanding how these features a…
Comment linking /mechanistic-interpretability
on “Extracting concepts from GPT-4”Submitted Intentionally Making Close Friends as a story · 3 points
HN discussionThis post we're commenting on. Also https://www.neelnanda.io/mechanistic-interpretability/othell... Also, there's a paper I can't find now which shows you can find the name of the operation in the middle layers if you provide example of what to do. That is, if you prompt: "9 2 5 -> 5 2 9, 0 7 3 ->", you'll find the internals generate "reverse" even though it doesn't appear in the output.
Comment linking /mechanistic-interpretability/othello
on “Generative Models: What do they know? Do they know things? Let's find out”Submitted Intentionally Making Close Friends as a story · 3 points
HN discussionNeel Nanda did it better: https://www.neelnanda.io/blog/34-learning
Comment linking /blog/34-learning
on “Learning to Learn”Submitted How to Teach Things Well as a story · 2 points
HN discussion>We don't build our world model fresh with every sentence someone says to us Neither do LLMs. The state for the current text perhaps. Definitely not the entire world model(s) which is learnt from the training process and stored in its weights. >They quite literally have no ability to have a "world model" You keep repeating this so let's get one thing straight. You're wrong. You're just wrong. I'm not trying to convi…
Comment linking /mechanistic-interpretability/othello
on “Hallucination is inevitable: An innate limitation of large language models”Submitted On Reflection as a story · 2 points
HN discussionYou seem to believe "concept" is a concept in English language. It is not. Concept is an abstraction layer above human languages. Here's a good article that touched on this topic: https://www.neelnanda.io/mechanistic-interpretability/glossa...
Comment linking /mechanistic-interpretability/glossary
on “AI: Accelerated Incompetence”Submitted Intentionally Making Close Friends as a story · 2 points
HN discussion> I'd argue that the biggest reason machines are black boxes are because no one is bothering to look inside of them. People do look, but it's extremely hard. Take a look at how hard the mechanistic interpretability people have to work for even small insights. Neel Nanda[1] has some very nice writeups if you haven't already seen them. [1]: https://www.neelnanda.io/mechanistic-interpretability
Comment linking /mechanistic-interpretability
on “The cultural divide between mathematics and AI”Submitted A Barebones Guide to Mechanistic Interpretability Prerequisites as a story · 2 points
HN discussionNeel Nanda is also very active in the field and writes some potentially more approachable articles, if you're interested: https://www.neelnanda.io/mechanistic-interpretability Much of their work is focused on discovering "circuits" that occur between layer activations as they process data, which correspond to dynamics the model has learned. So, as a simple hypothetical example, instead of embedding the answer to 1 m…
Comment linking /mechanistic-interpretability
on “'Positive review only': Researchers hide AI prompts in papers”Submitted What's Stopping You? as a story · 2 points
HN discussionNot going to comment on the thinking part, because who knows what that means, but there's evidence that transformers do in fact learn predictive models of their input space. There's a cool blog post on this here: https://www.neelnanda.io/mechanistic-interpretability/othell...
Comment linking /mechanistic-interpretability/othello
on “Reasoning in Large Language Models: A Geometric Perspective”Submitted On Systems: Living a Life of Zero Willpower as a story · 2 points
HN discussionRelated work: Interpreting Modular Addition in MLPs https://www.lesswrong.com/posts/cbDEjnRheYn38Dpc5/interpreti... Paper Replication Walkthrough: Reverse-Engineering Modular Addition https://www.neelnanda.io/mechanistic-interpretability/modula...
Comment linking /mechanistic-interpretability/modular-addition-walkthrough
on “Pre-Trained Large Language Models Use Fourier Features for Addition (2024)”Submitted Prioritisation, Part 1 – Finding Goals as a story · 1 points
HN discussion
