Top all-time · #25011
Georg Grab
Personal blog · georggrab.netaiprogramming
Georg Grab's personal blog at georggrab.net covers ai and programming. Since 2026, 3 different Hacker News readers have linked it 5 times, most often “Does Opus 4.6 find the needle in the haystack?”. No feed was found, so read it on the site itself.
Too few sampled comments (4) to say how often HN recommends it.?
Why HN links it
HN commenters cite this blog's experiment on Claude Opus 4.6 retrieval, which found the model answered from pre-training knowledge rather than the supplied context, as evidence of LLM failures.
Summarised by Claude from a sample of 4 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Software Engineer at Apple, M.Sc Math and Computer Science at MPI-IS and Heidelberg University. Personal Website and Blog.
Most-linked posts?
- Does Opus 4.6 find the needle in the haystack?3 people · 5 mentions · first linked 2026-02-10
What Hacker News says?
I've run that experiment now, spoiler: It cheated with its pre-training knowledge https://georggrab.net/content/opus46retrieval.html
Comment linking /content/opus46retrieval.html
on “Claude Opus 4.6”Submitted Does Opus 4.6 find the needle in the haystack? as a story · 3 points
HN discussionRelated: https://georggrab.net/content/opus46retrieval.html From 90% to 0% success rate once the problem was reformulated to exclude memorization.
Comment linking /content/opus46retrieval.html
on “Writing a "clear room" Z80 and Spectrum emulator with Claude Code”Well here's some: Confabulation/Hallucination - https://github.com/lechmazur/confabulations Failure to read context - https://georggrab.net/content/opus46retrieval.html Deleting tests to make them pass - https://www.linkedin.com/posts/jasongorman_and-after-it-did-... Going rogue and deleting data - https://x.com/jasonlk/status/1946069562723897802 Agent security nightmares because they are not in fact intelligent ass…
Comment linking /content/opus46retrieval.html
on “I don't know how you get here from "predict the next word."”I may be using the wrong terms, my impression was: 1. Weights in the model are created by ingesting the corpus 2. Techniques like reinforcement learning, alignment etc are used to adjust those weights before model release 3. The model is used and more context injected which then affects which words it will choose, though it is still heavily biased by the corpus and training. That could be way off base though, I'd we…
Comment linking /content/opus46retrieval.html
on “I don't know how you get here from "predict the next word."”