Top all-time · #16668
Arjun Bansal
Personal blog · arjunbansal.substack.comaiscience
Arjun Bansal writes LLMs for Engineers, a personal blog at arjunbansal.substack.com about ai and science. Since 2024, 4 different Hacker News readers have linked it 7 times, most often “LLMs know more than what they say”. It is still writing: the last post was on 2026-10-08, and you can follow it by feed.
Too few sampled comments (4) to say how often HN recommends it.?
Why HN links it
HN readers link this blog for posts on whether LLMs internally know when they hallucinate. Commenters cite it alongside papers on model uncertainty, and one promotes the Log10 AutoFeedback evals product.
Summarised by Claude from a sample of 4 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Writing about scaling LLM powered applications: from prototype to production.
Most-linked posts?
- LLMs know more than what they say2 people · 2 mentions · first linked 2024-08-19
What Hacker News says?
Thanks for the q. The LSR version of AutoFeedback is on our Pro tier. You can sign up for more info on that here: https://tally.so/r/w2XVQA In the free trial of Log10.io you can use the ICL (self-improving) version of AutoFeedback for hallucination detection and RAG evals. In generalize, you can customize evals for a wide range of use cases using AutoFeedback: https://docs.log10.io/feedback/auto_feedback https://git…
Comment linking /p/hybrid-evaluation-scaling-human-feedback
on “LLMs know more than what they say”Submitted LLMs know more than what they say as a story · 148 points
HN discussionRelated: GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - https://arxiv.org/abs/2310.06824 The Internal State of an LLM Knows When It's Lying - https://arxiv.org/abs/2304.13734 LLMs Know More Than What They Say…
Comment linking /p/llms-know-more-than-what-
on “LLMs know more than they show: On the intrinsic representation of hallucinations”Submitted AI-Generated Medical Text: How Accurate Is It Really? as a story · 2 points
HN discussionPretty much all research (and there's a fair few with different methodologies) on this converge on the same conclusion: LLMs internally know a lot more about the uncertainty and factualness of their predictions than they say. "LLMs are always hallucinating" is a popular stance but wrong all the same. Maybe rather than asking Why models hallucinate, the better question is to ask "Why not?". During pre-training, there…
Comment linking /p/llms-know-more-than-what-
on “GPTs and Hallucination”Submitted Open-Weight LLMs Have Caught Up on Accuracy as a story · 1 points
HN discussionThat's not true. There's been several papers probing this with different methodologies and the conclusion is pretty clear. LLMs know a whole lot more about the uncertainty of their predictions than they say. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 The Geometry of Truth: Emergent Linear Structure in Large Language…
Comment linking /p/llms-know-more-than-what-they-say
on “AI is growing faster than companies can secure it, warn industry leaders”