Top all-time · #2146
Vals AI Blog
Company blog · vals.ai/blogsaidata
Vals AI Blog is the company blog at vals.ai, writing about ai and data. Since 2025, 35 different Hacker News readers have linked it 40 times, most often “Fable 5.1 Solves the Cyphral Distich, a 370-year-old cipher”. It is still writing: the last post was on 2026-10-28, and you can follow it by feed.
Too few sampled comments (18) to say how often HN recommends it.?
Why HN links it
Vals AI publishes third-party benchmarks and leaderboards for AI models, covering agents, legal tasks, coding and cost. HN commenters cite them as checks on vendors' self-reported scores.
Summarised by Claude from a sample of 18 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Vals AI independently tests AI models on real-world work. Explore benchmark results, model comparisons and research across coding, finance, law and more.
Most-linked posts?
- Fable 5.1 Solves the Cyphral Distich, a 370-year-old cipher5 people · 5 mentions · first linked 2026-09-01
- Testing AI on Real-World Tasks2 people · 2 mentions · first linked 2026-07-20
- Mistral Large 4 Scores2 people · 2 mentions · first linked 2026-10-06
- /benchmarks/vals_index2 people · 2 mentions · first linked 2026-07-24
- How Far Will AI Assistants Go for a Refund They Can't Get?2 people · 2 mentions · first linked 2026-10-15
What Hacker News says?
Looks like it's about a year behind still. i.e. its intelligence is behind models from roughly a year ago. https://www.vals.ai/benchmarks/vals_index
Comment linking /benchmarks/vals_index
on “Mistral Large 4”Submitted Fable 5.1 Solves the Cyphral Distich, a 370-year-old cipher as a story · 1218 points
HN discussion- nl · 2026-10-07
This doesn't seem true. I'm not aware of any benchmarks that measure the first "analyze XYZ geopolitical situation" but legal reasoning is very closely related to "explain the ramifications of XYZ law". Legal Bench[1] measures legal reasoning. It's close to saturated (ie, there isn't a lot of room for improvement) but Fable scores 88% vs eg Opus 4.7 at 85%. There probably isn't a lot of room for improvement on somet…
Comment linking /benchmarks/legal_bench
on “Sharing AI progress in mathematics” Submitted Opus 5.5 agents discover two room-temperature magnetic semiconductor candidates as a story · 495 points
HN discussionThe benchmark numbers don't really mean anything -- Google says that Gemini 2.5 Pro has an AIME score of 86.7 which beats o3-mini's score of 86.5, but OpenAI's announcement post [1] said that o3-mini-high has a score of 87.3 which Gemini 2.5 would lose to. The chart says "All numbers are sourced from providers' self-reported numbers" but the only mention of o3-mini having a score of 86.5 I could find was from this o…
Comment linking /benchmarks/aime-2025-03-24
on “Gemini 2.5”Submitted How Far Will AI Assistants Go for a Refund They Can't Get? as a story · 108 points
HN discussionI like how they have totally excluded Meta Spark models in their blog post. They seem to be performing well on the benchmark: https://www.vals.ai/benchmarks/hlab
Comment linking /benchmarks/hlab
on “GLM-Legal”Submitted A Faster Shortest Path Algorithm as a story · 39 points
HN discussionI'm not familiar with the Vals AI Legal Research Benchmark. But their website has other frontier models' scores, and the scores OpenAI is now revealing for "Astra for Law" are slightly less than Claude and Muse: > The top is a three-way tie: Muse Spark 1.3 Max, Claude Opus 5, and Claude Fable 5.1 all reach 55.29% all-pass accuracy, a clear ~6-point step ahead of the next model. [Astra for Law reached 54.0%] > Under…
Comment linking /benchmarks/legal_research
on “Astra for Law”Submitted Fable 5.1 Solves the Cyphral Distich as a story · 10 points
HN discussionIt does still compare well against the others: https://www.vals.ai/benchmarks/swebench-2025-08-27
Comment linking /benchmarks/swebench-2025-08-27
on “Grok Code Fast 1”Submitted Frontier models score under 2/100 controlling a simulated humanoid as a story · 7 points
HN discussion"On the full subset of SWE-Bench-Verified, grok-code-fast-1 scored 70.8% using our own internal harness." Let's see this harness, then, because third party reports rate it at 57.6% https://www.vals.ai/models/grok_grok-code-fast-1
Comment linking /models/grok_grok-code-fast-1
on “Grok Code Fast 1”Submitted 2/3rds of MiMo v2.6's Coding Tasks Leak the Answer. Is Exploiting It Cheating? as a story · 7 points
HN discussionI know Arc AGI 2 has a private test set and they have a good amount of results[0] but it's not a conventional benchmark. Looking around, SWE Rebench seems to have decent protection against training data leaks[1]. Kagi has one that is fully private[2]. One on HuggingFace that claims to be fully private[3]. SimpleBench[4]. HLE has a private test set apparently[5]. LiveBench[6]. Scale has some private benchmarks but no…
Comment linking /about
on “Google releases Gemma 4 open models”Submitted Vibe Code Bench as a story · 5 points
HN discussionActually… (tw: bitter lesson) https://www.vals.ai/blogs/ai-detection-benchmark
Comment linking /blogs/ai-detection-benchmark
on “SynthID Detector”Submitted How pro-China are Chinese models? as a story · 4 points
HN discussionAlthough there is a part on the use of AI in the legal market, but it is neither comprehensive, nor informative. A much better view of the law-specific state of the art models is available at https://www.vals.ai/vlair (although registration is needed to access the report). That benchmark in the report clearly shows that there is not much added value or moat in the custom training of Harvey (costing supposedly ~5M$),…
Comment linking /vlair
on “AI Market Clarity”Submitted AI Cheating Is on the Rise as a story · 4 points
HN discussionBenchmarks: https://artificialanalysis.ai/models/mistral-large-4 https://www.vals.ai/models/mistralai_mistral-large-4
Comment linking /models/mistralai_mistral-large-4
on “Mistral Large 4”Submitted ProgramBench as a story · 4 points
HN discussionAlso the cost per task. https://www.vals.ai/benchmarks/vals_index !!! Vals !!! Vals Index Opus 4.8 > 5.0 goes from $2.90 to $8.54, for 4% gain ... That is a massive cost increase. Sure, 20% cheaper then Fable, but that is a 3x price increase compared to Opus 4.8 in that test. https://artificialanalysis.ai/models/claude-opus-5 https://artificialanalysis.ai/models/claude-opus-5#price-cos... !!! artificial analysis !!…
Comment linking /benchmarks/vals_index
on “Claude Opus 5”Submitted Mistral Large 4 Scores as a story · 3 points
HN discussion> Try grok with spacex api maybe They did, it lost to Opus 4.8: https://www.vals.ai/benchmarks/time_horizon_index
Comment linking /benchmarks/time_horizon_index
on “GPT-5.6 Sol vs. Kimi K3 Speedrunning Kerbal Space Program Live”Submitted Testing AI on Real-World Tasks as a story · 3 points
HN discussionthis is a much better link - https://www.vals.ai/blogs/fable-solves-cyphral-distich
Comment linking /blogs/fable-solves-cyphral-distich
on “Claude Fable 5.1 cracked a cipher that has survived for 370 years in 40 min.”Submitted Vals.ai Benchmark – International Olympiad in Informatics as a story · 3 points
HN discussionSorry, what I meant is if third party has them in their leaderboards. I don't usually trust most of what any of these vendors claim in their release notes without a third party. I know it says "verified" there, but I don't see were the SWE bench results are from a third party, whereas for the "HLE-Verified" they do have a citation to Hugging Face. I was looking for something closer to: https://www.vals.ai/benchmarks…
Comment linking /benchmarks/swebench
on “Qwen3.5: Towards Native Multimodal Agents”Submitted Claude Fable 5.1 Solves the Cyphral Distich as a story · 3 points
HN discussion
