Top all-time · #11088
Gert Labs
Company blog · gertlabs.com/blog · on HN as gertlabsaidata
Gert Labs is the company blog at gertlabs.com, writing about ai and data. Since 2026, 7 different Hacker News readers have linked it 95 times, most often “Games for Machines”. No feed was found, so read it on the site itself.
Too few sampled comments (6) to say how often HN recommends it.?
Why HN links it
HN readers cite Gert Labs' AI model rankings, which compare models on agentic coding and competitive game environments, to support points about how open and frontier models perform against each other.
Summarised by Claude from a sample of 6 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Unsaturated RL environments for frontier labs: coding, hardware design, and scientific reasoning, with verifiable rewards for training and evaluating AI models.
Most-linked posts?
- /rankings?mode=agentic_coding3 people · 11 mentions · first linked 2026-05-08
- /rankings2 people · 31 mentions · first linked 2026-05-01
- Games for Machines2 people · 12 mentions · first linked 2026-02-26
- Gbench Intelligence Benchmark2 people · 7 mentions · first linked 2026-04-16
What Hacker News says?
We evaluated Gemini 4 Argon in 100 unsaturated, competitive and cooperative coding environments. It's roughly in between Fable 5 and Fable 5.1 in our testing, so it's maybe 2 months behind the frontier. Google's major releases have usually done pretty well on our benchmarks because they're good at hard problems and open-ended decision making (with a well specified prompt), which obviously doesn't always translate to…
Comment linking /rankings
on “Gemini 4 Argon”Submitted LLM Paper Trading as a story · 6 points
HN discussionWe tested it across 100 unsaturated coding and engineering environments. Both Astra and 6.1-Sol are pretty comfortably ahead of Opus 5.5 in these types of evaluations, and both end up being cheaper than Opus via API usage. 6.1-Sol is also cheaper than Sonnet 5.5 and much smarter. The only verifiable domain Anthropic seems to be clearly ahead is chemistry (and perhaps also some unverifiable domains like being pleasan…
Comment linking /rankings
on “GPT 6.1 Sol: Near-Astra intelligence for a fifth of the price”Submitted State of the AI Frontier, April 2026 as a story · 5 points
HN discussion> Delivering the benefits of scientific progress and economic growth that very intelligent machines enable. I think we're very close to the point where AI-driven breakthroughs outside of pure math and software start to really affect the world. We evaluated GPT-6 Astra in 100 complex, unsaturated multi-agent coding environments, competing and cooperating with other models in open-ended tasks. It's the new frontier mo…
Comment linking /rankings
on “An Alien Mind”Submitted Gbench Intelligence Benchmark as a story · 4 points
HN discussionGemini models struggle with agentic coding/tool use/exploration, but they are actually quite smart in one-shot reasoning. They're not as far behind as people think. It's mostly post-training and productization issues, which are easier to fix than pre-training/mid-training issues. Agentic coding data: https://gertlabs.com/rankings?mode=agentic_coding
Comment linking /rankings?mode=agentic_coding
on “GPT-5.6”Submitted Benchmark Fatigue as a story · 3 points
HN discussionWe have it slightly ahead of Fable in our multi-agent coding evaluations. Fable's main advantage is that its average solution size is smaller. However, GPT 5.6 Sol is a substantial improvement from GPT 5.4/5.5 which would write verbose, defensive code. 31KB for GPT 5.4/5.5 down to 26KB for GPT 5.6 Sol, with better performance for Sol. Fable scores slightly lower, but with an average solution size of 12.2 KB. Data at…
Comment linking /rankings?mode=oneshot_coding
on “GPT-5.6”Submitted Games for Machines as a story · 3 points
HN discussionOne problem is that $30 per run is really noisy for many verifiable tasks. Ours usually run at least an order of magnitude more for a model in GLM 5.3's price class. We just published GLM 5.3 results on our multi-agent coding evaluations and it's definitely an impressive model, coming in around #6. For the price, it's actually not Pareto optimal, falling slightly behind Grok 4.6 (which is faster and the same price)…
Comment linking /rankings
on “GLM-5.3 (open-weight) beat Anthropic/OpenAI models – for 1/5 the cost”Submitted Decompressible Oracle Generators as a story · 2 points
HN discussionWe ran v4.1 Flash through our evaluations and found it to be smarter and faster than V4 Flash, with a commensurate price bump. Some notes: - Because it's a heavy reasoner, it sits near Gemini 3.7 Flash on the Pareto front (not as cheap as the price suggests in practice). - Closer than expected to the top open weights models (GLM 5.3 and Kimi K3) in agentic coding, at lower cost. - Chinese models have always been str…
Comment linking /rankings
on “DeepSeek v4.1 Flash Is Now Our Best Hacking Model”Submitted Social Intelligence Benchmark as a story · 2 points
HN discussionAppreciate that! Results are live: https://gertlabs.com/rankings Opus 4.8 is the first tangible improvement since Opus 4.5. And it doesn't seem to have the personality problems of the last release -- I've been enjoying using it.
Comment linking /rankings
on “Claude Opus 4.8”Submitted LLM Performance by Programming Language as a story · 2 points
HN discussionWe just finished our initial coding evals of Opus 4.8. Anthropic definitely heard the backlash from Opus 4.7 and they made up for it today. Subjectively, it's also quite enjoyable to use (although it feels a bit slower on max reasoning), and it's the first Anthropic model that can implement a complex feature without Codex finding 100 bugs. Data at https://gertlabs.com/rankings
Comment linking /rankings
on “Claude Opus 4.8”Submitted Gemma 4 and the Economics of Selling AI as a story · 2 points
HN discussion4.5/4.6 were roughly the same in our testing. Opus 4.7 is smarter, but it's difficult to use as a product for various personality issues. So far, Opus 4.8 seems to be going down that path (unusably slow, but this could be a launch day rollout problem). Full Opus 4.8 tests are in progress now. Data at https://gertlabs.com/rankings
Comment linking /rankings
on “Claude Opus 4.8”Submitted Creating Reliable AI for an Adversarial World as a story · 1 points
HN discussionWe measure skill differentiation between frontier / last-gen LLMs across our environments, and one of our curated coding environments is a closed-system market simulator, containing only other agents and some system participants (a market maker and a liquidity provider via issuance / buybacks) whose behavior is fully defined for all of the agents. This has the least measured skill differentiation of all of our envir…
Comment linking /spectate?game=market
on “Artificial intelligence now beats some of the best human forecasters”Submitted Better Agent Leagues and Decision Supervision from Branched Rollouts as a story · 1 points
HN discussion0731 is definitely tuned for coding. I mean: https://gertlabs.com/rankings?mode=agentic_coding But it is also a decent translator from English to Czech in my experience.
Comment linking /rankings?mode=agentic_coding
on “DeepSeek V4 Pro 0813”Submitted Gbench Intelligence Benchmark as a story · 1 points
HN discussionWe only have some basic time filtering ( https://gertlabs.com/?days=30 ), but most of our samples are from the last 2 months. This is a visualization we plan to add when we've collected more historical data. But we did heavily resample Claude Opus 4.6 during the height of the degraded performance fiasco, and my takeaway is that API-based eval performance was... about the same. Claude Opus 4.6 was just never signific…
Comment linking /?days=30
on “Claude Opus 4.7”Early benchmark results on our private complex reasoning suite: https://gertlabs.com/?mode=agentic_coding Opus 4.7 is more strategic, more intelligent, and has a higher intelligence floor than 4.6 or 4.5. It's roughly tied with GPT 5.4 as the frontier model for one-shot coding reasoning, and in agentic sessions with tools, it IS the best, as advertised (slightly edging out Opus 4.5, not a typo). We're still running…
Comment linking /?mode=agentic_coding
on “Claude Opus 4.7”DeepSeek v4 Pro struggles with a custom harness, and all the models ranked above it don't, so it gets downweighted in the agentic coding benchmarks (although it ranks better than Flash in one-shot problem solving: https://gertlabs.com/rankings?ow=1&mode=oneshot_coding ). We ran plenty of samples. MiMo v2.5 is on there, as well as the pro version. We found a few anomalies in our evaluations, which makes sense -- if e…
Comment linking /rankings?ow=1&mode=oneshot_coding
on “MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second”MiMo V2.5 Pro (regular speed) remains the strongest open weights agentic coding model we've tested -- it's been interesting to see how little attention it has received relative to some lower performing releases. And the "fast mode" pricing is very competitive here. Data at https://gertlabs.com/rankings
Comment linking /rankings
on “MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second”I'm glad we're seeing a shift towards objectively scored tests. We've been doing this at scale at https://gertlabs.com/rankings , and although the author looks to be running unique one-off samples, it's not surprising to see how well Kimi K2.6 performed. Based on our testing, for coding especially, Kimi is within statistical uncertainty of MiMo V2.5 Pro for top open weights model, and performs much better with tools…
Comment linking /rankings
on “Kimi K2.6 just beat Claude, GPT-5.5, and Gemini in a coding challenge”Update: Kimi K2.5 one-shot results are live. It wasn't a noteworthy release compared to K2.6: https://gertlabs.com/?mode=oneshot_coding
Comment linking /?mode=oneshot_coding
on “Kimi K2.6: Advancing Open-Source Coding”
