Top all-time · #125
Evan Miller
Personal blog · evanmiller.orgmathdataprogramming
Evan Miller's personal blog at evanmiller.org covers math, data and programming. Since 2009, 303 different Hacker News readers have linked it 430 times, most often “How Not to Sort by Average Rating (2009)”. The last post was in 2024, but the archive is still widely read.
Sometimes recommended · 12 of 40 sampled comments recommend it?
Why HN links it
HN readers link Evan Miller's posts on ranking by average rating and confidence intervals, A/B testing and statistics, plus his Perl 6 and nginx writing.
- Recommended for · 12 comments
- "Really recommend it", "excellent article", "more generally useful", "a much better option", "interesting counterpoints".
Summarised by Claude from a sample of 40 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Articles, software, calculators, and opinions.
Most-linked posts?
- How Not to Sort by Average Rating (2009)43 people · 46 mentions · first linked 2009-02-12
- How Not To Run An A/B Test28 people · 33 mentions · first linked 2010-04-19
- How not to sort by average rating (2009)27 people · 30 mentions · first linked 2017-12-17
- How Not To Run an A/B Test (2010)9 people · 11 mentions · first linked 2017-12-01
- The Golden Football and the Economics of Groupon9 people · 9 mentions · first linked 2009-05-30
- Sample Size Calculator: How many subjects are needed for an A/B test?8 people · 8 mentions · first linked 2013-08-05
- The Joy of Erlang; Or, How to Ride a Toruk (2011)6 people · 13 mentions · first linked 2011-03-25
- Emiller's Guide To Nginx Module Development6 people · 6 mentions · first linked 2009-10-01
- The Mathematical Hacker6 people · 6 mentions · first linked 2012-12-13
- Rank Hotness with Newton's Law of Cooling6 people · 6 mentions · first linked 2010-12-21
What Hacker News says?
Yes, if you "peek" at the results of an A/B test before it's done in order to decide whether to stop early, the numbers you "peeked" at have much lower statistical power than if you foreswore stopping early. Obviously, failing to take that into account when drawing conclusions about the effectiveness of the treatment is a colossal mistake. However, the decrease in statistical power is still quantifiable, and with th…
Comment linking /bayesian-ab-testing.html
on “Ebola Is Now Curable”Submitted How Not To Sort By Average Rating as a story · 750 points
HN discussionhttps://www.evanmiller.org/how-not-to-sort-by-average-rating...
Comment linking /how-not-to-sort-by-average-rating.html
on “92 of top 500 subreddits controlled by same 5 people”Submitted Four days of Go as a story · 496 points
HN discussionBetter results, yes, but that doesn't mean good results. It can only find local optima in a predetermined state space. Training a neural network involves (1) finding the right state space, and (2) choosing a suitable gradient function. If the Correct Solution isn't in the state space, or isn't reachable via gradual improvement, the neural network will never find it. An algorithm that can reason about the meaning of…
Comment linking /attention-is-off-by-one.html
on “Please don't mention AI again”Submitted Why I'm Betting On Julia as a story · 493 points
HN discussionOP really needs to read "How not to sort by average rating" https://www.evanmiller.org/how-not-to-sort-by-average-rating...
Comment linking /how-not-to-sort-by-average-rating.html
on “Show HN: Draw a fish and watch it swim with the others”Submitted Why I’m Learning Perl 6 as a story · 433 points
HN discussionSeems like one of those cases where solving a “little” issue would actually require rearchitecting the entire system. Namely, in this case, it seems like the “right thing” is for games to not derive their ELO contributions from pure win/loss/draw scorings at all, but rather for games to be converted into ELO contributions by how far ahead one player was over the other at the point when both players stopped playing f…
Comment linking /how-not-to-sort-by-average-rating.html
on “Level 3 Global Outage”Submitted Statistical Formulas For Programmers as a story · 395 points
HN discussionhttps://www.evanmiller.org Discusses: - Programming - Applied Math And some random stuff like: Evaluating splatoon's ranking system: https://www.evanmiller.org/evaluating-splatoons-ranking-syst... Really recommend it ;)
Comment linking /evaluating-splatoons-ranking-system.html
on “Ask HN: Favorite Blogs by Individuals?”Submitted How Not to Sort by Average Rating (2009) as a story · 383 points
HN discussionDate: 9:12 AM Wednesday, April 4, 2012 From: the boss To: dev3 Subject: URGENT - front page showcase selection broken!! Body: Hey bro, I was looking into it, and our ratings average equation is totally busted and products with just a few ratings are hogging space from proven winners when it's just a sample bias. This is costing us money and needs to be fixed NOW. I'd like this up before our morning meeting so I can…
Comment linking /rating-equation.png
on “How Not To Sort By Average Rating”Submitted Elixir RAM and the Template of Doom as a story · 357 points
HN discussion> we should look at the number of votes, divided by the number of views Closer, but still not quite what you want probably or a few stray votes can make a massive impact just from discretization effects. What you really care about is which answer is "best" by some metric, and you're trying to infer that as best as possible from the voting history. Average votes do a poor job. Check out this overview from the interbl…
Comment linking /how-not-to-sort-by-average-rating.html
on “The most copied StackOverflow snippet of all time is flawed (2019)”Submitted Why I program in Erlang (2012) as a story · 325 points
HN discussionPlenty. Read https://www.evanmiller.org/why-im-learning-perl-6.html or the follow-up https://www.evanmiller.org/a-review-of-perl-6.html Or, Why Perl 6 is the "Game of Thrones" of programming languages : https://techbeacon.com/app-dev-testing/why-perl-6-game-thron... Lots more where those came from...
Comment linking /a-review-of-perl-6.html
on “Larry Wall has approved renaming Perl 6 to Raku”Submitted How not to sort by average rating (2009) as a story · 325 points
HN discussionWhat do you mean? Perl6/Raku has some great features that make it much better than Python. In addition to no GIL, there's other advantages, like discussed here: https://techbeacon.com/app-dev-testing/why-perl-6-game-thron... and https://www.evanmiller.org/why-im-learning-perl-6.html The features demonstrated in this talk are worth a watch (or at least a scroll through the slide deck): http://tpm2016.zoffix.com/
Comment linking /why-im-learning-perl-6.html
on “Larry Wall has approved renaming Perl 6 to Raku”Submitted A Review of Perl 6 as a story · 293 points
HN discussionThat should not affect the rating by much with an appropriate sorting. See: https://www.evanmiller.org/how-not-to-sort-by-average-rating...
Comment linking /how-not-to-sort-by-average-rating.html
on “Ask HN: Recommend me a course on Coursera”Submitted The Mathematical Hacker as a story · 293 points
HN discussionAmen to that. It's a solved problem in statistics, too: https://www.evanmiller.org/how-not-to-sort-by-average-rating...
Comment linking /how-not-to-sort-by-average-rating.html
on “Amazon's Fake Review Problem”Submitted Why I Develop For The Mac as a story · 263 points
HN discussionIt turns out that Elixir/Phoenix templating is, in general, astoundingly fast, and that diffing operations are highly optimized at the language level due to the data structure (IO list) used. Here's some information: https://www.evanmiller.org/elixir-ram-and-the-template-of-do...
Comment linking /elixir-ram-and-the-template-of-doom.html
on “Build a real-time Twitter clone with LiveView and Phoenix 1.5”Submitted Statistical Formulas for Programmers (2013) as a story · 240 points
HN discussionMy current ranking stuff knowledge is coming from Evan Miller's blog, here's a post that deals with star ratings: https://www.evanmiller.org/ranking-items-with-star-ratings.h...
Comment linking /ranking-items-with-star-ratings.html
on “Show HN: TechYaks – Best of 50k tech talks ranked by confidence intervals”Submitted A Taste of Rust as a story · 235 points
HN discussionTrue-ish. Since the formula uses confidence intervals[0] it does better with recent talks than a simple (likes - dislikes). But you're right, breaking into "all time" status is tough because there are a number of talks with many hundreds of likes and 1 or 2 dislikes. 0: https://www.evanmiller.org/how-not-to-sort-by-average-rating...
Comment linking /how-not-to-sort-by-average-rating.html
on “Show HN: TechYaks – Best of 50k tech talks ranked by confidence intervals”Submitted How to Become a Software Contractor as a story · 219 points
HN discussion
