Top all-time · #1819
Richard Jones
Personal blog · metabrew.com/article · on HN as metabrewprogrammingsystems
Richard Jones's personal blog at metabrew.com covers programming and systems. Since 2008, 41 different Hacker News readers have linked it 53 times, most often “ Anti-RDBMS: A list of distributed key-value stores”. It is still writing: the last post was on 2024-12-09, and you can follow it by feed.
Sometimes recommended · 6 of 33 sampled comments recommend it?
Why HN links it
HN readers link Richard Jones's blog for its posts on handling a million concurrent connections with Erlang and for its comparisons of distributed and NoSQL databases. Some also cite its notes on how Last.fm uses IRC.
Summarised by Claude from a sample of 33 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Logbook / blog / devlog
Most-linked posts?
- Anti-RDBMS: A list of distributed key-value stores10 people · 10 mentions · first linked 2009-01-19
- A Million-user Comet Application with Mochiweb (2008)8 people · 9 mentions · first linked 2010-08-24
- /article/a-million-user-comet-application-with-mochiweb-part-37 people · 8 mentions · first linked 2010-01-21
- A Million-user Comet Application with Mochiweb, Part 34 people · 4 mentions · first linked 2008-11-04
- How we use IRC at Last.fm4 people · 4 mentions · first linked 2009-09-09
- /article/a-million-user-comet-application-with-mochiweb-part-12 people · 2 mentions · first linked 2021-03-11
- How we use IRC at Last.fm2 people · 2 mentions · first linked 2009-01-09
- A Million-user Comet Application with Mochiweb, Part 12 people · 2 mentions · first linked 2008-10-16
- Rewriting Playdar: C++ to Erlang, massive savings2 people · 2 mentions · first linked 2011-02-14
What Hacker News says?
Erlang is a terrific platform, but it isn't very accessible to the wider population. Timely, here's antirez on this topic when Redis was announced on HN ten years ago [0]: > There are a lot of projects similar to Redis in Erlang actually: http://www.metabrew.com/article/anti-rdbms-a-list-of-distrib... > Apart from speed what I like of C for this kind of projects is the self-contain-ness and the fact that most develo…
Comment linking /article/anti-rdbms-a-list-of-distributed-key-value-stores/
on “K3s – Lightweight Kubernetes”Submitted How we use IRC at Last.fm as a story · 109 points
HN discussionThe Linux tuning part seems to have been inspired by these blog posts from 14 years ago: https://www.metabrew.com/article/a-million-user-comet-applic... It's almost a little disappointing that beefy modern servers only manage a x5 scale improvement, though that could be due to the differences in runtime behaviour between Erlang and the JVM.
Comment linking /article/a-million-user-comet-application-with-mochiweb-part-1
on “Achieving 5M persistent connections with Project Loom virtual threads”Submitted Anti-RDBMS: A list of distributed key-value stores as a story · 75 points
HN discussionYep, you need multiple IP addresses and other tricks to create over 64K connections to a single backend: http://www.metabrew.com/article/a-million-user-comet-applica... One workaround that comes to mind is to have each backend listen on multiple ports.
Comment linking /article/a-million-user-comet-application-with-mochiweb-part-3
on “WebSocket proxy support added to nginx trunk”Submitted Rewriting Playdar: C++ to Erlang, with code statistics as a story · 42 points
HN discussionI rewrote a (~10k lines) C++ app to erlang back when I was learning erlang, and saw a ~75% reduction in lines of code. http://www.metabrew.com/article/rewriting-playdar-c-to-erlan... I expect the 'N' value varies wildly depending on what you are building, and whichever language you are comparing against.
Comment linking /article/rewriting-playdar-c-to-erlang-massive-savings
on “Joe Armstrong on Programmer Productivity”Submitted Live-upgrades to IRCCloud.com with Erlang/OTP as a story · 29 points
HN discussionYou can see this in the 1M connection test done here: http://www.metabrew.com/article/a-million-user-comet-applica... Look at the "Turning it up to 1 Million" section where he details the need to use 17 IPs for the client side.
Comment linking /article/a-million-user-comet-application-with-mochiweb-part-3/
on “Zed Shaw: "poll, epoll, science, and superpoll" with R”Submitted A Million-user Comet Application with Mochiweb, Part 1 as a story · 28 points
HN discussionIt's been done before. Here is an article using Erlang and Linux. http://www.metabrew.com/article/a-million-user-comet-applica... Part 3 is my favorite. http://www.metabrew.com/article/a-million-user-comet-applica...
Comment linking /article/a-million-user-comet-application-with-mochiweb-part-3
on “One Million Concurrent TCP connections”Submitted A Million-user Comet Application with Mochiweb, Part 3 as a story · 27 points
HN discussionLast.fm has a pretty nifty notification system that uses netcat and an IRC bot. http://www.metabrew.com/article/how-we-use-irc-at-lastfm/
Comment linking /article/how-we-use-irc-at-lastfm/
on “I never realized how useful netcat is”Submitted Rewriting Playdar: C++ to Erlang, massive savings as a story · 10 points
HN discussionI tested one server up to 1 million concurrent connections a couple of years ago, search for sysctl in this article to see the settings I used: http://www.metabrew.com/article/a-million-user-comet-applica...
Comment linking /article/a-million-user-comet-application-with-mochiweb-part-3
on “Linux Kernel Tuning for C500k”Submitted A Million-user Comet Application with Mochiweb, Part 2 as a story · 10 points
HN discussionWe use Freenode, #grooveshark is an open channel for random techie users who want to talk directly to developers (or each other), and people working on 3rd party extensions/add ons to Grooveshark. We have other internal invite-only channels for coordinating with external developers on internal projects. We also have github spamming a channel now whenever changes to a repo that a lot of people are coordinating on get…
Comment linking /article/how-we-use-irc-at-lastfm
on “Ask HN: Does anyone still use IRC?”Submitted Erlang releases + upgrades using rebar as a story · 4 points
HN discussionIt's called a C1M problem. From 2013: https://www.metabrew.com/article/a-million-user-comet-applic... . It's pretty easy. Been working on a product doing this for 500k connections per machine (MQTT) back in 2014. EC2, m3.xlarge instances back then.
Comment linking /article/a-million-user-comet-application-with-mochiweb-part-1
on “Building a high-scale chat server on Cloud Run”Submitted A Million-user Comet Application with Mochiweb, Part 1 as a story · 2 points
HN discussionSimilar article here, going to 11: http://www.metabrew.com/article/a-million-user-comet-applica...
Comment linking /article/a-million-user-comet-application-with-mochiweb-part-1
on “600K concurrent HTTP connections with Clojure and http-kit”Submitted A Million-user Comet Application with Mochiweb (2008) as a story · 2 points
HN discussionSee http://www.metabrew.com/article/a-million-user-comet-applica...
Comment linking /article/a-million-user-comet-application-with-mochiweb-part-1
on “App.net’s new direction”Submitted A Million-user Comet Application with Mochiweb as a story · 2 points
HN discussionWhatsApp had over 2 million users connected to their (Erlang) server last year. http://blog.whatsapp.com/index.php/2012/01/1-million-is-so-2... Here is the same in Erlang for reference (from a few years ago, I would be interested to see if there is a more efficient way now): http://www.metabrew.com/article/a-million-user-comet-applica...
Comment linking /article/a-million-user-comet-application-with-mochiweb-part-3
on “Ruby - Handling 1 Million Concurrent Connections”Submitted How we use IRC at Last.fm as a story · 2 points
HN discussionIf you're looking for a less functional Hubot that's been around for years, why not use Last.fm's irccat? http://www.metabrew.com/article/how-we-use-irc-at-lastfm It already does the whole "kick of a script with these params from irc" thing and can easily be piped to from any langage that can open a port.
Comment linking /article/how-we-use-irc-at-lastfm
on “Ted Dziuba's take on Hubot”https://www.metabrew.com/article/libketama-consistent-hashin... Ketama implementation of consistent hashing algorithm is really intuitive and battle tested.
Comment linking /article/libketama-consistent-hashing-algo-memcached-clients
on “Consistent hashing”I'm curious what kind of OS tuning was required for 500k concurrent connections. It's obvious that some ulimit tuning is necessary, but I'm wondering if any other hard limits beyond that were hit? Also, in case the original author is reading, you might find this interesting as well. It's an article on trying to reach 1MM concurrent connections with Erlang. http://www.metabrew.com/article/a-million-user-comet-applica…
Comment linking /article/a-million-user-comet-application-with-mochiweb-part-1
on “C500k in Action at Urban Airship”There may be a good point in here somewhere, but it's not clear what it is. Relational databases don't scale, so... try CouchDB! or Drizzle! ...which also don't scale. What? For a more complete list of RDBMS-killing science projects, try http://www.metabrew.com/article/anti-rdbms-a-list-of-distrib... (discussion: http://news.ycombinator.com/item?id=440687 )
Comment linking /article/anti-rdbms-a-list-of-distributed-key-value-stores/
on “Is the Relational Database Doomed?”
