Top all-time · #3046
Meltwater Engineering Blog
Company blog · underthehood.meltwater.com/blogprogrammingdata
Meltwater Engineering Blog is the company blog at underthehood.meltwater.com, writing about programming and data. Since 2015, 25 different Hacker News readers have linked it 37 times, most often “Running a 400+ Node Elasticsearch Cluster”. It is still writing: the last post was on 2026-10-13, and you can follow it by feed.
Too few sampled comments (5) to say how often HN recommends it.?
Why HN links it
Engineering posts from Meltwater on running large Elasticsearch clusters (shard placement, upgrades, memory spikes), nginx API gateway testing, and mob programming. Linked in threads on those topics.
Summarised by Claude from a sample of 5 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Insights, analysis, and product updates from Meltwater engineers and data teams.
Most-linked posts?
- Running a 400+ Node Elasticsearch Cluster3 people · 3 mentions · first linked 2018-02-06
- Mob Programming – The Good, the Bad and the Great3 people · 3 mentions · first linked 2016-06-07
- Using Machine Learning to Load Balance Elasticsearch Queries3 people · 3 mentions · first linked 2018-09-28
- Red Hat memory optimisations and Riak performance2 people · 2 mentions · first linked 2015-04-14
- Simple effective way to test your #nginx API gateway2 people · 2 mentions · first linked 2017-12-12
- Optimal Shard Placement in a Petabyte Scale Elasticsearch Cluster2 people · 2 mentions · first linked 2018-11-06
- How we optimized wildcards when upgrading our 3PB Elasticsearch cluster2 people · 2 mentions · first linked 2022-11-25
- Synthetic Monitoring: A Case Study of the Meltwater API2 people · 2 mentions · first linked 2018-05-07
What Hacker News says?
FWIW, I'm getting: uBlock Origin has prevented the following page from loading: http://underthehood.meltwater.com/blog/2016/06/01/mob-progra... Because of the following filter ||meltwater.com^
Comment linking /blog/2016/06/01/mob-programming/
on “Mob Programming – The Good, the Bad and the Great”Submitted Mob Programming – The Good, the Bad and the Great as a story · 312 points
HN discussionSince I see meltwater folks in this discussion, can you say a bit about the distribution of your queries? In the other blog post about routing [1], you mention that most are a few ms but others are minutes and more. What’s the rough break down? Are most users performing queries “scoped” to a fairly small subset? [1] https://underthehood.meltwater.com/blog/2018/09/28/using-mac...
Comment linking /blog/2018/09/28/using-machine-learning-to-load-balance-elasticsearch-queries/
on “Optimal Shard Placement in a Petabyte Scale Elasticsearch Cluster”Submitted We upgraded an old, 3PB large, Elasticsearch cluster without downtime as a story · 149 points
HN discussionHi. We are actually using an even older version, 1.7.6 for our cluster. This is off not an ideal set-up and we will upgrade as soon as we can. That is not an easy task in a cluster of this size and given our uptime requirements. We know that general search and indexing performance, shard allocation and general cluster state management has all improved a lot in later versions of elasticsearch but we still anticipate…
Comment linking /blog/2018/09/28/using-machine-learning-to-load-balance-elasticsearch-queries/
on “Optimal Shard Placement in a Petabyte Scale Elasticsearch Cluster”Submitted Optimal Shard Placement in a Petabyte Scale Elasticsearch Cluster as a story · 143 points
HN discussionShameless plug, but one of the other blog posts in the above series explain how we addressed high and unpredictable memory load in relation to wildcard searches. https://underthehood.meltwater.com/blog/2022/11/25/how-we-up... After those changes were made then we haven't had any major issues with heap spikes. But it depends on a lot of factors, how your data looks, how your queries look and what aggregations that ar…
Comment linking /blog/2022/11/25/how-we-upgraded-an-old-3pb-large-elasticsearch-cluster-without-downtime-part-3-search-performance-and-wildcards/
on “Things that improved between Elasticsearch 1 and 7”Submitted Dynamic Route53 records for AWS auto scaling groups with Terraform as a story · 87 points
HN discussionGenerally when running nginx as an api proxy it can be troublesome to get the routing correctly the first time, therefore testing is required. This repository showcases how to do nginx api-gateway route tests using RSpec. This repository also got a blog to explain it in detail: http://underthehood.meltwater.com/blog/2017/12/12/lightweigh...
Comment linking /blog/2017/12/12/lightweight-tests-for-your-nginx-api-gateway/
on “Nginx Api Gateway with testing for continuous delivery”Submitted Red Hat memory optimisations and Riak performance as a story · 41 points
HN discussionSubmitted Supercharging the Elasticsearch Percolator as a story · 40 points
HN discussionSubmitted Running a 400+ Node Elasticsearch Cluster as a story · 19 points
HN discussionSubmitted Locality-Sensitive Hashing in Elixir as a story · 18 points
HN discussionSubmitted Optimal Shard Placement in a Petabyte Scale Elasticsearch Cluster as a story · 11 points
HN discussionSubmitted Using Machine Learning to Load Balance Elasticsearch Queries as a story · 10 points
HN discussionSubmitted Getting the most out of your RabbitMQ cluster as a story · 10 points
HN discussionSubmitted Risk-Free Deployments with Immutable Web Apps as a story · 8 points
HN discussionSubmitted Immutable Web Apps and NPM – A recipe for stable deployments as a story · 7 points
HN discussionSubmitted Simple effective way to test your #nginx API gateway as a story · 6 points
HN discussionSubmitted How we optimized wildcards when upgrading our 3PB Elasticsearch cluster as a story · 4 points
HN discussionSubmitted Enriching 450M Docs Daily with a Boring Stream Processor as a story · 4 points
HN discussionSubmitted Making Drone Builds 10 Times Faster as a story · 3 points
HN discussionSubmitted Riak, Elasticsearch and Numad Walk into a Red Hat as a story · 3 points
HN discussionSubmitted Migrating a 1TB PostgreSQL from AWS to Azure with Pgcopydb as a story · 2 points
HN discussionSubmitted Things that improved between Elasticsearch 1 and 7 as a story · 2 points
HN discussionSubmitted We Love and Sponsor Benthos – Corporate Sponsorship of Open Source as a story · 2 points
HN discussionSubmitted Excel Friendly CSV Exports with Elixir as a story · 2 points
HN discussionSubmitted Why We Built a Dull Stream Processor as a story · 2 points
HN discussionSubmitted Running a 400+ Node Elasticsearch Cluster as a story · 2 points
HN discussionSubmitted Running a 400+ Node Elasticsearch Cluster as a story · 2 points
HN discussion