Top all-time · #4156
Robust Perception
Company blog · robustperception.ioprogrammingsystems
Robust Perception is the company blog at robustperception.io, writing about programming and systems. Since 2015, 19 different Hacker News readers have linked it 46 times, most often “Optimising startup time of Prometheus 2.6.0 with pprof”. The last post was in 2020, but the archive is still widely read.
Too few sampled comments (11) to say how often HN recommends it.?
Why HN links it
HN readers link this Prometheus monitoring blog for posts on federation, scaling, cardinality, RAM usage, and SSL certificate expiry alerts. Commenters cite it as a practical reference for Prometheus setups.
Summarised by Claude from a sample of 11 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Most-linked posts?
- /evaluating-performance-and-correctness2 people · 3 mentions · first linked 2019-12-12
- /scaling-and-federating-prometheus/2 people · 3 mentions · first linked 2016-05-01
- Optimising startup time of Prometheus 2.6.0 with pprof2 people · 2 mentions · first linked 2019-01-07
What Hacker News says?
I run a consulting/support business around Prometheus. I've found that giving talks and weekly writing blog posts ( https://www.robustperception.io/blog/ ) are great for acquiring customers. Beware though it's often at least 6-9 months before any of it directly bears fruit.
Comment linking /blog/
on “The Remote Freelancer: A list of remote work alternatives to Upwork”Submitted There are 100,000 Seconds in a Day as a story · 5 points
HN discussionI second the Thanos link, but if you want to only use Prometheus, I would look into federation: https://www.robustperception.io/scaling-and-federating-prome... https://prometheus.io/docs/prometheus/latest/federation/ You can set up a new "cold storage" Prometheus with a longer retention that scrapes select metrics from your regular 30d Prometheus to store for longer periods of time.
Comment linking /scaling-and-federating-prometheus
on “Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD”Submitted New Features in Prometheus 1.0.0 as a story · 4 points
HN discussionYou need a place with all that information anyway, otherwise how do you alert on something being missing? If you can't easily do that with your existing infrastructure, you should fix that first. I've written about this at http://www.robustperception.io/you-look-good-have-you-lost-m...
Comment linking /you-look-good-have-you-lost-machines/
on “Riemann – A network monitoring system”Submitted Conway’s Life in Prometheus as a story · 4 points
HN discussionI don't think there's any clear standard. There's many confusions about push vs pull that make the discussions hard to follow, as they often make apples to oranges comparisons. For example the push you're talking about in your comment is events, whereas a fair comparison for Prometheus would be with pushing metrics to Graphite. https://www.robustperception.io/which-kind-of-push-events-or... covers this in more detai…
Comment linking /which-kind-of-push-events-or-metrics
on “A simple way to get more value from metrics”Submitted Optimising startup time of Prometheus 2.6.0 with pprof as a story · 3 points
HN discussionThere is a transition feature: https://www.robustperception.io/accessing-data-from-promethe... The problem with data migration is that the two versions of the system lay out the data quite differently, so converting from one to the other would take a lot of disk seeks. In the worst case you could be looking potentially at days to convert the data over, which isn't really an option for most systems that care about ol…
Comment linking /accessing-data-from-prometheus-1-x-in-prometheus-2-0/
on “Prometheus 2.0”Submitted How Black Friday affected the latencies of top online retailers as a story · 3 points
HN discussionThese might be a good read if you are considering it: - https://www.robustperception.io/evaluating-performance-and-c... - https://medium.com/@valyala/evaluating-performance-and-corre...
Comment linking /evaluating-performance-and-correctness
on “Systems Monitoring with Prometheus and Grafana”Submitted Do you have basic infrastructure? as a story · 3 points
HN discussionUsually you start by some functional sharding (giving each team of services their own Prometheus servers), but also having per-datacenter Prometheus servers and then some hierarchical federation layer ontop of that. There's no built-in automatic horizontal scaling though (which would be going against the design goal of not having a clustered system, for reliability). Some resources: - Scaling in general: http://www.…
Comment linking /scaling-and-federating-prometheus/
on “Prometheus reaches 1.0”Submitted Why Prometheus does not have server side security features as a story · 2 points
HN discussionhttp://www.robustperception.io/scaling-and-federating-promet... explains it. Unless you're absolutely massive, it's fairly easy.
Comment linking /scaling-and-federating-prometheus/
on “Prometheus reaches 1.0”Submitted Cardinality Is Key as a story · 2 points
HN discussionThere's noone presently offering that, it's designed more with a view to being on-prem for reliability ( http://www.robustperception.io/monitoring-without-consensus/ ).
Comment linking /monitoring-without-consensus/
on “Prometheus reaches 1.0”Submitted How many metrics should an application return? as a story · 2 points
HN discussion> they skipped all the hard stuff (single node only, no HA We skipped the hard stuff on purpose, as hard stuff is extremely tricky to get right and liable to fail right when it's most needed. See http://www.robustperception.io/monitoring-without-consensus/ Per the above, there is HA.
Comment linking /monitoring-without-consensus/
on “Ask HN: Best monitoring system?”Submitted Measuring the performance impact of Meltdown/Spectre with Promtheus as a story · 2 points
HN discussionKibana is more logging, but Promethues covers the StatsD and Graphite requirements out of the box. Graphfana is one visualisation option to go with that ( http://www.robustperception.io/setting-up-grafana-for-promet... ).
Comment linking /setting-up-grafana-for-prometheus/
on “Time series database Graphite seems to be falling into disfavor”Submitted Translating between monitoring languages as a story · 2 points
HN discussionGreat to see an analysis of the various options. A few corrections on Prometheus: > an AlertManager which handles generating and transmitting alerts Alerts are generated in Prometheus, the Alertmanager then groups, deduplicates and transmits them. See https://www.robustperception.io/prometheus-and-alertmanager-... > Data Storage Model: Prometheus uses LevelDB as the storage backend LevelDB is only used for metadata…
Comment linking /prometheus-and-alertmanager-architecture/
on “A Comparison of Time Series Databases and Netsil’s Use of Druid”Submitted It’s overloaded? Try harder as a story · 2 points
HN discussionI do nightly binaries for Prometheus in all the possible archs, while things should compile perfectly there can be small things that prevent things from cross-compiling. darwin/arm, darwin/arm64, plan9/386, plan9/amd64 and solaris/amd64 don't compile for any of the 11 binaries (e.g. plan9 doesn't work due to logrus not having support for it). This leaves me with the list at http://www.robustperception.io/prometheus-…
Comment linking /prometheus-nightly-binaries/
on “Golang on ARM”Submitted Do you know what software you’re running? as a story · 2 points
HN discussionThis is basically the approach we take with Prometheus, with the option to add in additional stats like duration and processed records too. http://www.robustperception.io/monitoring-batch-jobs-in-pyth... is the full Python version, and the simple version is a bash one-liner too.
Comment linking /monitoring-batch-jobs-in-python/
on “Cron in production is a double-edged sword”Submitted Running into burning buildings because the fire alarm stopped (2017) as a story · 1 points
HN discussionThere's two issues with that, first the statsd event approach has scaling problems as relative to an instrumentation event a UDP packet is actually quite heavyweight ( https://www.robustperception.io/which-kind-of-push-events-or... ) and secondly how do you know that all devices that are meant to be sending are sending ( https://www.robustperception.io/push-needs-service-discovery... ).
Comment linking /push-needs-service-discovery/
on “InfluxDB now supports Prometheus remote reads and writes natively”Submitted How many metrics should an application return? as a story · 1 points
HN discussion
