Top all-time · #5035
Dustin Ingram
Personal blog · dustingram.com/articlesprogrammingsystems
Dustin Ingram's personal blog at dustingram.com covers programming and systems. Since 2016, 16 different Hacker News readers have linked it 21 times, most often “Powering the Python Package Index in 2021”. The last post was in 2021, but the archive is still widely read.
Too few sampled comments (11) to say how often HN recommends it.?
Why HN links it
HN commenters cite Dustin Ingram's posts on PyPI infrastructure, especially a 2021 post on how PyPI is served via Fastly, including its roughly 27PB/month traffic and $1.8M/month list price. The comments use them as sources.
Summarised by Claude from a sample of 11 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Most-linked posts?
- Powering the Python Package Index in 202111 people · 15 mentions · first linked 2021-05-14
What Hacker News says?
The core issue is that setup.py can be non-deterministic, so them doing it server side may not give the right results. https://dustingram.com/articles/2018/03/05/why-pypi-doesnt-k...
Comment linking /articles/2018/03/05/why-pypi-doesnt-know-dependencies/
on “Guido van Rossum joins Microsoft”- di · 2016-06-01
Submitted X for Humans: Find Software for Your Species as a story · 8 points
HN discussion There are 2 members in the disposable-email-domains organization on GitHub.[1] You (martenson) are one of them. The other member (di), who is a "core maintainer" of the project according to the README changelog,[2] is also a member of the Google organization on GitHub.[3] di describes himself as part of the "@google open source security team"[3] and the website linked from his profile says that he is "a Developer Ad…
Comment linking /
on “Mozilla's Firefox Relay to be added to disposable-email-domains blacklist”Submitted What does it take to power the Python Package Index? as a story · 3 points
HN discussionTaking PyPI as a central place of packages, it is known that their bandwidth bill would be $1.8+M per month ( https://dustingram.com/articles/2021/04/14/powering-the-pyth... ) were it not for Fastly giving them a 100% discount. Are there any reliable decentralized package distribution systems operating at within 2 orders of magnitude of that scale? How do they handle administrative issues such as malicious packages…
Comment linking /articles/2021/04/14/powering-the-python-package-index-in-2021/
on “I'm leaving Ruby Central”Submitted Powering the Python Package Index in 2021 as a story · 3 points
HN discussionSome think it is. Excerpt from this talk by Dustin Ingram: https://dustingram.com/talks/2018/10/23/inside-the-cheesesho... (halfway through) PyPI is sometimes called the Cheeseshop. This is a reference to this Monty Python skit, where a man goes into a cheeseshop which has no cheese for sale. The joke is that when PyPI was first created, there was nothing in it. Hence, "The Cheeseshop". But it does have stuff in it…
Comment linking /talks/2018/10/23/inside-the-cheeseshop/
on “PyPy has been working for me for several years now”- di · 2021-06-01
Submitted What does it take to power the Python Package Index? as a story · 2 points
HN discussion The official PyPI packkages are hosted on Fastly. I believe they donate the infrastructure costs through the Python Software Foundation, as the bandwidth is over 300TB a day, and would cost about 1.8 million dollars a month. Source: https://dustingram.com/articles/2021/04/14/powering-the-pyth...
Comment linking /articles/2021/04/14/powering-the-python-package-index-in-2021/
on “Show HN: Exatorrent – Self-hostable Torrent client written in Go”Submitted Powering the Python Package Index in 2021 as a story · 2 points
HN discussion> is cheap It's not even cheap, it's just possible to get companies to donate the resources to sustain it: https://dustingram.com/articles/2021/04/14/powering-the-pyth... What it is is feasible , and IMO the alternative you're suggesting is infeasible under our current model of global economics without some kind of massive government funding. > you're strongly incentivized to root your users simply because if you do…
Comment linking /articles/2021/04/14/powering-the-python-package-index-in-2021/
on “PyPI Prohibits inbox.ru email domain registrations”Submitted Powering the Python Package Index in 2021 – Dustin Ingram as a story · 2 points
HN discussion- di · 2021-05-21
Closer to 27PB/month: https://dustingram.com/articles/2021/04/14/powering-the-pyth...
Comment linking /articles/2021/04/14/powering-the-python-package-index-in-2021/
on “What to do about GPU packages on PyPI?” Submitted PyPI as a Service as a story · 2 points
HN discussion- di · 2021-05-21
About 27PB/month right now: https://dustingram.com/articles/2021/04/14/powering-the-pyth...
Comment linking /articles/2021/04/14/powering-the-python-package-index-in-2021/
on “What to do about GPU packages on PyPI?” Submitted Powering the Python Package Index in 2021 as a story · 1 points
HN discussionI think PyPI is mostly served via Fastly. https://dustingram.com/articles/2021/04/14/powering-the-pyth...
Comment linking /articles/2021/04/14/powering-the-python-package-index-in-2021/
on “PyPI in a Box (2020)”Submitted Powering the Python Package Index in 2021 as a story · 1 points
HN discussion>Running our test suite 2-3 times over the span of a couple hours would trigger the rate limit… Not to single you out, but not too hard to imagine why companies start enabling rate limiting. I oftentimes wonder the obscene amount of bandwidth being consumed by this developer automation. This page[0] indicates that PyPi serves 900 terabytes daily. Seems obvious we need more turn-key solutions that enable caching for…
Comment linking /articles/2021/04/14/powering-the-python-package-index-in-2021/
on “Pull Through Cache Repositories for Amazon Elastic Container Registry”Submitted PyPI is dead. Long live PyPI! as a story · 1 points
HN discussionFor its part, PyPI is apparently funded by the Python Software Foundation and run by volunteer admins, with the bulk of its data served gratis by Fastly. This is based on a blog post about its infrastructure from 2021 [1], which also says that the list price for the throughput they're doing is $1.8 million per month. [1] https://dustingram.com/articles/2021/04/14/powering-the-pyth...
Comment linking /articles/2021/04/14/powering-the-python-package-index-in-2021/
on “Hugging Face raises $235M from investors including Salesforce and Nvidia”
