Top all-time · #850
Leonard Richardson
Personal blog · crummy.comprogrammingwriting
Leonard Richardson writes News You Can Bruise, a personal blog at crummy.com about programming and writing. Since 2007, 79 different Hacker News readers have linked it 90 times, most often “Beautiful Soup”. It is still writing: the last post was on 2026-10-25, and you can follow it by feed.
Most-linked posts?
- Beautiful Soup19 people · 20 mentions · first linked 2008-01-08
- Beautiful Soup6 people · 6 mentions · first linked 2017-03-07
- What If Linus Torvalds Gets Hit by a Bus? (2000)5 people · 6 mentions · first linked 2017-11-17
- /software/BeautifulSoup/bs4/doc/5 people · 5 mentions · first linked 2017-05-05
- Secretly Public Domain: Most books published in the US before 19644 people · 4 mentions · first linked 2019-08-01
- Did You Know: BeautifulSoup's bits are rotting4 people · 4 mentions · first linked 2009-03-12
- Tool Safety: Beautiful Soup and Software Ethics3 people · 3 mentions · first linked 2017-10-20
- Bots Should Punch Up3 people · 3 mentions · first linked 2018-01-08
- /software/RubyfulSoup/3 people · 3 mentions · first linked 2008-09-15
- /software/BeautifulSoup/bs4/doc/2 people · 2 mentions · first linked 2017-11-14
What Hacker News says?
I use a modified Scrape ‘N Feed to scrape the sites I need without RSS. With this I can go another level deeper and scrape the entire article and return that back to the feed instead of just getting the snippet. https://www.crummy.com/software/ScrapeNFeed/
Comment linking /software/ScrapeNFeed/
on “RSS Is Better Than Twitter”Submitted Secretly Public Domain: Most books published in the US before 1964 as a story · 286 points
HN discussionThat really depends on your project and tech stack. If you're into Python and are going to deal with relatively static HTML, then the Python modules Scrapy [1], BeautifulSoup [2] and the whole Python data crunching ecosystem are at your disposal. There's lots of great posts about getting such a stack off the ground and using it in the wild [3]. It can get you pretty darn far, the architecture is solid and there are…
Comment linking /software/BeautifulSoup/bs4/doc/
on “Ask HN: What are best tools for web scraping?”Submitted Beautiful Soup as a story · 226 points
HN discussionYou can use the BeautifulSoup API with the `lxml` parser: https://www.crummy.com/software/BeautifulSoup/bs4/doc/#insta... I've heard that `lxml` can choke on certain badly-formed markup, but it's very fast. Personally has never failed on me.
Comment linking /software/BeautifulSoup/bs4/doc/
on “Ask HN: What are best tools for web scraping?”Submitted What If Linus Torvalds Gets Hit by a Bus? (2000) as a story · 171 points
HN discussionHey, I commented on this a bit further down in the topic: http://www.crummy.com/2012/1/22/0
Comment linking /2012/1/22/0
on “Python Libraries you should know about”Submitted Did You Know: BeautifulSoup's bits are rotting as a story · 72 points
HN discussionThe comparison should be moot, because Beautiful Soup 4 uses the lxml parser when it's available: http://www.crummy.com/software/BeautifulSoup/bs4/doc/#specif...
Comment linking /software/BeautifulSoup/bs4/doc/
on “Python Libraries you should know about”Submitted Richardson Maturity Model (2008) as a story · 19 points
HN discussionHey before BS 4 got released I looked forward to the very same thing you're talking about, using BS on top of lxml. But it seems the performance hasn't really improved: http://www.crummy.com/2012/1/22/0
Comment linking /2012/1/22/0
on “Python Libraries you should know about”Submitted Tool Safety: Beautiful Soup and Software Ethics as a story · 15 points
HN discussionhttps://www.crummy.com/software/BeautifulSoup/bs4/doc/ I mean... soup.find(id="link3") # <a class="sister" href=" http://example.com/tillie" id="link3">Tillie</a> Is this hard?
Comment linking /software/BeautifulSoup/bs4/doc/
on “Show HN: Spider Pro – easy and cheap way to scrape the internet”Submitted The Year 2000 (1967) as a story · 13 points
HN discussionThe big problem with the content is suggesting to use an XML parser to parse HTML. That might often work, and several XML parsers have some sort of HTML "mode", but in general, no, not really. HTML documents are frequently invalid XML. HTML documents lack an XML declaration, they don't close all tags, etc. They might even lack a root element. Someone mentioned this in a comment on SO, but it was (as far as I could s…
Comment linking /software/BeautifulSoup/
on “Is there a regular expression to detect a valid regular expression?”Submitted Beautiful Soup as a story · 5 points
HN discussionUnder US law, between 1925 and 1964, it was necessary to renew copyright every 28 years.[1] There are many works which have fallen out of copyright for nonrenewal, and there are projects which are now going through copyright renewal records to determine which of these are in fact now public domain in the US. Roughly 3/4 of potentially copyrighted works (published since 1924 and initially registered) have proved to b…
Comment linking /2019/07/22/0
on “DIY Book Scanner”Submitted Programmers at Work 1986: Where are they now? as a story · 5 points
HN discussionBeautifulSoup is in fact still actively maintained. “The current release is Beautiful Soup 4.1.3 (August 20, 2012).” http://www.crummy.com/software/BeautifulSoup/ I hear it recommended the most among Pythonistas, and it's plenty clean and fast for my use. But if you're skeptical, I'd still look for a more up to date benchmark (or run your own) rather than rely on results from >4 years ago.
Comment linking /software/BeautifulSoup/
on “I Don’t Need No Stinking API: Web Scraping For Fun and Profit”Submitted What If Linus Torvalds Gets Hit by a Bus? (2000) as a story · 4 points
HN discussionI learned Ruby and Ruby on Rails much faster than I had any right to, when my new job in NYC required it. I practiced not by writing Ruby on the job, but writing Ruby to scrape Craigslist apartment listings and feeding them into a spreadsheet. I started teaching myself computer programming in 2011/2012: I bought a bunch of Head First books published by O'Reilly, watched the entry-level CS video lectures that Stanfor…
Comment linking /software/BeautifulSoup/
on “Finding Free Food with Python”Submitted Beautiful Soup: We called him Tortoise because he taught us. HTML/XML parser in Python as a story · 4 points
HN discussionWhy not both? http://www.crummy.com/software/BeautifulSoup/bs4/doc/#specif... waiter there's some lxml in my soup!
Comment linking /software/BeautifulSoup/bs4/doc/
on “Web Scraping 101 with Python”Submitted What If Linus Torvalds Gets Hit by a Bus? as a story · 3 points
HN discussionOn the off chance you were not aware, bs4 also supports[0] getting parse events from html5lib[1] which (as its name implies) is far more likely to parse the text the same way the browser would 0: https://www.crummy.com/software/BeautifulSoup/bs4/doc/index.... 1: https://pypi.org/project/html5lib/
Comment linking /software/BeautifulSoup/bs4/doc/index.html
on “Beautiful Soup”Submitted Secretly Public Domain as a story · 3 points
HN discussionThe top python link, which should direct to http://www.crummy.com/software/BeautifulSoup/ instead directs to http://www.crummy.com/software/beautifulsoup/ --which causes crummy.com to throw a 404 error at you.
Comment linking /software/beautifulsoup/
on “The most popular links posted by developers to Stack Overflow”Submitted Bots Should Punch Up as a story · 2 points
HN discussionhttps://www.crummy.com/software/BeautifulSoup/ https://github.com/encode/httpx
Comment linking /software/BeautifulSoup/
on “Web Scraping with JavaScript”Submitted Bots Should Punch Up as a story · 2 points
HN discussion
