Top all-time · #7202
Pierluigi Vinciguerra
Personal blog · substack.thewebscraping.clubprogrammingdata
Pierluigi Vinciguerra writes The Web Scraping Club, a personal blog at substack.thewebscraping.club about programming and data. Since 2022, 11 different Hacker News readers have linked it 19 times. It is still writing: the last post was on 2026-11-01, and you can follow it by feed.
Too few sampled comments (3) to say how often HN recommends it.?
Why HN links it
HN readers link this newsletter for web scraping and headless browser topics. The comments cite it in discussions of scraping tools and headless Chrome but do not assess the writing itself.
Summarised by Claude from a sample of 3 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
News, solutions and interviews about web scraping. In this substack you will find weekly content about: - Web Scraping techniques - Interviews with key people in the industry - Anti bot infos and counter measures - Real world examples and code. Click to read The Web Scraping Club, a Substack publica
What Hacker News says?
Thanks! The three of us worked together at our former company - ecomm saas start up where we spent a ton of $ on scraping infrastructure spinning up headless Chrome instances. It started out as more of an R&D thesis - is it possible to strip out graphical rendering from Chrome headless? Turns out no - so we tried to build it from scratch. And the beta results validated the thesis. I wrote a whole thing about it here…
Comment linking /p/rethinking-the-web-browser
on “Show HN: Lightpanda, an open-source headless browser in Zig”Submitted Scraping data from an app: real world example as a story · 117 points
HN discussionThree real paths I'd evaluate 0) Tools: Puppeteer and Playwright are the cleanest way I've found to get proper JS page rendering and behavior control. Node is well suited for this but some prefer python.. Since you sound concerned with blocks I'm guessing plain HTML scrapers like BeautifulSoup or Cheerio would be insufficient but they're more robust in terms of sheer volume and overhead of course. 1) if you have mon…
Comment linking /
on “Ask HN: Best web scraping toolset for 2024?”Submitted Scraping Cloudflare websites without buying any tool as a story · 27 points
HN discussionI'm not sure I've the best antibot solution, but it's handled through some crawler options exposed to the user. Out of the box, I use a (fast) http crawler with my app's user-agent. It is not at all resilient to antibot. I direct users who are encountering issues to first try a user-agent override, and if that doesn't work, to next enable javascript crawling (think headless chrome), which is slower and heavier, but…
Comment linking /
on “Linkok.com – modern broken link checker I've been working on for 3 years”Submitted Ten years of selling web data: a personal perspective as a story · 13 points
HN discussionIt’s a custom GPT for learning scraping, the newsletter is called The Web Scraping Club and the archive of the article can be found here: https://substack.thewebscraping.club/archive
Comment linking /archive
on “I'm a custom GPT focused on web scraping, AMA”Submitted Is web scraping a profitable industry? as a story · 4 points
HN discussionSubmitted Scrapyd – managing a fleet of scrapers via web interface as a story · 4 points
HN discussionSubmitted Scraping Hotel Data for Investors as a story · 3 points
HN discussionSubmitted Web Scraping from 0 to hero: kickstart your career in web scraping as a story · 3 points
HN discussionSubmitted The DMCA Was Built to Stop DVD Piracy. Google Wants to Use It Against Scrapers as a story · 2 points
HN discussionSubmitted How scraping the web became an expensive business as a story · 2 points
HN discussionSubmitted Scraping the inflation: how web scraping can be used to calculate CPI as a story · 2 points
HN discussionSubmitted Scraping a single website once costed me thousands USD in proxies as a story · 2 points
HN discussionSubmitted How to Leverage IPv6 Subnets for Infinite Proxy Rotation as a story · 1 points
HN discussionSubmitted The AI-Powered tools for scraping landscape. Please, help me fill the map as a story · 1 points
HN discussionSubmitted Scrape like a pro but not like an AI company as a story · 1 points
HN discussionSubmitted The Python Starter Toolkit for Web Scraping (2022) as a story · 1 points
HN discussion