Top all-time · #2367
intoli.com
Company blog · intoli.com/blog · on HN as foobprogrammingwebai
intoli.com is the company blog at intoli.com, writing about programming, web and ai. Since 2017, 30 different Hacker News readers have linked it 101 times, most often “Terminal Recorders: A Comprehensive Guide”. The last post was in 2018, but the archive is still widely read.
Too few sampled comments (16) to say how often HN recommends it.?
Why HN links it
HN readers link Intoli's posts on making headless Chrome and Firefox undetectable to bot checks, running FFmpeg on AWS Lambda, and comparing terminal recorders. Commenters cite them as technical references.
Summarised by Claude from a sample of 16 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Machine learning consulting services.
Most-linked posts?
- Terminal Recorders: A Comprehensive Guide5 people · 7 mentions · first linked 2017-11-14
- Dangerous Pickles – Malicious Data Serialization in Python5 people · 7 mentions · first linked 2017-10-17
- Making Chrome Headless Undetectable5 people · 5 mentions · first linked 2017-08-09
- How F5Bot Slurps All of Reddit4 people · 5 mentions · first linked 2018-07-30
- Running FFmpeg on AWS Lambda for 1.9% the Cost of AWS Elastic Transcoder4 people · 4 mentions · first linked 2018-05-02
- Analyzing One Million Robots.txt Files3 people · 5 mentions · first linked 2017-09-19
- User-Agents – Generating random user agents using Google Analytics and CircleCI3 people · 3 mentions · first linked 2018-08-30
- Running Selenium with the New Headless Chrome2 people · 4 mentions · first linked 2017-04-14
- Predicting Hacker News article success with neural networks and TensorFlow2 people · 3 mentions · first linked 2017-05-23
- Finding Pareto Optimal Blogs on Hacker News2 people · 3 mentions · first linked 2017-06-13
What Hacker News says?
A bot absolutely can, you just need to use a more sophisticated bot. This article [1] is from August 2017, so the arms race has escalated and techniques improve, but the gist is the same: You just do a better job of simulating the "human" characteristics they monitor. Gen 4 bots (bots that run on an infected user's machine) can circumvent these measures as well. [1] https://intoli.com/blog/making-chrome-headless-und…
Comment linking /blog/making-chrome-headless-undetectable/
on “Why can’t a bot tick the 'I'm not a robot' box?”Submitted It is not possible to detect and block Chrome headless as a story · 360 points
HN discussionThis is pretty awesome. I'm a big fan of both SVG animations and terminal recorders, so this is right up my alley. I previously built a small library called svganimator for creating smooth animations between static SVG images [1], used generated SVG animations to help design the Wayback Machine loading animation [2], and also put together a fairly comprehensive comparison of terminal recording solutions [3]. I'll ne…
Comment linking /blog/terminal-recorders/
on “Show HN: Termtosvg – Record terminal sessions as SVG animations”Submitted How F5Bot Slurps All of Reddit as a story · 255 points
HN discussionThat file includes at least two non-standard syntax extensions[0]. Robots is just a de facto standard and respect of some directives varies[1]. So much for it being 'not difficult' while the task is not even clear because there isn't even a clear standard. Archive.org also dislikes how robots.txt is being used mainly for search engines and goes against their mission in particular[2]. Are they now hackers for not thr…
Comment linking /blog/analyzing-one-million-robots-txt-files/
on “Accessing Publicly Available Information on the Internet Is Not a Crime”Submitted Running FFmpeg on AWS Lambda for 1.9% the Cost of AWS Elastic Transcoder as a story · 214 points
HN discussionI recently wrote a guide to using headless Chrome with Selenium and Chrome WebDriver [1]. I thought that some people in this thread might find it useful considering that the submitted article doesn't mention Selenium at all. [1] - https://intoli.com/blog/running-selenium-with-headless-chrom...
Comment linking /blog/running-selenium-with-headless-chrome/
on “Getting Started with Headless Chrome”Submitted Analyzing One Million Robots.txt Files as a story · 157 points
HN discussionAwesome, I really love meta-analyses too! Our product involves web scraping so it's a natural fit to write articles about scraping and analyzing data in general. A few of our articles so far [1,2,3] would probably be classified as meta-analyses and we have some more in the pipeline! Feel free to sign up for the newsletter in the footer of http://intoli.com if you would like to read those articles after they're publi…
Comment linking /
on “Ask HN: Which companies have the best blogs written by their engineering team?”Submitted How are PCA and SVD related? as a story · 139 points
HN discussionWhat are peoples' thoughts on the marketing effectiveness of quality relative to quantity in blogging? We have recently started publishing what we consider to be very high quality material (the majority of them have front-paged on HN if that's any indication). Most of these have been published on [1] so far but we're in the process of transitioning to a more official company blog [2]. A close friend who really knows…
Comment linking /blog/
on “Ask HN: Which companies have the best blogs written by their engineering team?”Submitted Markov's and Chebyshev's Inequalities Explained as a story · 104 points
HN discussionAh, thanks, now I get it: it doesn't bother them that fingerprinting lets them be tracked; it bothers them that fingerprinting (and VPN IPs) mean their scrapers get hit with captchas. FWIW, it takes very little effort to completely conceal the fact that firefox is running headless under marionette/webdriver/geckodriver. Chrom(ium) takes more effort, but these guys have solved it (and built a business around it): htt…
Comment linking /blog/making-chrome-headless-undetectable/
on “Are top websites using WebGL for fingerprinting?”Submitted Predicting Hacker News article success with neural networks and TensorFlow as a story · 87 points
HN discussionLet me be the devil advocate: why not use ttyrec + seq2gif instead? Or check https://intoli.com/blog/terminal-recorders/ if you don't like my recommendation ttyrec is a standard format that most people and tools understand. And if you want something fancy that asciinema output, you can use seq2gif which support all the bells n whistles asciinema does, while allowing you to also upload the ttyrec file. However, I wou…
Comment linking /blog/terminal-recorders/
on “Asciinema – Record and share terminal sessions”Submitted Recreating Python's Slice Syntax in JavaScript Using ES6 Proxies as a story · 77 points
HN discussionI just finished writing an article that explains how to connect WebDriver to Firefox running in the new headless mode on Windows if anybody's interested: https://intoli.com/blog/running-selenium-with-headless-firef... It should be pretty similar on Linux, and probably macOS when it comes around.
Comment linking /blog/running-selenium-with-headless-firefox/
on “Firefox 56 supports headless mode on Windows”Submitted Building a YouTube MP3 Downloader with Exodus, FFmpeg, and AWS Lambda as a story · 76 points
HN discussionI'd guess if they're crawling purely for the purpose of detecting cloaking (which would be a much smaller-scale job than the standard Googlebot indexing), they'd just use Chrome Headless[0][1] [0] https://intoli.com/blog/not-possible-to-block-chrome-headles... [1] http://antoinevastel.github.io/bot%20detection/2018/01/17/de...
Comment linking /blog/not-possible-to-block-chrome-headless/
on “Googlebot’s JavaScript random() function is deterministic”Submitted Performing Efficient Broad Crawls with the Aopic Algorithm (2018) as a story · 54 points
HN discussionThe timing on this is quite coincidental, I just published a short article this morning about adding support to Selenium for installing WebExtensions to Firefox profiles [1]. I write and use extensions frequently for web scraping and browser automation, and I've really been thrilled about what Mozilla has done with the WebExtensions API. It was a quite miserable process to write cross-browser extensions in the past,…
Comment linking /blog/firefox-extensions-with-selenium/
on “Extensions in Firefox 58”Submitted Designing the Wayback Machine Loading Animation as a story · 42 points
HN discussionVery cool. I had the same idea a couple of years back and implemented a very similar interactive tool [1]. If you find the topic interesting, then you might also enjoy the analysis explained in that blog post. - [1] - https://intoli.com/blog/hacker-news-title-tool/
Comment linking /blog/hacker-news-title-tool/
on “Show HN: Testing HN titles against a neural network”Submitted Using Firefox WebExtensions with Selenium as a story · 36 points
HN discussionMy co-founder wrote a bit about Asciinema as part of a comprehensive comparison of terminal recorders [1]. There's a lot to love about it (wide availability, very nice JavaScript playback -- check out the Game of Life playback at [1]), but it's a bit of a chore to self host, if that's important for you. [1]: https://intoli.com/blog/terminal-recorders/
Comment linking /blog/terminal-recorders/
on “Asciinema – Record and share your terminal sessions”Submitted Dangerous Pickles – Malicious Data Serialization in Python as a story · 31 points
HN discussionVery cool, thanks for sharing! I did a somewhat similar analysis a while back [1], and I found that many of the top domains either had a YC affiliation or corresponded to extremely well-known companies or organizations. This made me interested in finding lesser known blogs that also produce high quality content. I tried to identify these by putting a limit on the number of unique users who had submitted content from…
Comment linking /blog/pareto-optimal-blogs/
on “Show HN: HN Domain Leaderboard”Submitted JavaScript Injection with Selenium, Puppeteer, and Marionette as a story · 23 points
HN discussionHere's an article which explains in detail how pickles/serialization work in Python, and how pickles can be constructed to evaluate malicious code when they are deserialized. That's specifically about Python, but the same issue exists in many other languages. [1] - https://intoli.com/blog/dangerous-pickles/
Comment linking /blog/dangerous-pickles/
on “Oracle plans to dump risky Java serialization”Submitted Terminal Recorders: A Comprehensive Guide as a story · 21 points
HN discussion
