Top all-time · #12117
Data Big Bang Blog
Company blog · blog.databigbang.com · on HN as wslhdataprogramming
Data Big Bang Blog is the company blog at blog.databigbang.com, writing about data and programming. Since 2011, 6 different Hacker News readers have linked it 135 times, most often “Google Search NoAPI”. The last post was in 2013, but the archive is still widely read.
Too few sampled comments (2) to say how often HN recommends it.?
Why HN links it
HN readers link this blog for web scraping tutorials, including a post on scraping AJAX and JavaScript pages. One commenter suggests it to someone asking how to scrape web pages.
Summarised by Claude from a sample of 2 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Most-linked posts?
- Google Search NoAPI2 people · 7 mentions · first linked 2011-03-13
- Extraction of Main Text Content Using the Google Reader NoAPI2 people · 4 mentions · first linked 2011-11-23
- /web-scraping-ajax-and-javascript-sites/2 people · 3 mentions · first linked 2012-02-28
- Integrating Google Analytics into your company loop with a Excel add-on2 people · 2 mentions · first linked 2012-02-02
What Hacker News says?
Google Reader is irreplaceable, it is not only about reading RSS. That is the easy part. It is about going back in time and accessing all past feeds in an organized way (it is difficult to rebuilt that from crawling and web scraping). If you add a blog now you can read articles that are not present in the current feed. Google Reader is the core of my information diet. Not twitter. Thousands of blogs starred, liked,…
Comment linking /the-data-portability-fact-sheet/
on “Google Reader shutting down”Submitted Scraping web sites which dynamically load data (like Twitter) as a story · 85 points
HN discussionOMeta is/was a pleasure to work with. I used it for parsing s-expressions [1]. I also chose it because there were (experimental) C# implementations. Now ANTLR 4 also supports C# but OMeta has less friction for learners. [1] http://blog.databigbang.com/parsing-s-expressions-in-c-using...
Comment linking /parsing-s-expressions-in-c-using-ometa/
on “Flex”Submitted Google Search NoAPI as a story · 10 points
HN discussionIn 2013 I elaborated about this topic: http://blog.databigbang.com/letters-from-the-future-challeng... I would add that in 2021 we can easily do Natural Language Understading (NLU) and Natural Language Generation (NLG) and can build zillions of web pages that don't follow the original page ranking concept of Google. Probably important sites share less low rank pages and there are many more link rings and clusters. M…
Comment linking /letters-from-the-future-challenging-googles-search-engine/
on “Search engines and SEO spam”Submitted Big data and data science blogs ordered by Google PageRank as a story · 6 points
HN discussionGoogle supports AND, OR, NOT and NEAR. There is an AROUND operator that is similar to NEAR. I think the way DDG can gain more traction is motivating top sites to enhance their local search pagerank. It's incredible how Google search restricted to Stack Overflow works better than the Stack Overflow local search engine. I wrote a rant about this at Challenging Google’s Search Engine [1]. [1] http://blog.databigbang.co…
Comment linking /letters-from-the-future-challenging-googles-search-engine/
on “Inside DuckDuckGo, Google's Tiniest, Fiercest Competitor”Submitted Running Your Own Anonymous Rotating Proxies as a story · 4 points
HN discussionIn my spare time I am doing something like it. It's called Egont (No Ego OR Ego New Technology). If you liked this article, you might like: i) Egont, a web orchestration language: http://blog.databigbang.com/ideas-egont-a-web-orchestration-... ii) Egont part II: http://blog.databigbang.com/egont-part-ii/ iii) Parsing s-expressions in C# using OMeta: http://blog.databigbang.com/parsing-s-expressions-in-c-using... Bec…
Comment linking /parsing-s-expressions-in-c-using-ometa/
on “A Personal API”Submitted Free distributed crawling and scraping with multiple Tor circuits as a story · 4 points
HN discussionI've not seen these ones on any list here: http://blog.databigbang.com http://blog.nektra.com http://www.yosefk.com/blog http://www.cocos2d-iphone.org/ http://blogs.southworks.net http://awelonblue.wordpress.com/
Comment linking /
on “Ask HN: What unknown technical blogs or sites do you read?”Submitted Google Search NoAPI as a story · 3 points
HN discussionI don't think this is about a niche but rather a broader need for lists, more specifically, integrated lists from users. Pinterest, for example, is built around lists, Amazon has wishlists, IMDb[1] provides movie rankings, etc. From a data portability perspective, the real challenge is getting users to connect all these platforms to a kind of unified list, making it accessible for queries, machine learning, or other…
Comment linking /egont-part-ii/
on “Most-Watched Software Engineering Talks of 2024”Submitted Web Scraping Ajax and JavaScript Sites (2011) as a story · 2 points
HN discussionThey are not the same. With Google Reader you had a virtually infinity history of the feed while with feed clients you only receive articles from the current and past feed files seen. An article about this topic: http://blog.databigbang.com/tag/google-reader/
Comment linking /tag/google-reader/
on “O Reader My Reader – Remembering Google Reader”Submitted A slight modification of the Knuth-Morris-Pratt algorithm for streams as a story · 2 points
HN discussionMy 4 cents: - I like the idea of a multiparadigm programming language (many exists) but where you can write part of the code in a different language, not trying to embed everything in the same syntax. I think in this way you can write code and express your ideas differently. - A [social] programming language where some variables and workflows are shared between users [1][2]. - A superreflective programming language…
Comment linking /egont-part-ii/
on “Some programming language ideas”Submitted The Data Portability Fact Sheet (2011) as a story · 2 points
HN discussionA social data flow engine called Egont. You can take a look at these articles: - Egont, A Web Orchestration Language: http://blog.databigbang.com/ideas-egont-a-web-orchestration-... - Egont Part II: http://blog.databigbang.com/egont-part-ii/ You can define things like this using s-expressions: (let mytwitter (twitter "databigbang") Then you can do (twitterdb store (twitter.tweets)) and for every tweet your defines d…
Comment linking /egont-part-ii/
on “Ask HN: What projects are you working on?”Submitted Web scraping 101: pulling stories from Hacker News as a story · 2 points
HN discussionIf you liked this article try this "Challenging Google’s Search Engine" [1] from early 2013. [1] http://blog.databigbang.com/tag/duck-duck-go/
Comment linking /tag/duck-duck-go/
on “We need more boutique search engines”Submitted Using Google Chrome for web scraping as a story · 2 points
HN discussionI use HTMLUnit for scraping javascript based sites. Since Python is a lovely language and HTMLUnit is written in Java I use Jython. You can take a look at one of my articles on the subject: http://blog.databigbang.com/web-scraping-ajax-and-javascript...
Comment linking /web-scraping-ajax-and-javascript-sites/
on “The uncanny valley of web scraping”Submitted Enriching a list of URLs with Google Page Rank as a story · 2 points
HN discussionI have another alternative to retrieve the historic RSSs beyond the actual one. Using the Google Reader "NoAPI": http://blog.databigbang.com/extraction-of-main-text-content/ And there are additional resources at the end.
Comment linking /extraction-of-main-text-content/
on “The uncanny valley of web scraping”Submitted Running your own anonymous rotating proxies as a story · 2 points
HN discussionLet's be honest: Native Apps can be reverse engineered, the Web/Cloud don't. And reverse engineering is an important right. When web applications have only limited APIs, limited call rates, and changing everytime I can only think that I can turn on my Apple II and continue to use the same applications as 30 years ago. Again, don't compare Google Spreadsheet with Microsoft Excel, look at the Microsoft balance sheet.…
Comment linking /google-search-no-api/
on “Apps are too much like 1990's CD-ROMs and not enough like the Web”Submitted Automated discovery of company blog feeds and social media accounts as a story · 2 points
HN discussionThe approach that I am working on is a web orchestration language. You can write s-expressions specifying the dataflow. Obviously alternative frontends can be built to output the s-expression. I published more information on: - Ideas: Egont, A Web Orchestration Language: http://blog.databigbang.com/ideas-egont-a-web-orchestration-... - Egont Part II: http://blog.databigbang.com/egont-part-ii/
Comment linking /egont-part-ii/
on “The Forgotten Yahoo Project That Inspired Two Recently Funded Startups”Submitted Integrating Google Analytics into your Company Loop with an Excel Add-on as a story · 2 points
HN discussion