Top all-time · #6416
Qubole
Company blog · qubole.com/blogdataprogrammingsystems
Qubole is the company blog at qubole.com, writing about data, programming and systems. Since 2014, 12 different Hacker News readers have linked it 14 times, most often “List 24K dirs takes 1700 seconds in AWS S3”. It is still writing: the last post was on 2025-10-13, and you can follow it by feed.
Too few sampled comments (7) to say how often HN recommends it.?
Why HN links it
HN commenters link Qubole's blog posts on Spark tuning, Presto, Hadoop networking and cloud storage benchmarks, mostly as references or while promoting Qubole's service. The comments do not discuss the writing itself.
Summarised by Claude from a sample of 7 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Qubole is the open data lake company . Open, simple and secure data lakes for machine learning, streaming analytics, data exploration, and ad-hoc analytics.
Most-linked posts?
- List 24K dirs takes 1700 seconds in AWS S32 people · 2 mentions · first linked 2016-02-15
What Hacker News says?
Couple of other points: 1. Enhanced Networking (SRIOV) only works in a VPC and not in EC2-Classic. 2. I think the 4x instances don't support 10Gb ethernet. If that is the case, it would also be instructive to test the 8x instances on S3. For some very application (Hadoop) specific tests of Enhanced Networking, please take a look at https://www.qubole.com/blog/product/hadoop-enhanced-networki...
Comment linking /blog/product/hadoop-enhanced-networking-aws
on “Feeding data to 1000 CPUs – comparison of S3, Google, Azure storage”Submitted Spark on Lambda as a story · 17 points
HN discussionWith Qubole you can offload data engineering to their platform. Cluster management is super simple. Hand rolled solutions in my experience are a pain and elastic cloud features take up time to build. Qubole's offering provides out of the box experience for most big data engines out there. Presto/ Spark/ Hive/ Pig - what have you - all work with your data living in S3 (or any other object storage). I believe they hav…
Comment linking /blog/product/optimizing-s3-bulk-listings-for-performant-hive-queries/
on “Data Wrangling at Slack”Submitted Learn Apache Spark Basics for SQL, Python and Scala in 30 Minutes as a story · 5 points
HN discussion> I'd hope so if they're selling a product based on FOSS code, there has to be a value add to justify it They have some proprietary features like DBIO [1]. They also have some cloud-specific features like storage autoscaling [2] that would not be available in OSS Spark. Even Delta Lake [3] used to be proprietary, but I suspect the rise of open-source frameworks like Iceberg led them to open-source it. Shameless plug…
Comment linking /blog/auto-scaling-in-qubole-with-aws-elastic-block-storage/
on “Snowflake’s response to Databricks’ TPC-DS post”Submitted List 24K dirs takes 1700 seconds in AWS S3 as a story · 2 points
HN discussionQubole offers Presto as a Service with a free trial ( http://www.qubole.com ) with a bunch of interesting enhancements for the cloud. We've found Presto to be a great engine for realtime queries over large datasets.
Comment linking /
on “Using Presto in Our Big Data Platform on AWS”Submitted Caching in Presto as a story · 2 points
HN discussionYou can try Qubole [0]. The pricing is a small percentage of what you pay to the cloud provider, so it's predictable to an extent. [0]: https://www.qubole.com/ Disclosure: I work here.
Comment linking /
on “Ask HN: What are the alternatives of hosting Apache Spark?”Submitted Data Governance for SparkSQL as a story · 1 points
HN discussionGetting this out of the way first, I've only started exploring non-Hadoop/non-HDFS Spark execution beyond some limited Amazon EMR work, but I'm interested in learning more about it. What follows is a combination of work experience and armchair research in the evenings. But I'm not claiming to be an expert. Have grown to really appreciate Spark in the Hadoop space. Started with plans to go with Impala, then went to H…
Comment linking /blog/spark-on-aws-lambda/
on “Ask HN: Why is there nobody talking about hadoop anymore?”Submitted How Qubole manages open source forks as a story · 1 points
HN discussionBlog post with previous discussion on the tool https://www.qubole.com/blog/introducing-quboles-spark-tuning...
Comment linking /blog/introducing-quboles-spark-tuning-tool/
on “Qubole Sparklens tool for performance tuning Apache Spark”Submitted Cost saving upto 90% in Hadoop clusters on AWS by using heterogeneous nodes as a story · 1 points
HN discussion
