Top all-time · #2115
WarpStream Blog
Company blog · warpstream.com/blogsystemsprogrammingdata
WarpStream Blog is the company blog at warpstream.com, writing about systems, programming and data. Since 2023, 35 different Hacker News readers have linked it 59 times, most often “Garbage collection of object storage at scale”. It is still writing: the last post was on 2026-10-28, and you can follow it by feed.
WarpStream is a diskless, Kafka-compatible streaming platform on object storage. Zero disks, zero inter-AZ costs, infinite scalability in your VPC.
Most-linked posts?
- Garbage collection of object storage at scale5 people · 5 mentions · first linked 2025-04-10
- /5 people · 5 mentions · first linked 2024-10-23
- S3 Express Is All You Need3 people · 4 mentions · first linked 2023-11-28
- Kafka Is Dead, Long Live Kafka3 people · 3 mentions · first linked 2023-08-07
- Minimizing S3 API costs with distributed MMAP2 people · 3 mentions · first linked 2023-10-11
- Kafka Transactions Explained (Twice)2 people · 2 mentions · first linked 2025-01-14
- /2 people · 2 mentions · first linked 2024-01-15
- The case for an iceberg-native database2 people · 2 mentions · first linked 2025-09-17
- The Art of Being Lazy(log)2 people · 2 mentions · first linked 2026-02-04
- Deterministic simulation testing for our entire SaaS2 people · 2 mentions · first linked 2024-03-12
What Hacker News says?
This is awesome! We got some really technical deep dives here https://www.warpstream.com/blog-category/engineering
Comment linking /blog-category/engineering
on “Show HN: Engineering.fyi – Search across tech engineering blogs in one place”Submitted Bento: Open-source fork of the project formerly known as Benthos as a story · 219 points
HN discussionThis isn't appropriate for all use-cases, but one way to address your and GP's problem is as follows: 1. Aggregate (in-memory or on cheap storage) events in the publisher application into batches. 2. Ship those batches to S3/alike, NFS that clickhouse can read, or equivalent (even a dead-simple HTTP server that just receives file POSTs and writes them to disk, running on storage that clickhouse can reach). The tool…
Comment linking /
on “A new JSON data type for ClickHouse”Submitted Deterministic simulation testing for our entire SaaS as a story · 203 points
HN discussionThis is clever. And as I understand it, one of the tricks WarpStream ( https://www.warpstream.com ) use to reduce the costs of operating a Kafka cluster.
Comment linking /
on “Slashing data transfer costs in AWS”Submitted Garbage collection of object storage at scale as a story · 96 points
HN discussionIt's a bit sad that many modern databases were recently acquired. They had the potential to bring a lot of innovations. 1. https://www.warpstream.com/ 2. https://www.orioledb.com/ 3. https://quickwit.io/
Comment linking /
on “Datadog acquires Quickwit”Submitted Tiered storage won't fix Kafka as a story · 65 points
HN discussionUpdate: Just read this article[1] which clarifies S3 Express One Zone. Yes, performance is greatly improved, but actually storage costs are 8x more than a standard S3 bucket. The naming S3 Express One Zone is terrible and a bit misleading on pricing changes. [1] https://www.warpstream.com/blog/s3-express-is-all-you-need
Comment linking /blog/s3-express-is-all-you-need
on “Colossus for Rapid Storage”Submitted Warpstream Joined Confluent as a story · 35 points
HN discussionNot to be confused with https://www.warpstream.com/
Comment linking /
on “Jepsen: Bufstream 0.1”Submitted Kafka Is Dead, Long Live Kafka as a story · 29 points
HN discussionhttps://www.warpstream.com/blog/rss.xml
Comment linking /blog/rss.xml
on “Deterministic simulation testing for our entire SaaS”Submitted The case for an iceberg-native database as a story · 28 points
HN discussionOff topic: warpstream's calculator on the pricing page is pretty cool https://www.warpstream.com/pricing That breakdown switch is a lovely touch.
Comment linking /pricing
on “Deterministic simulation testing for our entire SaaS”Submitted The Original Sin of Cloud Infrastructure as a story · 28 points
HN discussionFor Rust-based Kafka alternatives, I like Tansu[1]. It at least provides Kafka API parity, and critically also gives users a pluggable backend (embedded SQLite, S3 for low cost diskless type workloads and Postgres because just use Postgres) It’s nice to try and out innovate Kafka, but I fear the network effect can’t be beaten unless the alternative is 10x better. Something like Warpstream’s architecture[2] had a sho…
Comment linking /
on “Show HN: Walrus – a Kafka alternative written in Rust”Submitted Multiple Regions, Single Pane of Glass as a story · 20 points
HN discussionIt's my understanding that the newer generation of data lakes still make use of a tiny, strongly consistent metadata database to keep track of what is where. This is orders of magnitudes smaller than what you'd have by putting everything in the same database, but it's still there. This is also the case in newer data streaming platforms (e.g. https://www.warpstream.com/blog/kafka-is-dead-long-live-kafk... ). I'm curi…
Comment linking /blog/kafka-is-dead-long-live-kafka
on “Transactional Object Storage?”Submitted Zero Disks Is Better (For Kafka) as a story · 16 points
HN discussionShort answer is that MSK has (almost) all the same problems as OSS Kafka. Kinesis streams is a different beast that would require its own blog post. RE:numbers: https://www.warpstream.com/blog/warpstream-benchmarks-and-tc...
Comment linking /blog/warpstream-benchmarks-and-tco
on “Tiered storage won't fix Kafka”Submitted Kafka as a KV Store: deduplicating keys with just 128 MiB of RAM as a story · 16 points
HN discussionI talk about this more here: https://www.warpstream.com/blog/s3-express-is-all-you-need RE: comparing to a single-zone Kafka cluster. A lot of people really dislike operating Kafka. Some people don't mind it and that's cool too, but its not the majority in my experience.
Comment linking /blog/s3-express-is-all-you-need
on “Tiered storage won't fix Kafka”Submitted Dealing with Rejection (In Distributed Systems) as a story · 12 points
HN discussionReminds me of FoundationDB and Antithesis[0]. [0]: https://www.warpstream.com/blog/deterministic-simulation-tes...
Comment linking /blog/deterministic-simulation-testing-for-our-entire-saas
on “Deterministic Simulation Testing in Rust: A Theater of State Machines”Submitted Anatomy of a serverless usage based billing system as a story · 9 points
HN discussionI think Ducklake[1] is a terrific example of this. They said "look, let's build a lake house over S3, but for the bit that needs strong consistency (the manifest of which S3 blobs are in play), let's use Postgres". Postgres as a metadata catalog or control plane is brilliant for this, since you get strong consistency and the scaling story around a metadata catalog is far different than the volume of data you need to…
Comment linking /
on “Postgres transactions are a distributed systems superpower”Submitted How we built a proxy to migrate Kafka producers with zero downtime as a story · 8 points
HN discussionReminds me of WarpStream: https://www.warpstream.com Similar idea but you have the power of S3 scale (if you really need it). For context, I do not work at WS. My company switched to it recently and we've seen great improvements over traditional Kafka.
Comment linking /
on “A distributed queue in a single JSON file on object storage”Submitted The Case for an Iceberg-Native Database as a story · 8 points
HN discussion
