Top all-time · #10485
Earthmover
Company blog · earthmover.io/blog · on HN as tomnicholas1datasystems
Earthmover is the company blog at earthmover.io, writing about data and systems. Since 2022, 7 different Hacker News readers have linked it 21 times, most often “Earthmover – The cloud platform for scientific data teams”. It is still writing: the last post was on 2026-10-30, and you can follow it by feed.
The Science Data Cloud. Accelerating progress by making the world's scientific data accessible and fast for humans and AI.
Most-linked posts?
- Earthmover – The cloud platform for scientific data teams2 people · 2 mentions · first linked 2024-04-16
- Tensors vs. Tables2 people · 2 mentions · first linked 2025-04-03
What Hacker News says?
Great post! Hi Ali! I think what's missing here is an analysis of what is gained by moving the weather data into a RDBMS. The motivation is to speed up queries. But what's the baseline? As someone very familiar with this tech landscape (maintainer of Xarray and Zarr, founder of https://earthmover.io/ ), I know that serverless solutions + object storage can deliver very low latency performance (sub second) for timese…
Comment linking /
on “Loading a trillion rows of weather data into TimescaleDB”Submitted Icechunk: An open-source, cloud-native transactional tensor storage engine as a story · 15 points
HN discussion> Data from physics, chemistry, biology experiments There are PetaBytes of important scientific data locked in archival file formats. The first step is to make this efficiently readable. https://www.earthmover.io/blog/virtual-zarr
Comment linking /blog/virtual-zarr
on “Google DeepMind CEO Demis Hassabis is stepping down”Submitted Cloud-optimizing the GOES-16 satellite data archive without copying data as a story · 6 points
HN discussionYou should look at Icechunk. Your imaging data is structured (it's a multidimensional array), so it should be possible be to represent it as "Virtual Zarr". Then you could commit it to an Icechunk store. https://earthmover.io/blog/icechunk
Comment linking /blog/icechunk
on “Data Version Control”Submitted What Is Cloud-Optimized Scientific Data? as a story · 3 points
HN discussionIf you're wondering this you should look at Icechunk too, which was open-sourced just this week. It's Apache Iceberg but for multidimensional data (e.g. Zarr). https://earthmover.io/blog/icechunk https://news.ycombinator.com/item?id=41850352
Comment linking /blog/icechunk
on “Data Version Control”Submitted Exploring Icechunk scalability: untangling S3's prefix story as a story · 3 points
HN discussionAnyone interested in accessing GOES data at scale will find this interesting - I created a Zarr index over the 7 billion chunks of data in the GOES-16 archive. https://www.earthmover.io/blog/virtual-zarr
Comment linking /blog/virtual-zarr
on “Goes-19 weather satellite enters Safe Hold mode”Submitted Tensors vs. Tables as a story · 3 points
HN discussionAgree 100%. This is big part of the motivation behind our new startup Earthmover: https://earthmover.io/ Our mission is to make it easier to work with scientific data at scale in the cloud, focusing mainly on the climate, weather, and geospatial vertical. My cofounder Joe Hamman and I are climate scientists who helped create the Pangeo project. We are also core devs on the Python packages Xarray and Zarr. We think t…
Comment linking /posts/earthmover-is-hiring/
on “Request for Startups: Climate Tech”Submitted Tensors vs. Tables: Why tabular tools trip over gridded data as a story · 3 points
HN discussionThe generalized form of this range-request-based streaming approach looks something like my project VirtualiZarr [0]. Many of these scientific file formats (HDF5, netCDF, TIFF/COG, FITS, GRIB, JPEG and more) are essentially just contiguous multidimensional array(/"tensor") chunks embedded alongside metadata about what's in the chunks. Efficiently fetching these from object storage is just about efficiently fetching…
Comment linking /blog/announcing-flux
on “Show HN: Streaming gigabyte medical images from S3 without downloading them”Submitted Icechunk 1.0: Production-Grade Cloud-Native Array Storage Is Here as a story · 1 points
HN discussionSubmitted Earthmover – The cloud platform for scientific data teams as a story · 1 points
HN discussionSubmitted Arraylake now available in private beta as a story · 1 points
HN discussion