Top all-time · #5257
Hopsworks
Company blog · hopsworks.ai/postaidata
Hopsworks is the company blog at hopsworks.ai, writing about ai and data. Since 2020, 14 different Hacker News readers have linked it 120 times, most often “Make better prediction models with ML Pipelines”. No feed was found, so read it on the site itself.
Too few sampled comments (6) to say how often HN recommends it.?
Why HN links it
HN readers link Hopsworks pages as references: a research paper on HopsFS-S3 storage, a guide to file formats, and a glossary entry on RLHF. Commenters cite them as sources, not for their writing.
Summarised by Claude from a sample of 6 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Build, deploy, and scale production ML systems with Hopsworks. The Feature Store and MLOps platform for real-time AI, trusted by leading teams.
Most-linked posts?
- Make better prediction models with ML Pipelines4 people · 5 mentions · first linked 2023-11-14
- The 10 Fallacies of MLOps3 people · 5 mentions · first linked 2025-02-26
- "Machine Learning Pipeline" as a mental map should be "considered harmful"3 people · 4 mentions · first linked 2023-09-21
- Build a Feature Pipeline with Apache Flink in Hopsworks3 people · 3 mentions · first linked 2023-07-01
- A Programmer's Intro to Feature Types for Machine Learning3 people · 3 mentions · first linked 2022-05-31
- /2 people · 5 mentions · first linked 2020-04-28
- Real-AI needs rolling aggregations and AI won't build them for you2 people · 5 mentions · first linked 2026-03-04
- The Feature Store: what's new – a deep dive2 people · 4 mentions · first linked 2023-12-21
- Improve prediction quality with RAG LLMs2 people · 3 mentions · first linked 2023-11-02
- Vibe Migrating off SaaS >1k Pages and Losing 80% of our traffic2 people · 3 mentions · first linked 2026-02-05
What Hacker News says?
Architecturally, it is a scale-out metadata filesystem [ref]. Other related distributed file systems are Collosus, Tectonic (Meta), ADLSv2 (Microsoft), HopsFS (Hopsworks), and I think PolarFS (Alibaba). They all use different distributed row-oriented DBs for storing metadata. S3FS uses FoundationDB, Collosus uses BigTable, Tectonic some KV store, ADLSv2 (not sure), HopsFS uses RonDB. What's important here with S3FS…
Comment linking /post/scalable-metadata-the-new-breed-of-file-systems
on “An intro to DeepSeek's distributed file system”Submitted Migrating from AWS to a European Cloud – How We Cut Costs by 62% as a story · 143 points
HN discussionAt Hopsworks we built HopsFS-S3 to improve things like listing (becomes a partition pruned scan of a in-memory DB), atomic renames and added a block/object caching layer using NVMe drives. You can read the research paper here if you are curious: https://www.hopsworks.ai/research-papers/hopsfs-s3-extending...
Comment linking /research-papers/hopsfs-s3-extending-object-stores-with-posix-like-semantics
on “S3 is files, but not a filesystem”Submitted Machine Learning Myths Debunked as a story · 12 points
HN discussionI was expecting a discussion of the foundations of data modelling: star schema vs snowflake schema data models vs one big table. The benefits of 3NF vs when you have to denormalize. This underlying choice of data model actually does define your destiny. What I think the author was thinking of is domain modelling and correct entity identification, which is also important. It's a layered approach - and if you ignore t…
Comment linking /post/the-journey-from-star-schema-to-snowflake-schema-in-the-feature-store
on “Your data model is your destiny”Submitted Testing feature logic, transformations, and feature pipelines with pytest as a story · 12 points
HN discussionLogical Clocks | Mid/Senior Frontend Developer (React) | Europe (Stockholm or Remote) | Full-time | https://logicalclocks.com Logical Clocks is the company behind Hopsworks. Hopsworks is an open-source feature store that allows teams to organize and scale their feature engineering efforts. ( https://hopsworks.ai ) We recently raised our series A (5M euro) and we are expanding the teams. We are looking for a mid/seni…
Comment linking /
on “Ask HN: Who is hiring? (August 2021)”Submitted The Big Dictionary of MLOps as a story · 9 points
HN discussionYou should checkout Hopsworks (disclaimer: work on it). It does all of the above (including a Feature Store, notebooks as jobs, airflow for ML pipelines, model serving (TensorFlowServing, Flask, and soon KFserving), experiments, a project-based multi-tenancy model that supports sensitive data on a shared cluster, and a UI. It does not have a labelling system - but you can pip install libraries. You don't need to lea…
Comment linking /
on “Netflix's Metaflow: Reproducible machine learning pipelines”Submitted "Machine Learning Pipeline" as a mental map should be "considered harmful" as a story · 8 points
HN discussion> 1. Fine-tuning OSS embedding models on your real-world query patterns This is not as easy as you make it sound :) Typically, the embeddings are multi-modal: the query string maps to a relevant document that I want to add as context to my prompt. If i collect lots of new query strings, i need to know the ground truth "relevant document" it maps to. Then I can use the two-tower embedding model to learn the "correct"…
Comment linking /dictionary/two-tower-embedding-model
on “Are we at peak vector database?”Submitted From Pandas to Features to Models to Predictions – Diving into the Hopsworks API as a story · 8 points
HN discussionhttps://logicalclocks.com with their ML + Feature Store open source platform Hopsworks and their managed cloud version https://hopsworks.ai
Comment linking /
on “We Need DevOps for ML Data”Submitted Migrating our SaaS off AWS along with 8k Data Scientists as a story · 7 points
HN discussionYeah, https://hopsworks.ai is an alternative to EMR that includes HopsFS out-of-the-box. You get $4k free credits right now - it includes Spark, Flink, Python, Hive, Feature Store, GPUs (tensorflow), Jupyter, notebooks as jobs, AirFlow. And a UI for development and operations. So kind of like Databricks with more of a ML focus, but still supporting Spark.
Comment linking /
on “HopsFS: 100x Times Faster Than AWS S3”Submitted RonDB Upholds the Principles Redis Left Behind as a story · 7 points
HN discussionNot really, but you can try it out on https://hopsworks.ai It's conceptually similar to EMR in the way it works. You connect your AWS account and we'll deploy a cluster there. HopsFS will run on top a S3 bucket in your organization. You get a fully featured Spark environment (With metrics and logging included - no need for cloudwatch). UI with Jupyter notebooks, the Hopsworks feature store and ML capabilities that E…
Comment linking /
on “HopsFS: 100x Times Faster Than AWS S3”Submitted The Feature Store: what's new – a deep dive as a story · 7 points
HN discussionI am writing a tabular data textbook for O'Reilly - on building ML systems with a feature store. I try to be opinionated about modelling - XGBoost is all you really need, but the challenges are more like you say - how to prevent data leakage (ASOF LEFT JOIN (or use a feature store)), separating model-independent data transformations from model-specific data transformations, APIs for things like time-series splits, l…
Comment linking /lp/oreilly-book-building-ml-systems-with-a-feature-store
on “Applied Machine Learning for Tabular Data”Submitted Use more features in online ML models with the Snowflake Schema Data Model as a story · 6 points
HN discussionWe switched to OpenSearch from Elastic, as we have an open source platform. It's actually better than open source elastic - with the security plugin and k-nn plugin (an embedding store): https://www.hopsworks.ai/post/open-distro-for-elasticsearch-...
Comment linking /post/open-distro-for-elasticsearch-dynamic-support-hopsworks
on “OpenSearch – open-source search and analytics based on Apache 2.0 Elasticsearch”Submitted From Lakehouse to AI Lakehouse with a Python-Native Query Engine as a story · 6 points
HN discussionhttps://xkcd.com/927/ ? We still use HDF ( https://en.wikipedia.org/wiki/Hierarchical_Data_Format ). But I wonder, if I would choose a new file format today, what to choose? Nimble is maybe too new and there is too less experience with it (outside Meta). Is there anywhere a good overview of all available options, and some fair comparison? Some that I found, but older: https://www.hopsworks.ai/post/guide-to-file-form…
Comment linking /post/guide-to-file-formats-for-machine-learning
on “The Nimble File Format by Meta”Submitted The 10 Fallacies of MLOps as a story · 5 points
HN discussionHere's a reference describing what a context window for LLMs is: https://www.hopsworks.ai/dictionary/context-window-for-llms
Comment linking /dictionary/context-window-for-llms
on “Streaming LLM – No limit on context length for your favourite LLM”Submitted Polars vs. Pandas for Feature Engineering as a story · 5 points
HN discussionS3 Files was launched today without support for atomic rename. This is not something you can bolt on. Can you imagine running Claude Code on your S3 Files and it just wants to do a little house cleaning, renaming a directory and suddenly a full copy is needed for every file in that directory? The hardest part in building a distributed filesystem is atomic rename. It's always rename. Scalable metadata file systems, l…
Comment linking /post/scalable-metadata-the-new-breed-of-file-systems
on “S3 Files and the changing face of S3”Submitted Multi-Region High Availability Comes to Feature Stores as a story · 5 points
HN discussionHere is an article I wrote this week with a section on Feldera - how it uses its incremental compute engine to compute "rolling aggregates" (the most important real-time feature for detecting changes in user behavior/pricing/anamalies). https://www.hopsworks.ai/post/rolling-aggregations-for-real-...
Comment linking /post/rolling-aggregations-for-real-time-ai
on “Nobody ever got fired for using a struct”Submitted The definitive guide to feature stores as a story · 5 points
HN discussion
