Top all-time · #1150
Dagster Blog
Company blog · dagster.io/blogdataprogramming
Dagster Blog is the company blog at dagster.io, writing about data and programming. Since 2020, 61 different Hacker News readers have linked it 84 times, most often “Tracking the Fake GitHub Star Black Market with Dagster, Dbt and BigQuery”. It is still writing: the last post was on 2026-09-08, and you can follow it by feed.
Too few sampled comments (29) to say how often HN recommends it.?
Why HN links it
HN readers mostly link Dagster's blog as a source on data orchestration: LangChain/LLM doc Q&A, fake GitHub stars, software-defined assets, and dbt/DuckDB stacks. Few comments judge the writing itself.
Summarised by Claude from a sample of 29 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Dagster is the data orchestrator platform that helps you build, schedule, and monitor reliable data pipelines—fast, flexible, and built for teams.
Most-linked posts?
- Tracking the Fake GitHub Star Black Market with Dagster, Dbt and BigQuery8 people · 8 mentions · first linked 2023-03-16
- Build a GitHub Support Bot with GPT3, LangChain, and Python7 people · 9 mentions · first linked 2023-01-09
- Dagster: Data flow engine from first principle; unlike AirFlow5 people · 6 mentions · first linked 2021-05-13
- Experience with Dagster.io?5 people · 6 mentions · first linked 2020-11-24
- The Rise of Medium Code5 people · 5 mentions · first linked 2024-06-10
- Dagster4 people · 4 mentions · first linked 2021-04-26
- Rebundling the Data Platform3 people · 3 mentions · first linked 2022-02-17
- Adding types to a large Python codebase2 people · 2 mentions · first linked 2022-11-08
- Postgres: A better message queue than Kafka?2 people · 2 mentions · first linked 2022-10-04
- Data pipelines are not workflows2 people · 2 mentions · first linked 2023-01-09
What Hacker News says?
Dagster Labs | Software Engineers, Product Managers, Designers | SF, Minneapolis, NYC or Remote | Full-time | Open-source Dagster is the data orchestration platform built for productivity. Data engineers all around the world, in every industry, use Dagster to build scalable and maintainable data applications like ETL pipelines, ML training pipelines, data integrations and similar systems. Data applications are the t…
Comment linking /careers
on “Ask HN: Who is hiring? (July 2024)”Submitted Postgres: A better message queue than Kafka? as a story · 65 points
HN discussionAlso not to be confused with https://dagster.io/
Comment linking /
on “Dagger: a new way to build CI/CD pipelines”Submitted Counterintuitive web worker behavior as a story · 51 points
HN discussionAuthor here. Agreed — I think adding some comparisons to other database partitioning strategies would be helpful. My 2 cents, specifically about manually partitioning Postgres/MySQL (rather than using something like Citus or Vitess): SQLite-on-the-server works similarly to Cassandra/DynamoDB in how it partitions data. The number of partitions is decoupled from the number of databases you're running, since data is au…
Comment linking /glossary/data-rebalancing
on “SQLite-on-the-server is misunderstood: Better at hyper-scale than micro-scale”Submitted We deploy 5X faster with warm Docker containers as a story · 28 points
HN discussionhttps://www.dagster.io/blog/software-defined-assets Of course, nothing stops Airflow or other tools fron thinking this way as well.
Comment linking /blog/software-defined-assets
on “Airflow's Problem”Submitted Rebundling the Data Platform as a story · 13 points
HN discussionI've had good success with https://dagster.io , which is much more opinionated about your pipelines, including properly typing inputs and outputs.
Comment linking /
on “Airflow's Problem”Submitted Dagster as a story · 10 points
HN discussionFunny enough this post mirrors quite a bit of our thinking over at Dagster! https://dagster.io/blog/rebundling-the-data-platform
Comment linking /blog/rebundling-the-data-platform
on “The Unbundling of Airflow”Submitted Dagster 0.11.0 – Supporting Dynamic Orchestration and Asset Lineage as a story · 8 points
HN discussionI wondered how it related to https://dagster.io/
Comment linking /
on “Dagger Python SDK: Develop Your CI/CD Pipelines as Code”Submitted Build a poor man’s data lake from scratch with DuckDB as a story · 7 points
HN discussionI have worked with Airflow during the past three years, but recently we adopted Dagster and I have been using it for the past 3 months. I have found it quite joyful to use and the experience has been very positive. Its main advantages compared to Airlfow (IMO): - A great UI - It forces you to clearly define inputs, outputs and types. - Separation of concerns: Between configuration and data, between processing and IO…
Comment linking /blog/dagster-airflow
on “ETL Pipelines with Airflow: The Good, the Bad and the Ugly”Submitted Data pipelines are not workflows as a story · 6 points
HN discussionWhat is the difference to https://dagster.io/ ?
Comment linking /
on “Show HN: Benthos Studio – A modern take on Yahoo Pipes”Submitted Dagster: Data flow engine from first principle; unlike AirFlow as a story · 5 points
HN discussionWe have been using pex to deploy code to running docker containers in ECS to avoid the cold start delay. Cuts down the iteration loop time for development significantly. https://dagster.io/blog/fast-deploys-with-pex-and-docker
Comment linking /blog/fast-deploys-with-pex-and-docker
on “Pex: A tool for generating .pex (Python EXecutable) files, lock files and venvs”Submitted Dagster: The Data Orchestrator as a story · 5 points
HN discussionVersioning is indeed a hard topic, especially for data science/engineering projects in production. When you have a pipleine defined as a complex DAG of operations, you can't just version the entire thing, unless you have enough resources to re-compute from scratch with every change, which is wasteful. So then, you have to keep track of data dependencies and their versions if you would like to ensure reproducibility.…
Comment linking /
on “API versioning has no “right way” (2017)”Submitted Dagster Cloud GA Launch as a story · 4 points
HN discussionIf anyone is tempted to do this, here is a good blog post on how to build your own retrieval-based LLM powered conversational agent. https://dagster.io/blog/chatgpt-langchain
Comment linking /blog/chatgpt-langchain
on “Sundar Pichai: An important next step on our AI journey”Submitted Rebundling the Data Platform – Dagster Blog as a story · 4 points
HN discussionhttps://dagster.io/
Comment linking /
on “AWS Managed Workflows for Apache Airflow”Submitted Moving past Airflow: Why Dagster is the next-generation data orchestrator as a story · 4 points
HN discussionDagster already offers Dagster Cloud which came out a month or so ago[1]. It's a fully hosted solution for teams that don't want to run their own. [1] https://dagster.io/cloud
Comment linking /cloud
on “Dabbling with Dagster vs. Airflow”Submitted Prefect Is Acquiring Dagster as a story · 3 points
HN discussionFor bigger workflows, this declarative pattern is already implemented by orchestrators like Dagster, Flyte, and recently Airflow; e.g., https://dagster.io/blog/declarative-scheduling [fixed]
Comment linking /blog/declarative-scheduling
on “The Missing Manual for Signals: State Management for Python Developers”Submitted Mental Models for Data Platform Engineers (Inspired by Poor Charlie's Almanack) as a story · 3 points
HN discussion
