Top all-time · #11041
Great Expectations • Great Expectations
Company blog · greatexpectations.io/blogdataprogramming
Great Expectations • Great Expectations is the company blog at greatexpectations.io, writing about data and programming. Since 2020, 7 different Hacker News readers have linked it 10 times, most often “Great Expectations: data testing, documentation, and profiling”. No feed was found, so read it on the site itself.
Too few sampled comments (3) to say how often HN recommends it.?
Why HN links it
HN commenters point to this site for Great Expectations, an open-source tool for validating data with assertions and auto-generated docs, often in Airflow pipelines. They recommend the tool itself, not the writing.
Summarised by Claude from a sample of 3 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
GX Core is an open source framework for testing, validating, and documenting data quality across modern data pipelines, workflows, and teams.
Most-linked posts?
- /2 people · 2 mentions · first linked 2020-08-19
- Great Expectations: data testing, documentation, and profiling2 people · 2 mentions · first linked 2021-01-12
- Why data quality is key to successful ML Ops2 people · 2 mentions · first linked 2020-09-28
What Hacker News says?
You can use the now widely adopted Great Expections[0] library, which fits exactly this use-case for data validation! [0] https://greatexpectations.io
Comment linking /
on “Parse, Don't Validate (2019)”Submitted Great Expectations: data testing, documentation, and profiling as a story · 35 points
HN discussionCheckout great expectations ( https://greatexpectations.io/ ). You basically run assertions on your data. It auto-generates documentation for you and you can also store results of your validations in a flat file or database. I think there are dbt modules that try to mimic great expectations' validation but I like the docs that come with great expectations
Comment linking /
on “ETL Pipelines with Airflow: The Good, the Bad and the Ugly”Submitted We don't do transformations for Expectations as a story · 1 points
HN discussionTesting Pipeline in Airflow is bit pain, but great expectation makes runtime pipeline validation much easier. https://greatexpectations.io For unit/integration tests we ended up doing lot of Docker in Docker setup.
Comment linking /
on “Effective Airflow Development”Submitted Why data quality is key to successful ML Ops as a story · 1 points
HN discussionSubmitted Why data quality is key to successful ML Ops as a story · 1 points
HN discussionSubmitted Data Validation Tool Webinar: “Great Expectations 101 Getting Started” as a story · 1 points
HN discussionSubmitted Just One More Stratification Or: How to say “no” as a data person“ as a story · 1 points
HN discussionSubmitted Poll Results: A pod of whales. A murder of crows. A [_____] of data scientists as a story · 1 points
HN discussion