Top all-time · #9819
Cleanlab
Company blog · cleanlab.ai/blogaidata
Cleanlab is the company blog at cleanlab.ai, writing about ai and data. Since 2022, 8 different Hacker News readers have linked it 38 times, most often “Overcoming Hallucinations with the Trustworthy Language Model”. No feed was found, so read it on the site itself.
Too few sampled comments (8) to say how often HN recommends it.?
Why HN links it
Commenters link cleanlab.ai as a product for scoring the trustworthiness of LLM outputs and as a data-centric ML tool. The comments are product pitches and citations rather than judgments of the writing.
Summarised by Claude from a sample of 8 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Cleanlab helps teams build safer AI agents by preventing incorrect responses from reaching users. Detect and remediate incorrect responses from any AI agent to ensure safety, compliance, and trust at scale.
Most-linked posts?
- Overcoming Hallucinations with the Trustworthy Language Model2 people · 3 mentions · first linked 2024-04-25
- Automated Data Quality at Scale2 people · 2 mentions · first linked 2023-07-27
- Datalab: A Linter for ML Datasets2 people · 2 mentions · first linked 2023-05-16
What Hacker News says?
Exactly, that's why my startup recommends all LLM outputs should come with trustworthiness scores: https://cleanlab.ai/tlm/
Comment linking /tlm/
on “Cursor IDE support hallucinates lockout policy, causes user cancellations”Submitted An open-source platform to catch all sorts of issues in all sorts of datasets as a story · 6 points
HN discussionMy startup is working on this fundamental problem. You can try out our early product here: https://cleanlab.ai/tlm/ (free to try, we'd love to hear your feedback)
Comment linking /tlm/
on “Cursor IDE support hallucinates lockout policy, causes user cancellations”Submitted Scaling human LLM Evals with open source crowdlab as a story · 4 points
HN discussionI see this fallacy often too. My company provides hallucination detection software: https://cleanlab.ai/tlm/ But we somehow end up in sales meetings where the person who requested the meeting claims their AI does not hallucinate ...
Comment linking /tlm/
on “Cursor IDE support hallucinates lockout policy, causes user cancellations”Submitted CleanVision: Audit Your Image Data for Better Computer Vision as a story · 3 points
HN discussionThere is however a subfield of statistical ML of model uncertainty quantification. I've developed a product by applying to it to LLMs that can score the trustworthiness of any LLM response. Like any ML-based product, my tool is not perfect, but it can detect incorrect LLM responses with pretty high precision/recall across applications spanning RAG / Q&A, data extraction, classification, summarization, ... I've publi…
Comment linking /blog/trustworthy-language-model/
on “Detecting when LLMs are uncertain”Submitted OpenAI's o1 surpassed using the Trustworthy Language Model as a story · 2 points
HN discussionHere are some benchmarks I ran that compare the precision/recall of various LLM error-detection methods, including logprobs and LLM self-evaluation / verbalized confidence: https://cleanlab.ai/blog/4o-claude/ These approaches can detect errors better than random guessing, but there are other approaches that are significantly more effective in practice.
Comment linking /blog/4o-claude/
on “SimpleQA”Submitted Overcoming Hallucinations with the Trustworthy Language Model as a story · 2 points
HN discussionTo try out an existing product that quantifies LLM uncertainty (accurately incorporating both aleatoric & epistemic uncertainty), you can try this Trustworthy Language Model I built (after similar research): https://tlm.cleanlab.ai/ TLM is an API you can use to quantify uncertainty of any LLM model: https://help.cleanlab.ai/tutorials/tlm/ Benchmarks showing these estimates more reliably detect bad answers & hallucin…
Comment linking /blog/trustworthy-language-model/
on “To Believe or Not Believe Your LLM”Submitted Detecting Errors in Numerical Data via Any Regression Model as a story · 2 points
HN discussionThis one is popular https://cleanlab.ai/
Comment linking /
on “Introduction to Data-Centric AI”Submitted The Future of Relevance Determination: Leveraging AI for Enhanced E-Discovery as a story · 2 points
HN discussionNew feature alert: Auto-train & deploy reliable ML models (more accurate than fine-tuned OpenAI LLMs) on messy real-world data — all in just a few clicks! Common reasons companies struggle to quickly get good ML models deployed and generating business value: messy data full of issues, a need to explore many ML models to train a good one, and infrastructural challenges serving predictions from the model. Now you can…
Comment linking /blog/model-deployment/
on “Train and Deploy Reliable Models on Messy Real-World Data with a Few Clicks”Submitted Train and Deploy Reliable Models on Messy Real-World Data with a Few Clicks as a story · 2 points
HN discussionSubmitted Detecting Dataset Drift and Non-IID Sampling: A K-Nearest Neighbors Approach as a story · 2 points
HN discussionSubmitted ActiveLab: Active Learning with Data Re-Labeling as a story · 2 points
HN discussionSubmitted Crowdlab: Effective algorithms to handle data labeled by multiple annotators as a story · 2 points
HN discussionSubmitted Cleanlab 2.0: Automatically Find Errors in ML Datasets as a story · 2 points
HN discussionSubmitted Detecting Annotation Errors in Semantic Segmentation Data as a story · 1 points
HN discussionSubmitted Automatically Find and Fix Issues in Multi-Label Datasets as a story · 1 points
HN discussionSubmitted Generate Better Synthetic Image Datasets with Prompt Engineering and Evaluation as a story · 1 points
HN discussionSubmitted Automated Quality Assurance for Object Detection Datasets as a story · 1 points
HN discussionSubmitted Automated Correction of Satellite Imagery Data as a story · 1 points
HN discussionSubmitted Cleanlab now supports all major ML tasks as a story · 1 points
HN discussionSubmitted Ensuring Reliable Few-Shot Prompt Selection for LLMs as a story · 1 points
HN discussionSubmitted Most AI and Analytics are impaired by data issues. Now AI can help you fix them as a story · 1 points
HN discussionSubmitted Automated Data Quality at Scale as a story · 1 points
HN discussionSubmitted Automated Data Quality at Scale as a story · 1 points
HN discussion