Top all-time · #10075
Traceloop
Company blog · traceloop.com/blogaiprogramming
Traceloop is the company blog at traceloop.com, writing about ai and programming. Since 2023, 8 different Hacker News readers have linked it 14 times. No feed was found, so read it on the site itself.
Too few sampled comments (4) to say how often HN recommends it.?
Why HN links it
HN readers link Traceloop for OpenLLMetry, its open-source OpenTelemetry-based LLM observability tooling and docs. Commenters mostly cite it in comparisons of LLM observability products.
Summarised by Claude from a sample of 4 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Traceloop turns evals and monitors into a continuous feedback loop - so every release gets better
What Hacker News says?
I've been keeping a rolling list of LLMOps/AI agent observability products funded by YC. What problems does Lucidic solve that the others do not? https://hegel-ai.com https://www.vellum.ai/ https://www.parea.ai http://baserun.ai https://www.traceloop.com https://www.trychatter.ai https://talc.ai https://langfuse.com https://humanloop.com https://uptrain.ai https://athina.ai https://relari.ai https://phospho.ai https…
Comment linking /
on “Launch HN: Lucidic (YC W25) – Debug, test, and evaluate AI agents in production”Submitted Lessons Learned from Using LLMs to Evaluate LLMs as a story · 4 points
HN discussionYou don't need to create an account in 2 services - you can just connect the SDK directly to Grafana - https://traceloop.com/docs/integration/grafana Happy to assist if needed over slack - https://traceloop.com/slack
Comment linking /slack
on “Show HN: You don't need to adopt new tools for LLM observability”Submitted Observability for LLM's with Opentelemetry as a story · 4 points
HN discussionThanks for the issues - I'll fix it! :sweat_smile: Reg. Grafana and others - it's simple, just set the env vars - https://www.traceloop.com/docs/openllmetry/integrations/intr...
Comment linking /docs/openllmetry/integrations/introduction
on “Show HN: You don't need to adopt new tools for LLM observability”Submitted Traceloop Is Joining ServiceNow as a story · 2 points
HN discussionI was looking to see what the actual metrics would be for a completion, to see if this is something of interest to me. So I tried to run the example here: https://www.traceloop.com/openllmetry Problem 1 (very minor): it's missing an `import os` Problem 2: I need an API key. Problem 3: The link that it tells me to go to for an API key is malformed: https://https//app.traceloop.com/settings/api-keys Is there a way to…
Comment linking /openllmetry
on “Show HN: You don't need to adopt new tools for LLM observability”Submitted Evaluating LLM quality with the ROUGE metric as a story · 1 points
HN discussionSuper easy - you can just use the standalone instrumentations directly - https://www.traceloop.com/docs/openllmetry/tracing/without-s...
Comment linking /docs/openllmetry/tracing/without-sdk
on “Show HN: You don't need to adopt new tools for LLM observability”I tend to find classic NLP metric more predictable and stable than "LLM as a judge" metrics so I'd try to see if you rely on them more. We've written a couple of blog posts about some of them: https://www.traceloop.com/blog
Comment linking /blog
on “Launch HN: Traceloop (YC W23) – Detecting LLM Hallucinations with OpenTelemetry”We trained our own models for some of them, and we combined some well known NLP metrics (like Gruen [1]) to make this work. You're right that it's hard to figure out how to "trust" these metrics. But you shouldn't look at them as a way to get an objective number about your app's performance. They're more of a way to detect deltas - regressions or changes in performance. When you get more alerts, or more negative res…
Comment linking /blog/gruens-outstanding-performance-in-llm-quality-evaluation
on “Launch HN: Traceloop (YC W23) – Detecting LLM Hallucinations with OpenTelemetry”From what I can see, Openllmetry asks you to manually call tracer for non-default OpenAI libraries (i.e. not Python/Node.js) [1] OpenLIT might be easier to integrate with any language that supports OTEL for HTTP clients — you just trace your HTTP calls to OpenAI. [1] https://www.traceloop.com/docs/openllmetry/getting-started-r...
Comment linking /docs/openllmetry/getting-started-ruby
on “Show HN: OpenLIT – Open-Source LLM Observability with OpenTelemetry”Traceloop's landing page is all about model quality, not metrics. Their open source OpenLLMetry is the metrics part and hooks into the OpenTelemetry ecosystem. There should be no issue with getting alerts via the ecosystem, it's promanent on their pages. https://www.traceloop.com/
Comment linking /
on “Launch HN: Openlayer (YC S21) – Testing and Evaluation for AI”
