Top all-time · #3993
Vellum
Company blog · vellum.ai/blogaistartups
Vellum is the company blog at vellum.ai, writing about ai and startups. Since 2023, 19 different Hacker News readers have linked it 22 times, most often “The six levels of agentic behavior”. No feed was found, so read it on the site itself.
An assistant that knows you deeply, evolves alongside you, and belongs to no one else. Powered by memory that remembers the way you do.
Most-linked posts?
- /llm-leaderboard3 people · 3 mentions · first linked 2025-02-05
- The six levels of agentic behavior3 people · 3 mentions · first linked 2025-02-14
- /2 people · 3 mentions · first linked 2023-08-08
- Gemini Pro outperforms Claude 2 and GPT-4 on a classification task2 people · 2 mentions · first linked 2023-12-13
What Hacker News says?
Some attempts to benchmark and compare LLM's: https://artificialanalysis.ai/ https://benchlm.ai/ https://tokenscost.com https://www.vellum.ai/llm-leaderboard Are there any projects out there for doing something similar?
Comment linking /llm-leaderboard
on “Claude Haiku 5.5”Submitted I'd pay $2k/mo out of my own pocket to keep using Cursor as a story · 19 points
HN discussionGemini is not mediocre, have you used it lately? https://www.vellum.ai/llm-leaderboard
Comment linking /llm-leaderboard
on “DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]”Submitted The FaceMash of Prompt Evaluation as a story · 9 points
HN discussionSome of my saved bookmarks: - https://aider.chat/docs/leaderboards/ - https://www.prollm.ai/leaderboard - https://www.vellum.ai/llm-leaderboard - https://lmarena.ai/?leaderboard
Comment linking /llm-leaderboard
on “Gemini 2.0 is now available to everyone”Submitted Gemini Pro outperforms Claude 2 and GPT-4 on a classification task as a story · 5 points
HN discussionidk man https://js.langchain.com/docs/introduction/ https://www.vellum.ai/products/workflows-sdk https://github.com/transitive-bullshit/agentic which is not to say any of them got it right or wrong, but it is by no means "missing". the big question w all of them is do they deliver enough value to last. kudos to those who at least try, of course
Comment linking /products/workflows-sdk
on “Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby”Submitted The six levels of agentic behavior as a story · 4 points
HN discussionThat sounds like L1 in this article (there are six) https://www.vellum.ai/blog/levels-of-agentic-behavior
Comment linking /blog/levels-of-agentic-behavior
on “New tools for building agents”Submitted LLM Benchmarks: Overview, Limits and Model Comparison as a story · 3 points
HN discussion> LoRA does exactly the same thing as normal fine-tuning You wrote exactly so I'm going to say "no". To clarify what I mean: LoRA seeks to accomplish a similar goal as "vanilla" fine-tuning but with a different method (freezing existing model weights while adding adapter matrices that get added to the original). LoRA isn't exactly the same mathematically either; it is a low-rank approximation (as you know). > LoRA d…
Comment linking /blog/how-we-reduced-cost-of-a-fine-tuned-model-by-90-by-dynamically-swapping-lora-weights
on “Fine-tuning LLMs is a waste of time”Submitted AI psychosis is real, and you probably have it as a story · 2 points
HN discussionI've been keeping a rolling list of LLMOps/AI agent observability products funded by YC. What problems does Lucidic solve that the others do not? https://hegel-ai.com https://www.vellum.ai/ https://www.parea.ai http://baserun.ai https://www.traceloop.com https://www.trychatter.ai https://talc.ai https://langfuse.com https://humanloop.com https://uptrain.ai https://athina.ai https://relari.ai https://phospho.ai https…
Comment linking /
on “Launch HN: Lucidic (YC W25) – Debug, test, and evaluate AI agents in production”Submitted The LLM agent build guide as a story · 2 points
HN discussionThis is really cool. The annotation-to-testing-to-annotation-etc. feedback loop makes a ton of sense, and I'd encourage others who may be confused on this post to look at the Automotus case study https://encord.com/customers/automotus-customer-story/ which has a great diagram. For those of us with similar needs for annotation and "unit testing," but on text corpuses, I'm aware of https://prodi.gy/ for the annotation…
Comment linking /blog/introducing-vellum-test-suites
on “Launch HN: Encord (YC W21) – Unit testing for computer vision models”Submitted The Six Levels of Agentic Behavior as a story · 2 points
HN discussionI am frequently frustrated when I see people doing "studies" of LLMs where they don't put in the prompt engineering work. I came upon an example [1] recently where someone compared GPT-4 to Gemini Pro and Claude 2. The results hardly matter to me because they didn't put in the prompt work: they didn't give the model space to think (demanding it return only true/false), and they didn't give it higher-level instructio…
Comment linking /blog/best-at-text-classification-gemini-pro-gpt-4-or-claude2
on “OpenAI: Prompt Engineering”Submitted We surveyed 1,285 AI developers on State of AI; only 25% deployed to production as a story · 2 points
HN discussionThey were using techniques like PPO that have a model (like a critic!) that evaluates whether the new model gives accurate responses. With GRPO, the don't have that and instead evaluate the answers based on predefined rules like coherence/ formatting. For example, for math problems, these rules will check if the answer adheres to math principles or logic! I wrote more here (lmk if this is useful): https://www.vellum…
Comment linking /blog/the-training-of-deepseek-r1-and-ways-to-use-it
on “How Deepseek R1 Was Trained”Submitted AskHN: Are you still evaluating alternatives to OpenAI after things stabilized? as a story · 2 points
HN discussionLooks nice, and it's nice that it also supports function call simulation. I've been collecting a list of tools for prompt engineering, I've added Knit now. Newly added: https://promptknit.com/ Newly added: https://github.com/promptfoo/promptfoo Newly added: https://promptlayer.com https://promptable.ai/ https://github.com/ianarawjo/ChainForge https://trywale.com/ https://promptknit.com/ https://promptrefine.com/ htt…
Comment linking /
on “Show HN: Knit – A Better LLM Playground”LLM observability strikes me as an extremely, extremely crowded space. And YC has funded an enormous number of them. What do you think is the key differentiator between you and everyone else? Is vendor lock-in really that huge of an issue? [0] https://hegel-ai.com , https://www.vellum.ai/ , https://www.parea.ai , http://baserun.ai , https://www.trychatter.ai , https://talc.ai , https://github.com/BerriAI/bettertest…
Comment linking /
on “a thread”
