Top all-time · #4265
Promptfoo Blog
Company blog · promptfoo.dev/blog · on HN as dangelosaurusaisecurity
Promptfoo Blog is the company blog at promptfoo.dev, writing about ai and security. Since 2023, 18 different Hacker News readers have linked it 46 times, most often “Political-bias benchmark for Grok 4, GPT-4.1, Gemini 2.5 Pro and Claude Opus 4”. It is still writing: the last post was on 2026-03-12, and you can follow it by feed.
Too few sampled comments (12) to say how often HN recommends it.?
Why HN links it
HN readers link the Promptfoo blog for its Grok political-bias analysis and its announcement of joining OpenAI. Other comments point to Promptfoo's docs and GitHub project for LLM testing.
Summarised by Claude from a sample of 12 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
The AI Security Platform that catches vulnerabilities in development. Trusted by 156 of the Fortune 500 and 300,000+ developers worldwide.
Most-linked posts?
- Political-bias benchmark for Grok 4, GPT-4.1, Gemini 2.5 Pro and Claude Opus 45 people · 6 mentions · first linked 2025-07-25
- Promptfoo: Secure your AI from prompt to production3 people · 3 mentions · first linked 2024-05-28
- Questions censored by DeepSeek2 people · 2 mentions · first linked 2025-01-28
- Will agents hack everything?2 people · 2 mentions · first linked 2025-11-15
- Promptfoo Is Joining OpenAI2 people · 2 mentions · first linked 2026-03-09
- How to replicate the Claude Code attack with Promptfoo2 people · 2 mentions · first linked 2025-11-19
- Llama 2 uncensored vs. GPT3.5 benchmarking2 people · 2 mentions · first linked 2023-08-10
What Hacker News says?
Thanks to Meta for their work on safety, particularly Llama Guard. Llama Guard 3 adds defamation, elections, and code interpreter abuse as detection categories. Having run many red teams recently as I build out promptfoo's red teaming featureset [0], I've noticed the Llama models punch above their weight in terms of accuracy when it comes to safety. People hate excessive guardrails and Llama seems to thread the need…
Comment linking /docs/red-team/
on “Open source AI is the path forward”Submitted Questions censored by DeepSeek as a story · 384 points
HN discussionManually, From https://news.ycombinator.com/item?id=40859434 : > E.g promptfoo and chainforge have multi-LLM workflows. > Promptfoo has a YAML configuration for prompts, providers,: https://www.promptfoo.dev/docs/configuration/guide/ openai/evals//docs/build-eval.md: https://github.com/openai/evals/blob/main/docs/build-eval.md From https://news.ycombinator.com/item?id=45267271 ; > API facades like OpenLLM and model…
Comment linking /docs/configuration/guide/
on “Claude Sonnet 4.5”Submitted Show HN: Automated red teaming for your LLM app as a story · 23 points
HN discussionI ran a red team eval on GPT-5.2 within 30 minutes of release: Baseline safety (direct harmful requests): 96% refusal rate With jailbreaking : 22% refusal rate 4,229 probes across 43 risk categories. First critical finding in 5 minutes. Categories with highest failure rates: entity impersonation (100%), graphic content (67%), harassment (67%), disinformation (64%). The safety training works against naive attacks but…
Comment linking /blog/gpt-5.2-trust-safety-assessment/
on “GPT-5.2”Submitted Promptfoo Is Joining OpenAI as a story · 15 points
HN discussionPromptfoo | Multiple Roles | Remote US (HQ: San Mateo, CA) About us: Promptfoo builds the leading open-source framework for LLM security and evaluation. Our tools help over 50,000 developers test and secure AI applications. Backed by a16z and led by YC alumni, we are shaping the future of AI safety. Open Roles: - Staff Engineers - Research Engineers - Developer Relations How to apply: - Try Promptfoo at < https://pr…
Comment linking /
on “Ask HN: Who is hiring? (February 2025)”Submitted Building a Security Scanner for LLM Apps as a story · 7 points
HN discussionPromptfoo | Senior Software Engineer (AI Security) | SF Bay Area or Remote (US) | Full-time | https://promptfoo.dev Promptfoo is building open-source pentesting tools for LLMs. Backed by a $5.3M seed round led by Andreessen Horowitz, we’re looking for senior engineers passionate about AI security to join our growing team. We offer a competitive salary and equity, along with a fully remote, async-first culture. This…
Comment linking /
on “Ask HN: Who is hiring? (September 2024)”Submitted Found a CVSS 10.0 bypass in Hugging Face's model scanner. We open-sourced ours as a story · 6 points
HN discussionPromptfoo | Multiple Roles | Remote US (HQ: San Mateo, CA) We’re building the leading open-source framework for LLM security and evaluation, trusted by 40,000+ developers. Backed by a16z and led by YC alumni, we are shaping the future of AI safety and reliability. Open Roles: - Staff Engineers - Research Engineers - DevRel We value experience with open-source projects and a strong interest in AI/ML evaluation, safet…
Comment linking /
on “Ask HN: Who is hiring? (December 2024)”Submitted How to replicate the Claude Code attack with Promptfoo as a story · 6 points
HN discussionPromptfoo | Software Engineer | SF Bay Area or Remote (US) | Full-time | https://promptfoo.dev Promptfoo recently raised $5M in a seed round led by Andreessen Horowitz to help developers identify and fix vulnerabilities in their AI applications. We're a small, dedicated team passionate about AI security and reliability. Why Promptfoo? - Impactful Work: Join us in creating the first pentesting product specifically ta…
Comment linking /
on “Ask HN: Who is hiring? (August 2024)”Submitted Will agents hack everything? as a story · 6 points
HN discussionPromptfoo | Applied ML + Founding Sales Role | San Mateo, CA or REMOTE (US) | Full-time | https://promptfoo.dev Promptfoo builds open-source pentesting and redteaming tools for LLMs. Our tools are used by over 35,000 developers at companies like Shopify, Amazon, and Anthropic, and OpenAI. We are backed by a16z and we’re hiring: 1. Senior / Staff Applied ML Develop redteaming tools, implement research, and secure LLM…
Comment linking /
on “Ask HN: Who is hiring? (October 2024)”Submitted Political-bias benchmark for Grok 4, GPT-4.1, Gemini 2.5 Pro and Claude Opus 4 as a story · 3 points
HN discussionPromptfoo | Senior/Staff Engineers, Former Technical Founders & Experienced Operators | Remote (US time zones) / Hybrid San Mateo CA | Full-time Promptfoo is the MIT-licensed open-source toolkit 100 k+ developers use to evaluate and secure their LLM apps. We are funded by top investors and operate as a tight, all-senior team of high-agency builders, former founders, and owner-operators. 10+ open roles - Senior Full-…
Comment linking /careers/
on “Ask HN: Who is hiring? (July 2025)”Submitted Llama 2 uncensored vs. GPT3.5 benchmarking as a story · 3 points
HN discussionbraintrust-proxy: https://github.com/braintrustdata/braintrust-proxy LocalAI: https://github.com/mudler/LocalAI E.g promptfoo and chainforge have multi-LLM workflows. Promptfoo has a YAML configuration for prompts, providers,: https://www.promptfoo.dev/docs/configuration/guide/ What is the system prompt, and how does a system prompt also bias an analysis? /? "system prompt" https://hn.algolia.com/?dateRange=all&page…
Comment linking /docs/configuration/guide/
on “Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks”Submitted Will Agents Hack Everything? as a story · 2 points
HN discussionPromptfoo | Senior/Staff Engineers, Security Researchers, GTM & Founding Operators | REMOTE (North America) / Hybrid San Mateo CA | Full-time | https://promptfoo.dev Promptfoo is the MIT-licensed open-source toolkit 125 000+ developers use to evaluate and secure LLM apps. We just closed an $18.4 M Series A led by Insight Partners with participation from a16z and are scaling a small, senior team of high-agency builde…
Comment linking /careers/
on “Ask HN: Who is hiring? (August 2025)”Submitted How to replicate the Claude Code attack with Promptfoo as a story · 2 points
HN discussionI have not heard of the biased prompt and since prompts are constantly refined, how do you know it's still in use? But I found this analysis interest... https://www.promptfoo.dev/blog/grok-4-political-bias/ *Our measurements show that: Grok is more right leaning than most other AIs, but it's still left of center. GPT 4.1 is the most left-leaning AI, both in its responses and in its judgement of others. Surprisingly,…
Comment linking /blog/grok-4-political-bias/
on “Grokipedia is the antithesis of Wikipedia”Submitted Evaluating Political Bias in LLMs as a story · 2 points
HN discussionhttps://www.promptfoo.dev/blog/grok-4-political-bias/ > Grok is more right leaning than most other AIs, but it's still left of center. https://github.com/promptfoo/promptfoo/tree/main/examples/gr... > Universal Left Bias: All major AI models (GPT-4.1, Gemini 2.5 Pro, Claude Opus 4, Grok 4) lean left of center if every AI "leans left" then that should hopefully indicate to you that your notion of "center" is actually…
Comment linking /blog/grok-4-political-bias/
on “Measuring political bias in Claude”Submitted Promptfoo: Secure your AI from prompt to production as a story · 2 points
HN discussionI did similar measurements back in July ( https://www.promptfoo.dev/blog/grok-4-political-bias/ , dataset: https://huggingface.co/datasets/promptfoo/political-question... ). Anthropic's "even-handedness" asks: does the model engage with both sides fairly? My study asked: where does the model actually land when it takes positions? A model can score 95% on even-handedness (engages both sides well) while still taking c…
Comment linking /blog/grok-4-political-bias/
on “Measuring political bias in Claude”Submitted Automated jailbreaking techniques with DALL-E as a story · 2 points
HN discussionThe library supports the model-graded factuality prompt used by OpenAI in their own evals. So, you can do automatic grading if you wish (using GPT 4 by default, or your preferred LLM). Example here: https://promptfoo.dev/docs/guides/factuality-eval
Comment linking /docs/guides/factuality-eval
on “Comparing 60 LLMs with a set of 20 prompts”Submitted Benchmark Command R vs. GPT/Claude on your own data as a story · 2 points
HN discussion
