Top all-time · #7476
Kai Greshake
Personal blog · kai-greshake.de/postsaisecurity
Kai Greshake writes Kai's Blog, a personal blog at kai-greshake.de about ai and security. Since 2023, 11 different Hacker News readers have linked it 22 times, most often “In Escalating Order of Stupidity”. The last post was in 2023, but the archive is still widely read.
Too few sampled comments (9) to say how often HN recommends it.?
Why HN links it
HN readers link Kai Greshake's posts on prompt injection against LLM-integrated applications, such as Inject my PDF and the escalating-order-of-stupidity article, as examples of real-world risks.
Summarised by Claude from a sample of 9 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Personal blog & homepage of Kai Greshake.
Most-linked posts?
- In Escalating Order of Stupidity5 people · 7 mentions · first linked 2023-05-12
- Inject My PDF: Prompt Injection for Your Resume5 people · 5 mentions · first linked 2023-05-16
- We Broke LLMs: Indirect Prompt Injection3 people · 4 mentions · first linked 2023-04-28
- Invisible Indirect Injection: A Puzzle for ChatGPT2 people · 2 mentions · first linked 2023-06-02
What Hacker News says?
> Since everything from RAG runs through the prompt, unintended prompt-induced behavior is still an issue, even if its not an information-leak issue It also doesn't completely safeguard a system against attacks. See https://kai-greshake.de/posts/inject-my-pdf/ as an example of how information poisoning can be a problem even if there's no risk of exfiltration and even if the data is already public. I have seen debate…
Comment linking /posts/inject-my-pdf/
on “Purple Llama by Meta AI”Submitted We Broke LLMs: Indirect Prompt Injection as a story · 4 points
HN discussionAgreed, and notably, the people with safety concerns are already regularly "losing" to product designers who want more capabilities. Wuzzie's blog ( https://embracethered.com/blog/ ) has a number of examples of data exfiltration that would be largely prevented by merely sanitizing Markdown output and refusing to auto-fetch external resources like images in that Markdown output. In some cases, companies have been con…
Comment linking /posts/in-escalating-order-of-stupidity/
on “Purple Llama by Meta AI”Submitted Unreal Computing as a story · 2 points
HN discussionOh yeah! I totally forgot about Intel's on-chip AI acceleration[1]. I'm not sure how well it works, but I know that it exists and nothing major is currently taking advantage of it AFAIK. Local Copilot sounds interesting, but I'd imagine it would be akin to the old Siri on iPhones where its use would be limited to controlling on-device settings. I've heard and read quite a lot about the dangers of that, though: with…
Comment linking /posts/llm-malware/
on “How a Windows shake-up could position Microsoft to capitalize on AI PCs”Submitted Invisible Indirect Injection: A Puzzle for ChatGPT as a story · 2 points
HN discussionhttps://kai-greshake.de/posts/puzzle-22745/
Comment linking /posts/puzzle-22745/
on “ChatGPT Is Drunk: Hidden Prompt Injection with Browsing”Submitted In Escalating Order of Stupidity as a story · 2 points
HN discussionIf the only thing you're building is a chat app, and the only thing you're worried about is it swearing at the user, then sure, GPT is great for that. If you're building a Twitch bot, if you're building this into a game or making a quick display or something, then yeah, go wild. But people are wiring GPT up to real-world applications beyond just content generation. Summarizing articles, invoking APIs, managing event…
Comment linking /posts/in-escalating-order-of-stupidity/
on “Prompt injection explained, with video, slides, and a transcript”Submitted We Broke LLMs: Indirect Prompt Injection as a story · 2 points
HN discussionI just published a blog post showing that that is not what is happening. Companies are plugging LLMs into absolutely anything, including defense/threat intelligence/cybersecurity/legal etc. applications: https://kai-greshake.de/posts/in-escalating-order-of-stupidi...
Comment linking /posts/in-escalating-order-of-stupidity/
on “Prompt injection explained, with video, slides, and a transcript”Submitted Inject My PDF: Prompt Injection for Your Resume as a story · 1 points
HN discussionIt's also under the control of anything that it finds in its current "context window" that it decides to act upon, not just Microsoft... https://greshake.github.io https://kai-greshake.de/posts/in-escalating-order-of-stupidi...
Comment linking /posts/in-escalating-order-of-stupidity/
on “Bringing the power of AI to Windows 11”Yep exactly that, it worked also Inject my PDF https://kai-greshake.de/posts/inject-my-pdf/
Comment linking /posts/inject-my-pdf/
on “Texas professor fails entire class from graduating- claiming they used ChatGTP”You're missing the more important vector for prompt injection: Indirect injection through the "search engine context". It's not just a matter of blocking bad user questions to fend off reputational harms. See also my work on https://kai-greshake.de/
Comment linking /
on “Show HN: GalenAI – An AI Powered Search Engine for Clinicians”Check out my blog where I show even more up-to-date techniques and the insane ways vulnerable applications are being deployed: https://kai-greshake.de/ Here I go through all of the unsafe products (including military LLMs): https://kai-greshake.de/posts/in-escalating-order-of-stupidi... Here you can add prompt injections to your resume for free to get your dream job: https://kai-greshake.de/posts/inject-my-pdf/
Comment linking /posts/inject-my-pdf/
on “Compromising LLM-Integrated Applications with Indirect Prompt Injection”I've written about this extensively. My latest article goes into the consequences. How about going from Prompt Injection to airstrike? https://kai-greshake.de/posts/in-escalating-order-of-stupidi...
Comment linking /posts/in-escalating-order-of-stupidity/
on “Delimiters won’t save you from prompt injection”Also check out a blogpost on the same subject: https://kai-greshake.de/posts/llm-malware/
Comment linking /posts/llm-malware/
on “Compromising Real LLM-Integrated Applications with Indirect Prompt Injection”There's also this which uses LLM prompt injection -- https://kai-greshake.de/posts/inject-my-pdf/ -- which is funnier and more absurd. No clue if it works in the wild. Maybe? Discussed here: https://news.ycombinator.com/item?id=35993498
Comment linking /posts/inject-my-pdf/
on “Job applicants are battling AI résumé filters with ‘white fonting’”Don't underestimate the absolutely silly ways people will try integrating LLMs into existing applications. As with previous hype cycles, this has the potential to be a real nightmare. Edit: I just spotted this article about vulnerable uses of LLMs that are already deployed - https://kai-greshake.de/posts/in-escalating-order-of-stupidi...
Comment linking /posts/in-escalating-order-of-stupidity/
on a deleted thread
