Top all-time · #18666
Embedded LLM
Company blog · embeddedllm.com/blogaiprogramming
Embedded LLM is the company blog at embeddedllm.com, writing about ai and programming. Since 2023, 4 different Hacker News readers have linked it 5 times, most often “LLM Inference with VLLM and AMD: Achieving LLM Inference Parity with Nvidia”. No feed was found, so read it on the site itself.
Too few sampled comments (3) to say how often HN recommends it.?
Why HN links it
Embedded LLM's posts cover running LLM inference on AMD GPUs, including vLLM on ROCm and GGUF support on Radeon cards. HN commenters link them as sources for AMD inference performance.
Summarised by Claude from a sample of 3 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
The goal was never more AI. The goal is governable AI. TokenVisor gives you visibility, policy, routing, budgets and an audit trail across every model your organisation runs.
Most-linked posts?
- /blog/vllm-now-supports-running-gguf-on-amd-radeon-gpu2 people · 2 mentions · first linked 2024-12-24
- LLM Inference with VLLM and AMD: Achieving LLM Inference Parity with Nvidia2 people · 2 mentions · first linked 2023-10-27
What Hacker News says?
Just an FYI, this is writeup from August 2023 and a lot has changed (for the better!) for RDNA3 AI/ML support. That being said, I did some very recent inference testing on an W7900 (using the same testing methodology used by Embedded LLM's recent post to compare to vLLM's recently added Radeon GGUF support [1]) and MLC continues to perform quite well. On Llama 3.1 8B, MLC's q4f16_1 (4.21MB weights) performed +35% fa…
Comment linking /blog/vllm-now-supports-running-gguf-on-amd-radeon-gpu
on “Making AMD GPUs competitive for LLM inference (2023)”Submitted Liger Kernels Leap CUDA Moat: LinkedIn SOTA Training Kernels on AMD GPU as a story · 6 points
HN discussionAMD GPUs are becoming a serious contender for LLM inference. vLLM is already showing impressive performance on AMD [1], even with consumer-grade Radeon cards (even support GGUF) [2]. This could be a game-changer for folks who want to run LLMs without shelling out for expensive NVIDIA hardware. [1] https://blog.vllm.ai/2024/10/23/vllm-serving-amd.html [2] https://embeddedllm.com/blog/vllm-now-supports-running-gguf-...
Comment linking /blog/vllm-now-supports-running-gguf-on-amd-radeon-gpu
on “Making AMD GPUs competitive for LLM inference (2023)”Submitted LLM Inference with VLLM and AMD: Achieving LLM Inference Parity with Nvidia as a story · 1 points
HN discussionhttps://www.databricks.com/blog/training-llms-scale-amd-mi25... https://embeddedllm.com/blog/vllm_rocm/
Comment linking /blog/vllm_rocm/
on “Azure announces new AI optimized VM series featuring AMD's flagship MI300X GPU”
