Top all-time · #14619
Prem Blog
Company blog · blog.premai.ioaidata
Prem Blog is the company blog at blog.premai.io, writing about ai and data. Since 2024, 5 different Hacker News readers have linked it 8 times, most often “Benchmark popular LLM inference engines”. It is still writing: the last post was on 2026-11-02, and you can follow it by feed.
Prem is an applied AI research lab dedicated to creating a future where everyone can access sovereign, private, and personalized AI.
Most-linked posts?
- Benchmark popular LLM inference engines3 people · 3 mentions · first linked 2024-05-14
What Hacker News says?
Yeah, that makes sense. Here's another benchmark that shows trt being about 2-3x faster, 2x on fp16 and 3x on int4. https://blog.premai.io/prem-benchmarks/ Though to be fair it's 512 tokens and not 4k tokens. Would love to see more public testing done for all the different variations.
Comment linking /prem-benchmarks/
on “AMD MI300x GPUs with GEMM tuning improves throughput and latency by up to 7.2x”Submitted Small Models, Big Wins: Agentic AI in Enterprise Explained as a story · 2 points
HN discussionCheckout Benchmarks v2 https://github.com/premAI-io/benchmarks It benchmarks Llama 2 and Mistral v0.1 across all the popular inference engines out there, this includes TensorRT LLM, vLLM, Llama CPP, CTranslate2, DeepSpeed etc etc. Total 13 + inference engines and still counting. For each engine benchmark is also done across four precisions fp32/16 and int8/4. Benchmarking is done on the following parameters throughp…
Comment linking /prem-benchmarks/
on “Benchmarked Llama2 and mistral across popular inference engines and precisions”Submitted Small Models, Big Wins: Agentic AI in Enterprise Explained as a story · 2 points
HN discussionSubmitted Small Language Models (SLMs) for Efficient Edge Deployment as a story · 2 points
HN discussionSubmitted Benchmark popular LLM inference engines as a story · 2 points
HN discussionSubmitted Small Models, Big Wins: Agentic AI in Enterprise Explained as a story · 1 points
HN discussionSubmitted SLM vs. LoRA LLM: Edge Deployment and Fine-Tuning Compared as a story · 1 points
HN discussion