Top all-time · #15076
Kog Labs
Company blog · blog.kog.aiaisystems
Kog Labs is the company blog at blog.kog.ai, writing about ai and systems. Since 2026, 5 different Hacker News readers have linked it 8 times, most often “Real-time LLM Inference on Standard GPUs (3k tokens/s per request)”. It is still writing: the last post was on 2026-06-25, and you can follow it by feed.
Too few sampled comments (3) to say how often HN recommends it.?
Why HN links it
Kog Labs posts on real-time LLM inference on standard GPUs, including a monokernel GPU deep dive and delayed tensor parallelism research. Commenters debated the fairness of its speed benchmark.
Summarised by Claude from a sample of 3 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Kog AI Tech and Research Blog
Most-linked posts?
- Real-time LLM Inference on Standard GPUs (3k tokens/s per request)3 people · 4 mentions · first linked 2026-05-28
What Hacker News says?
Follow-up reading the most technical and research people here: Monokernel deep dive (GPU Engineering): http://blog.kog.ai/building-a-single-kernel-latency-optimize... Delayed Tensor Parallelism (research): http://blog.kog.ai/delayed-tensor-parallelism-for-faster-tra... To try the speed on the playground: http://playground.kog.ai
Comment linking /delayed-tensor-parallelism-for-faster-transformer-inference
on “Real-time LLM Inference on Standard GPUs: 3k tokens/s per request”Submitted Real-time LLM Inference on Standard GPUs: 3k tokens/s per request as a story · 6 points
HN discussionGreat points. We strived to be fair as possible in the benchmark, but it's indeed not perfect. Taalas should have been added in the dedicated hardware section, even though they use 3-bit quantization when we are on FP16 (to be fair in both directions) and they burn the model directly on the card. Our tech preview is about the speed (hence the small dense model, it was easier to implement). The math checks out though…
Comment linking /real-time-llm-inference-on-standard-gpus-3-000-tokens-s-per-request/
on “Real-time LLM Inference on Standard GPUs: 3k tokens/s per request”Submitted Real-time LLM Inference on Standard GPUs (3k tokens/s per request) as a story · 6 points
HN discussionFair point - this tech preview is about the speed (hence the small dense model, it was easier to implement). The math checks out though to allow support for large frontier MoE models at similar speeds. At batch size 1, GPT-OSS-120B has 5.1B active parameters - in FP8, it's in the same size ballpark than our 2B model in FP16 (5.1 GB vs 4GB). DeepSeek V4 Flash has 13B in mixed FP4/FP8. Check out the math at the end of…
Comment linking /real-time-llm-inference-on-standard-gpus-3-000-tokens-s-per-request/
on “Real-time LLM Inference on Standard GPUs: 3k tokens/s per request”Submitted Kog Laneformer 2B: The Latency-First Model Behind Kog Inference Engine as a story · 1 points
HN discussionSubmitted Delayed Tensor Parallelism for Faster Transformer Inference as a story · 1 points
HN discussion