Top all-time · #13150
Inferless Blog
Company blog · inferless.com/learnaisystems
Inferless Blog is the company blog at inferless.com, writing about ai and systems. Since 2023, 6 different Hacker News readers have linked it 16 times, most often “The state of serverless GPUs”. It is still writing: the last post was on 2025-10-21, and you can follow it by feed.
Too few sampled comments (2) to say how often HN recommends it.?
Why HN links it
HN commenters link Inferless posts as overviews of GPU inference and text-to-speech model comparisons, such as a benchmark run on an NVIDIA L4 with 24GB VRAM. The comments do not judge the writing.
Summarised by Claude from a sample of 2 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Blazing fast serverless GPU inference to deploy ML models. Use Inferless for scalable and effortless custom machine learning model deployment!
Most-linked posts?
- The state of serverless GPUs2 people · 4 mentions · first linked 2023-11-07
- The State of Serverless GPUs2 people · 2 mentions · first linked 2023-04-18
- LLMs Tokens/Second Benchmark ( Mistral, Llama2, Gemma) – Independent Research2 people · 2 mentions · first linked 2024-03-25
What Hacker News says?
You can check out this technical deep dive on Serverless GPUs offerings/Pay-as-you-go way. This includes benchmarks around cold-starts, performance consistency, scalability, and cost-effectiveness for models like Llama2 7Bn & Stable Diffusion across different providers - https://www.inferless.com/learn/the-state-of-serverless-gpus... .Can save months of your time. Do give it a read. P.S: I am from Inferless.
Comment linking /learn/the-state-of-serverless-gpus-part-2
on “Web scraping with GPT-4o: powerful but expensive”Submitted LLMs Tokens/Second Benchmark ( Mistral, Llama2, Gemma) – Independent Research as a story · 2 points
HN discussionIsn't SXM5 higher bandwidth? It's 900 GB/s of bidirectional bandwidth per GPU across 18 NVLink 4 channels. The NVL's are on PCIe 5, and even w/ NVLink only get to 600 GB/s of bandwidth across 3 NVLink bridges (across only pairs of cards)? I haven't done a head to head and I suppose it depends on whether tensor parallelism actually scales linearly or not, but my understanding is since the NVL's are just PCIe/NVLink p…
Comment linking /learn/exploring-llms-speed-benchmarks-independent-analysis
on “AMD's MI300X Outperforms Nvidia's H100 for LLM Inference”Submitted Qwen2-7B-Instruct with TensorRT-LLM: consistently high tokens/SEC as a story · 1 points
HN discussionThis is a good way to do math. But honestly, how many products actually have 100% utilisation. I did some math a few months ago but mostly on the basis of active users, on what would be the % difference if you have 1k to 10K users/mo. You can run this as low as $0.3K/mo on Serverless GPUs and $0.7K/mo on EC2. The pricing is outdated now. Here is the piece - https://www.inferless.com/learn/unraveling-gpu-inference-co…
Comment linking /learn/unraveling-gpu-inference-costs-for-llms-openai-aws-and-inferless
on “Cost of self hosting Llama-3 8B-Instruct”Submitted Fast Cold-starts for Serverless GPU Inference is becoming a reality as a story · 1 points
HN discussionThis is true especially if you are deploying custom or fine-tuned models. Infact, for my company i also ran benchmark tests where we tested cold-starts, performance consistency, scalability, and cost-effectiveness for models like Llama2 7Bn & Stable Diffusion across different providers - https://www.inferless.com/learn/the-state-of-serverless-gpus... Can save months of evaluation time. Do give it a read. P.S: I am f…
Comment linking /learn/the-state-of-serverless-gpus-part-2
on “Show HN: Open-source LLM provider price comparison”Submitted The state of serverless GPUs as a story · 1 points
HN discussionYou can check out this technical deep dive on Serverless GPUs offerings/Pay-as-you-go way. This includes benchmarks around cold-starts, performance consistency, scalability, and cost-effectiveness for models like Llama2 7Bn & Stable Diffusion across different providers - https://www.inferless.com/learn/the-state-of-serverless-gpus ... .Can save months of your time. Do give it a read. P.S: I am from Inferless
Comment linking /learn/the-state-of-serverless-gpus
on “AWS AI Stack – Ready-to-Deploy Serverless AI App on AWS and Bedrock”Submitted Deploying Hugging Face Models on Nvidia Triton Inference Server at Scale as a story · 1 points
HN discussionHere is an overview: https://www.inferless.com/learn/comparing-different-text-to-... Also keep in mind the processing time. The ^ article above used a NVIDIA L4 with 24-GB VRAM. Sopro claims 7.5 second processing time on CPU for 30 seconds of audio! If you want to get real good quality TTS, you should check out elevenlabs.io Different tools for different goals.
Comment linking /learn/comparing-different-text-to-speech---tts--models-part-2
on “Sopro TTS: A 169M model with zero-shot voice cloning that runs on the CPU”Submitted The State of Serverless GPUs as a story · 1 points
HN discussionYou can also find the same if you know the tokens/sec for different input and tokens variation. In case you are interested to see results for speed ( tokens/second) I Ran some tests between LLama2 7Bn, Gemma 7Bn, Mistral 7Bn to compare tokens/second on 6 different libraries with 5 different input tokens range (20 to 5000) and three different output tokens (100,200 and 500) on A100. These are the results : https://in…
Comment linking /learn/exploring-llms-speed-benchmarks-independent-analysis
on “Ask HN: How to easily benchmark LLM without elaborate setup?”Submitted The State of Serverless GPUs as a story · 1 points
HN discussionCheck this out - https://www.inferless.com/learn/the-state-of-serverless-gpus... Btw, I am the founder of Inferless.
Comment linking /learn/the-state-of-serverless-gpus-part-2
on “Ask HN: Rentable cloud GPU provider with credit API?”This is a great list. If you want to read about deploying HF models on triton inference server at scale for inference. Find this great read - https://www.inferless.com/learn/nvidia-triton-inference-infe...
Comment linking /learn/nvidia-triton-inference-inferless
on “LLM Training/Application Frameworks to use and avoid”