Top all-time · #21106
Beam
Company blog · beam.cloud/blogaisystems
Beam is the company blog at beam.cloud, writing about ai and systems. Since 2023, 3 different Hacker News readers have linked it 15 times. No feed was found, so read it on the site itself.
Too few sampled comments (1) to say how often HN recommends it.?
Why HN links it
A GPU compute platform for AI agents and models. The one comment is a user describing a switch from Replicate for faster, more consistent cold starts and model-file caching. It does not discuss the writing.
Summarised by Claude from a sample of 1 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Compute for agents. Bare-metal servers around the world as VMs, sandboxes, task queues, functions and endpoints, billed only while your code runs.
What Hacker News says?
Replicate has really long boot times for custom models - 2/3 minutes if you are lucky and up to 30 minutes if they are having problems. While we loved the dev experience we just couldn’t make it work with frequently switching models / LORA weights. We switched to beam ( https://www.beam.cloud ) and it’s so much better. Their cold start times are consistently small and they provide caching layer for model files i.e v…
Comment linking /
on “Replicate vs. Fly GPU cold-start latency”Submitted The Architecture Behind Lovable and Bolt as a story · 73 points
HN discussionThe major clouds don't support serverless GPU because the architecture is fundamentally different from running CPU workloads. For Lambda specifically, there's no way of running multiple customer workloads on a single GPU with Firecracker. A more general issue is that the workloads that tend to run on GPU are much bigger than a standard Lambda-sized workload (think a 20Gi image with a smorgasbord of ML libraries). I'…
Comment linking /blog/serverless-platform-guide
on “AWS AI Stack – Ready-to-Deploy Serverless AI App on AWS and Bedrock”Submitted Show HN: Open Jev Playground – try all the open source alternatives to Jev as a story · 5 points
HN discussionWhat are you using for K8s autoscaling? We initially tried a few standard K8s scaling mechanisms and found that they didn't work well for GPU workloads. For example, if we were serving a low-RAM Huggingface model on GPU, it wouldn't trigger autoscaling. But since the GPU can only process one request at a time, the system would get bottlenecked while it waited to process requests one-by-one. We wrote a bit about this…
Comment linking /blog/serverless-autoscaling
on “ChatGPT Is Down”Submitted Ngrok for Python Functions as a story · 2 points
HN discussionServerless only works if the cold boot is fast. For context, my company runs a serverless cloud GPU product called https://beam.cloud , which we've optimized for fast cold start. We see Whisper in production cold start in under 10s (across model sizes). A lot of our users are running semi-real time STT, and this seems to be working well for them.
Comment linking /
on “Workers AI: serverless GPU-powered inference on Cloudflare’s global network”Submitted The Unsung Hero: Microsoft Visual Basic for Internal Tools as a story · 2 points
HN discussionYou could easily host your model on https://beam.cloud (I'm a founder). You just add a decorator to your existing Python code: from beam import App, Runtime app = App(name="gpu-app", runtime=Runtime(gpu="T4")) @app.rest_api() def inference(): print("This is running on a GPU") Then run beam deploy {your-app}.py and boom, it's running on the cloud
Comment linking /
on “Why host your own LLM?”Submitted What Is a Container, Really? Five Years of GPU Infrastructure as a story · 1 points
HN discussionYou could try https://beam.cloud (disclaimer, I’m the co-founder) You can run popular models on remote GPUs from your laptop with two commands. For example: $ beam create-app stable-diffusion-gpu $ beam start app.py
Comment linking /
on “Ask HN: What is the best place to play around with AI?”Submitted What Is a Container, Really? Five Years of GPU Infrastructure as a story · 1 points
HN discussionYou should checkout https://beam.cloud (I'm the founder), it'll give you access to plenty of cloud GPU resources for training or inference. Right now it's pretty hard to get GPU quota on AWS/GCP, so hopefully this is useful for you.
Comment linking /
on “Ask HN: Where to find an env with GPU for model training?”Submitted Building a Modern Serverless Cloud for Bioinformatics as a story · 1 points
HN discussionCloudflare AI and Replicate are great for running off-the-shelf models, but anything custom is going to incur a 10+ minute cold start. For running custom fine-tuned models on serverless, you could look into https://beam.cloud which is optimized for serving custom models with extremely fast cold start (I'm a little biased since I work there, but the numbers don't lie)
Comment linking /
on “Cloudflare launches new AI tools to help customers deploy and run models”