Top all-time · #8538
Cloud infrastructure for machine learning at scale
Company blog · cortex.dev/postaiprogramming
Cloud infrastructure for machine learning at scale is the company blog at cortex.dev, writing about ai and programming. Since 2019, 9 different Hacker News readers have linked it 22 times, most often “Lessons Learned Building an Open Source MLOps Platform”. No feed was found, so read it on the site itself.
Too few sampled comments (7) to say how often HN recommends it.?
Why HN links it
HN readers link cortex.dev for Cortex, an open-source tool for deploying ML models as HTTP endpoints on AWS, with threads on its Kubernetes design, spot-instance cost savings, and its Go CLI.
Summarised by Claude from a sample of 7 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Deploy, manage, and scale machine learning models in production.
Most-linked posts?
- Lessons Learned Building an Open Source MLOps Platform2 people · 3 mentions · first linked 2020-11-09
- Cortex: Model Serving for Developers2 people · 2 mentions · first linked 2020-10-01
- /2 people · 2 mentions · first linked 2020-04-06
What Hacker News says?
That makes sense. Programmatically updating cortex.yaml is a common use case especially when you're thinking about continuous deployment. We also have a Python client which can replace the cortex.yaml file ( https://www.cortex.dev/deployments/python-client ).
Comment linking /deployments/python-client
on “Ask HN: What is your ML stack like?”Submitted Netflix's Metaflow: Reproducible machine learning pipelines as a story · 254 points
HN discussionHey Aaron, I work on Cortex which is a tool for continuously deploying models as HTTP endpoints on AWS. Under the hood we use Kubernetes instead of Lambda to avoid cold starts, enable more flexibility with customizing compute and memory usage (e.g. running inference on GPUs), and support spot instances. Could you clarify your comment regarding editing of config files? Is it still a problem if the configuration is de…
Comment linking /
on “Ask HN: What is your ML stack like?”Submitted Why we use YAML, not notebooks, for machine learning as a story · 4 points
HN discussionWe thought similarly to you about the relative ease of "pip install"—which is why we originally wrote the CLI in Python. Pretty immediately, however, we heard back from users who experienced friction. With the Go binary, we're able to share a one line bash command that users on Mac/Linux (Windows coming soon) can run to install the Cortex CLI, which removed the need for us to instruct users on how to configure their…
Comment linking /install
on “Why we’re writing machine learning infrastructure in Go, not Python”Submitted Lessons Learned Building an Open Source MLOps Platform as a story · 3 points
HN discussionWhat does this offer that cortex[1] does not? With cortex supporting scale to zero it costs me less than $1/mo to idle and autoscales to a cap only when I need it even using spot instances when available if it is configured. Disclaimer, not affiliated just enjoy the entire experience of using cortex. We are moving to KubeFlow because we need scale and more features but cortex is great for what it does. [1]: https://…
Comment linking /
on “Show HN: Deploy ML Models on a Budget”Submitted Building Better Batch Inference as a story · 2 points
HN discussionIn general, Cortex will be significantly cheaper because you're only paying AWS for EC2 (the bulk of the bill) and the other AWS services used (a much smaller portion of the bill). With SageMaker, you're paying the EC2 bill plus a ~40% premium. To keep the AWS bill as low as possible, Cortex supports inference on spot instances, which are unused instances that AWS sells at a steep (as in 90%) discount. The drawback…
Comment linking /cluster-management/spot-instances
on “Show HN: Cortex – Open-source alternative to SageMaker for model serving”Submitted Lessons Learned Building an Open Source MLOps Platform as a story · 2 points
HN discussionHi, I'm one of the maintainers of Cortex >> how does your system work in the IRIS or other examples you have? It looks like it hides under the hood a few things compared to this implementation - type of the model (tensorflow) and the logic that fetches it from S3, right? That's exactly right. Cortex has three runtimes: The Predictor runtime, which is used in this post, can run arbitrary Python. There is an optional…
Comment linking /deployments/onnx
on “An Open Source Stack for Managing and Deploying Models”Submitted Lessons learned building an open source MLOps platform as a story · 1 points
HN discussioncould check out: https://www.cortex.dev/
Comment linking /
on “What's the biggest challenge in putting AI into production?”Submitted Machine learning isn't expensive anymore as a story · 1 points
HN discussionSubmitted How to serve 1k models on one CPU with TensorFlow Serving as a story · 1 points
HN discussionSubmitted Why we’re writing machine learning infrastructure in Go, not Python as a story · 1 points
HN discussionSubmitted How to deploy PyTorch Lightning models to production as a story · 1 points
HN discussionSubmitted Cortex: Open-source serverless machine learning as a story · 1 points
HN discussionSubmitted Cortex: Model Serving for Developers as a story · 1 points
HN discussionSubmitted Cortex – Turn Your Tf.Estimators into a JSON API as a story · 1 points
HN discussion
