Top all-time · #130
Databricks
Company blog · databricks.com/blogdataaiprogramming
Databricks is the company blog at databricks.com, writing about data, ai and programming. Since 2013, 305 different Hacker News readers have linked it 531 times, most often “Free Dolly: Introducing the First Truly Open Instruction-Tuned LLM”. It is still writing: the last post was on 2026-10-31, and you can follow it by feed.
Rarely recommended · 2 of 40 sampled comments recommend it?
Why HN links it
Linked mostly as a reference for Apache Spark internals, DBRX and other LLM releases, training LLMs on AMD GPUs, and glossary pages. Few comments judge the writing itself.
- Recommended for · 2 comments
- Commenters call the Databricks posts on training LLMs on AMD GPUs good links and point readers to them.
Summarised by Claude from a sample of 40 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Databricks offers a unified platform for data, analytics and AI. Build better AI with a data-centric approach. Simplify ETL, data warehousing, governance and AI on the Data + AI Platform.
Most-linked posts?
- Free Dolly: Introducing the First Truly Open Instruction-Tuned LLM8 people · 8 mentions · first linked 2023-04-12
- Benchmarking Coding Agents on Databricks' Multi-Million Line Codebase7 people · 8 mentions · first linked 2026-07-08
- Omnigent: A Meta-Harness to Combine, Control and Share Your Agents6 people · 6 mentions · first linked 2026-06-13
- LLM Inference Performance Engineering: Best Practices5 people · 6 mentions · first linked 2023-10-13
- Spark as a Compiler: Joining a Billion Rows per Second on a Laptop5 people · 5 mentions · first linked 2016-05-23
- Speedy Scala Builds with Bazel at Databricks4 people · 9 mentions · first linked 2019-02-27
- Training LLMs at Scale with AMD MI250 GPUs4 people · 8 mentions · first linked 2023-10-31
- Introducing Spark Datasets4 people · 4 mentions · first linked 2016-01-04
- Deep Dive into Apache Spark SQL’s Catalyst Optimizer4 people · 4 mentions · first linked 2015-04-13
- DBRX: 135B MoE LLM trained and open sourced by Databricks4 people · 4 mentions · first linked 2024-03-27
What Hacker News says?
It's not always 7-8. From Databricks: "DBRX has 16 experts and chooses 4, while Mixtral and Grok-1 have 8 experts and choose 2. This provides 65x more possible combinations of experts and we found that this improves model quality. DBRX uses rotary position encodings (RoPE), gated linear units (GLU), and grouped query attention (GQA). It uses the GPT-4 tokenizer as provided in the tiktoken repository. We made these c…
Comment linking /blog/introducing-dbrx-new-state-art-open-llm
on “Meta Llama 3”Submitted DBRX: A new open LLM as a story · 866 points
HN discussionHere's an older study that includes Claude 3.5: https://www.databricks.com/blog/long-context-rag-capabilitie... ?
Comment linking /blog/long-context-rag-capabilities-openai-o1-and-google-gemini
on “Claude 4”Submitted Databricks acquires Neon as a story · 389 points
HN discussionYes, there are several. See this blog post from Databricks describing the landscape of LLMs trained on AMD hardware for example: https://www.databricks.com/blog/training-llms-scale-amd-mi25... The most interesting one IMO is OLMo from AI2, which is truly open. You can read their blog post about it ( https://blog.allenai.org/hello-olmo-a-truly-open-llm-43f7e73... ) but basically it is open everything - they released…
Comment linking /blog/training-llms-scale-amd-mi250-gpus
on “Llm.c – LLM training in simple, pure C/CUDA”Submitted Spark 2.0 Technical Preview as a story · 259 points
HN discussionSo basically you're talking about the design and limitations of https://databricks.com/blog/2015/04/13/deep-dive-into-spark-... I take it? The implication of this work I thought is that it could be further expanded to other languages and DSLs. However, Spark's SQL generator still being very JVM-dependent and this optimizer generating bytecode kind of makes it pretty specific to JVM-supporting languages only. This wo…
Comment linking /blog/2015/04/13/deep-dive-into-spark-sqls-catalyst-optimizer.html
on “Microsoft Launches Its .NET Distribution for Linux and Mac”Submitted Scala at scale at Databricks as a story · 251 points
HN discussionYeah, I did a presentation at the DC Area Apache Spark meetup... slides: http://www.slideshare.net/RichardSeymour3/2015-0224-washingt... associated blog post: https://www.endgame.com/blog/streaming-data-processing-pyspa... I've done a bit of scala spark as well, and my initial thought was prototype in pyspark and then rewrite in scala if necessary. Just this week DataBricks announced they are working on changing som…
Comment linking /blog/2015/04/28/project-tungsten-bringing-spark-closer-to-bare-metal.html
on “Microsoft Launches Its .NET Distribution for Linux and Mac”Submitted Spark as a Compiler: Joining a Billion Rows per Second on a Laptop as a story · 251 points
HN discussionYes, there are using different wording in different articles: https://www.databricks.com/blog/introducing-dbrx-new-state-a... The only mention of open source is: > DBRX outperforms established open source models https://www.databricks.com/blog/announcing-dbrx-new-standard... Open source is mentioned 10+ times > Databricks is the only end-to-end platform to build high quality AI applications, and the release today of…
Comment linking /blog/announcing-dbrx-new-standard-efficient-open-source-customizable-llms
on “DBRX: A new open LLM”Submitted Deep Learning with Spark and TensorFlow as a story · 228 points
HN discussionIts literally described as open source all over. https://www.databricks.com/blog/announcing-dbrx-new-standard... Its even implied in comparisons everywhere: > Figure 1: DBRX outperforms established open source models on language understanding (MMLU), Programming (HumanEval), and Math (GSM8K). > The aforementioned three reasons lead us to believe that open source LLMs will continue gaining momentum. In particular, we…
Comment linking /blog/announcing-dbrx-new-standard-efficient-open-source-customizable-llms
on “DBRX: A new open LLM”Submitted Databricks response to Snowflake's accusation of lacking integrity as a story · 217 points
HN discussionThey do have a solid focus on doing so, it’s just not exclusive. https://www.databricks.com/product/machine-learning/large-la...
Comment linking /product/machine-learning/large-language-models-oss-guidance
on “DBRX: A new open LLM”Submitted Databricks is raising a Series K Investment at >$100B valuation as a story · 182 points
HN discussionMosaic AI Training ( https://www.databricks.com/product/machine-learning/mosaic-a... ) as it's mentioned in the announcement blog ( https://www.databricks.com/blog/announcing-dbrx-new-standard... - it's a bit less technical)
Comment linking /blog/announcing-dbrx-new-standard-efficient-open-source-customizable-llms
on “DBRX: A new open LLM”Submitted Announcing Apache Spark 1.4 as a story · 154 points
HN discussionIt's similar to llama2. > If, on the DBRX version release date, the monthly active users of the products > or services made available by or for Licensee, or Licensee’s affiliates, is > greater than 700 million monthly active users in the preceding calendar > month, you must request a license from Databricks, which we may grant to you > in our sole discretion, and you are not authorized to exercise any of the > right…
Comment linking /legal/open-model-license
on “DBRX: A new open LLM”Submitted Spark Breaks Previous Large-Scale Sort Record as a story · 142 points
HN discussionWhile pretraining a decent-sized LLM from scratch is not financially feasible for the average person, it is very much feasible for the average YC/VC backed startup (ignoring the fact that it's almost always easier to just use something like Mixtral or LLaMa 2 and fine-tune as necessary). >Introducing MPT-7B, the first entry in our MosaicML Foundation Series. MPT-7B is a transformer trained from scratch on 1T tokens…
Comment linking /blog/mpt-7b
on “Implementing a ChatGPT-like LLM from scratch, step by step”Submitted Introducing DataFrames in Spark for Large Scale Data Science as a story · 138 points
HN discussionRemember: inference is memory bound. https://www.databricks.com/blog/llm-inference-performance-en...
Comment linking /blog/llm-inference-performance-engineering-best-practices
on “Nvidia's Project Digits is a 'personal AI supercomputer'”Submitted MLflow: An Open Source Machine Learning Platform as a story · 135 points
HN discussionSpark 1.1 switched default sorting algorithm from quicksort to TimSort as well in both the map and reduce phases- https://databricks.com/blog/2014/10/10/spark-petabyte-sort.h...
Comment linking /blog/2014/10/10/spark-petabyte-sort.html
on “Proving that Android’s, Java’s and Python’s sorting algorithm is broken”Submitted Intelligent Kubernetes Load Balancing at Databricks as a story · 130 points
HN discussionSpark 2.0: Rearchitecting Spark for Mobile Platforms https://databricks.com/blog/2015/04/01/spark-2-rearchitectin... This is probably among the most technical, nerdy April fool's.
Comment linking /blog/2015/04/01/spark-2-rearchitecting-spark-for-mobile.html
on “List of April Fools' Day Announcements”Submitted Announcing Spark 1.6 as a story · 104 points
HN discussionDatabricks -- San Francisco -- https://databricks.com * Software Engineer (ONSITE) * Software Engineer Intern (ONSITE) * Product Manager (ONSITE) Databricks was founded in 2013 by the team that started Apache Spark, meaning you might not only use Spark but also get to work on it :). We provide a cloud data processing platform based on Apache Spark used by customers including top 5 banks, healthcare and media compani…
Comment linking /company/careers
on “Ask HN: Who is hiring? (July 2016)”Submitted Announcing Spark 1.3 as a story · 103 points
HN discussion
