Top all-time · #3502
EleutherAI
Company blog · blog.eleuther.aiai
EleutherAI is the company blog at blog.eleuther.ai, writing about ai. Since 2021, 22 different Hacker News readers have linked it 24 times, most often “EleutherAI: Transformer Math 101”. No feed was found, so read it on the site itself.
Too few sampled comments (7) to say how often HN recommends it.?
Why HN links it
Readers link EleutherAI's blog for technical posts on large language models, including transformer memory estimation, GPT model sizes, its first-year history and the 20B model release.
Summarised by Claude from a sample of 7 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Research notes, announcements, and technical essays from EleutherAI.
Most-linked posts?
- EleutherAI: Transformer Math 1013 people · 3 mentions · first linked 2023-04-19
- Why Release a Large Language Model?2 people · 2 mentions · first linked 2021-06-24
- EleutherAI One Year Retrospective2 people · 2 mentions · first linked 2021-07-08
- Rotary Embeddings: A Relative Revolution2 people · 2 mentions · first linked 2021-04-21
- Announcing GPT-NeoX-20B2 people · 2 mentions · first linked 2022-02-02
What Hacker News says?
Well I can't be positive, but it looks like some of the factors that support a long context length might be set wrong. https://blog.eleuther.ai/yarn/
Comment linking /yarn/
on “QwQ-32B: Embracing the Power of Reinforcement Learning”Submitted Announcing GPT-NeoX-20B as a story · 200 points
HN discussion[Disclaimer: I am an author of the above paper and played a rather minimal role. I am also a prominent member of EleutherAI.] "Instruction-tuning" is clearly in the air. Simultaneous work at Google (released less than two weeks ago) on a model they call FLAN can be found here: https://ai.googleblog.com/2021/10/introducing-flan-more-gene... EleutherAI attempted to do something similar several months ago, but didn't s…
Comment linking /tuning-on-eval-harness/
on “T0* – Series of encoder-decoder models trained on a large set of different tasks”Submitted EleutherAI One Year Retrospective as a story · 142 points
HN discussionIt's fairly simple to estimate ram requirements based on parameter count: https://blog.eleuther.ai/transformer-math/
Comment linking /transformer-math/
on “Ask HN: What's the best self hosted/local alternative to GPT-4?”Submitted Pile-T5 as a story · 59 points
HN discussionI'd recommend reading the blog post about their first year, it's really fun and informative https://blog.eleuther.ai/year-one/
Comment linking /year-one/
on “Connor Leahy on EleutherAI, Replicating GPT-2/GPT-3, AI Risk and Alignment”Submitted Yi-34B, Llama 2, and common practices in LLM training as a story · 41 points
HN discussionnot nano, Neo, and its successor NeoX, one of the largest public GPT models available https://blog.eleuther.ai/announcing-20b/
Comment linking /announcing-20b/
on “EleutherAI announces it has become a non-profit”Submitted EleutherAI announces it has become a non-profit as a story · 12 points
HN discussionOpenAI named their models alphabetically in order of the model size: https://blog.eleuther.ai/gpt3-model-sizes/ Davinci performed similarly to a 175B GPT-3 model and the next smallest model is Curie which corresponds to a 6.7B GPT model. Now, there have been two GPT models fine tuned on Code, namely davinci-codex and cushman-codex. Since cushman != curie, it is presumably based on the GPT-model in between, namely th…
Comment linking /gpt3-model-sizes/
on “Copilot Internals”Submitted Rotary Embeddings: A Relative Revolution as a story · 6 points
HN discussionI clicked because I thought they were defining LLM developer as "someone training LLMs", but instead they define it as "someone integrating LLMs into their application". If you also had the same initial thought as me, this is an excellent article - https://blog.eleuther.ai/transformer-math/ .
Comment linking /transformer-math/
on “Numbers every LLM Developer should know”Submitted The Common Pile v0.1 as a story · 4 points
HN discussionSubmitted The Foundation Model Development Cheatsheet as a story · 2 points
HN discussionSubmitted Llemma: An Open Language Model for Mathematics as a story · 2 points
HN discussionSubmitted Exploratory Analysis of TRLX RLHF Transformers with TransformerLens as a story · 2 points
HN discussionSubmitted Why Release a Large Language Model? as a story · 2 points
HN discussionSubmitted The Practitioner's Guide to the Maximal Update Parameterization as a story · 1 points
HN discussionSubmitted Experiments in Weak-to-Strong Generalization as a story · 1 points
HN discussionSubmitted Minetester: A fully open RL environment built on Minetest as a story · 1 points
HN discussionSubmitted EleutherAI: Transformer Math 101 as a story · 1 points
HN discussionSubmitted Why Release a Large Language Model? as a story · 1 points
HN discussionSubmitted Rotary Embeddings: A Relative Revolution as a story · 1 points
HN discussion
