Top all-time · #1909
Non_Int
Personal blog · nonint.comaiprogramming
Non_Int is a personal blog at nonint.com about ai and programming. Since 2022, 41 different Hacker News readers have linked it 53 times, most often “The "it" in AI models is the dataset”. No feed was found, so read it on the site itself.
Writing about software, machine learning, and generative models.
Most-linked posts?
- The "it" in AI models is the dataset14 people · 17 mentions · first linked 2023-06-12
- My Deep Learning Rig5 people · 5 mentions · first linked 2023-01-09
- Tortoise Text-to-Speech5 people · 5 mentions · first linked 2022-04-27
- General Intelligence (2024)4 people · 4 mentions · first linked 2024-06-11
- GPT might be an information virus (2023)3 people · 4 mentions · first linked 2023-03-19
- Is the Reversal Curse in LLMs a generalization problem?2 people · 2 mentions · first linked 2023-11-23
- Learned Structures2 people · 2 mentions · first linked 2024-04-17
- Grokking Diffusion Models2 people · 2 mentions · first linked 2022-10-31
- Friends don’t let friends train small diffusion models2 people · 2 mentions · first linked 2022-05-24
What Hacker News says?
The developer behind Tortoise is experimenting with using diffusion for music generation: https://nonint.com/2022/05/04/friends-dont-let-friends-train...
Comment linking /2022/05/04/friends-dont-let-friends-train-small-diffusion-models/
on “Imagen, a text-to-image diffusion model”Submitted GPT might be an information virus (2023) as a story · 124 points
HN discussionDon't end to end trained models already do this to some extent? Like raising the pitch towards a question mark, like a human would. TortoiseTTS has a few examples under prompt engineering on their demo site: https://nonint.com/static/tortoise_v2_examples.html
Comment linking /static/tortoise_v2_examples.html
on “Generate audiobooks from E-books with Kokoro-82M”Submitted The "it" in AI models is the dataset as a story · 108 points
HN discussionI dare say RAG with vector DBs is underwhelming because embeddings are not underrated but appropriately rated, and will not give you relevant info in every case. In fact, the way LLMs retrieve info internally [0] already works along the same principle and is a large factor in their unreliability. [0] https://nonint.com/2023/10/18/is-the-reversal-curse-a-genera...
Comment linking /2023/10/18/is-the-reversal-curse-a-generalization-problem/
on “Embeddings are underrated”Submitted Friends don’t let friends train small diffusion models as a story · 92 points
HN discussionMost of these are just an EPYC server platform, some cursed risers and multiple PSUs (though cryptominer server PSU adapters are probably better). See https://nonint.com/2022/05/30/my-deep-learning-rig/ and https://www.mov-axbx.com/wopr/wopr_concept.html .
Comment linking /2022/05/30/my-deep-learning-rig/
on “Making AMD GPUs competitive for LLM inference (2023)”Submitted Google/Rulesofthumb as a story · 53 points
HN discussionhttps://nonint.com/2023/11/05/compute-multipliers/
Comment linking /2023/11/05/compute-multipliers/
on “Reflections on our Responsible Scaling Policy”Submitted The Paradigm as a story · 9 points
HN discussionIn the spirit of hacker news, one can also build a rig such as this one: https://nonint.com/2022/05/30/my-deep-learning-rig/ and land a job at OpenAI as a side effect. Of all places, I wasn't expecting such pushback here
Comment linking /2022/05/30/my-deep-learning-rig/
on “New GeForce RTX 4080 Super, RTX 4070 Ti Super, RTX 4070 Super”Submitted General Intelligence (2024) as a story · 4 points
HN discussionJames Betker in the tortoise-tts repo, which is similar, says he spent $15k for his home rig. I'm not finding right now how long it took to train the tortoise model but feel like I read him say weeks/months somewhere. Obvs all kinds of variations depending on coding efficiency and dataset size, but another datapoint https://nonint.com/2022/05/30/my-deep-learning-rig/ https://github.com/neonbjb/tortoise-tts
Comment linking /2022/05/30/my-deep-learning-rig/
on “Microsoft’s new text-to-speech model can duplicate anyone's voice in 3 seconds”Submitted Grokking Diffusion Models as a story · 4 points
HN discussioni suppose its not news, but there was a long thread about it but it came from this post from june: https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dat...
Comment linking /2023/06/10/the-it-in-ai-models-is-the-dataset/
on “Understanding Deep Learning”Submitted The "it" in AI models is the dataset as a story · 3 points
HN discussionCheck out https://nonint.com/static/tortoise_v2_examples.html it's also non-realtime, provides great quality and it's open source.
Comment linking /static/tortoise_v2_examples.html
on “Neural Codec Language Models Are Zero-Shot Text to Speech Synthesizers”Submitted General Intelligence (2024) as a story · 3 points
HN discussionFair push back, but I do think the LSTM vs Transformers point kinda supports my position in the limit, not refutes. Once the compute bottleneck is removed, LSTMs scale favourably. https://arxiv.org/pdf/2510.02228 (I believe there's similar work done on vanilla LSTMs, but I'd have to go digging) So the bottleneck was compute. Which is compatible with 'data or compute'. But to accept your point, at the time the algoro…
Comment linking /2023/06/10/the-it-in-ai-models-is-the-dataset/
on “I tried Karpathy's Autoresearch on an old research project”Submitted GPT might be an information virus as a story · 3 points
HN discussionIt's going to get worse. Here's a great primer on the problem we're facing by a Google -> OpenAI engineer. "I think that being able to automatically produce arbitrary content that is indistinguishable from human-generated content at scale is the death knell of the web as we know it." https://nonint.com/2023/03/09/gpt-might-be-an-information-vi...
Comment linking /2023/03/09/gpt-might-be-an-information-virus/
on “Google has sent internet into 'spiral of decline', claims DeepMind co-founder”Submitted Learned Structures as a story · 2 points
HN discussionANNs aren't top down either though. The architecture isn't as important as people think. https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dat... Even then, the architectures we use are essentially stumbled on. This is alchemy, not modern chemistry.
Comment linking /2023/06/10/the-it-in-ai-models-is-the-dataset/
on “Building AI Without a Neural Network”Submitted Beating Arc the Hard Way as a story · 2 points
HN discussionThe best is probably tortoise but you have to run it yourself https://github.com/neonbjb/tortoise-tts here are some demos https://nonint.com/static/tortoise_v2_examples.html
Comment linking /static/tortoise_v2_examples.html
on “Ask HN: Are there any good open source Text-to-Speech tools?”Submitted The "it" in AI models is the dataset as a story · 2 points
HN discussion>ChatGPT has a model of all the text information on the internet, but it remains to be seen what the hard limits of this model are. Before training is complete and loss is maxed, there will be limits on what the "learned so far" model can do that say absolutely nothing about the limits of a perfect(or very close to it) model. It really looks like anything will converge with enough compute. I don't think architecture…
Comment linking /2023/06/10/the-it-in-ai-models-is-the-dataset/
on “I disagree with Geoff Hinton regarding "glorified autocomplete"”Submitted Learned Structures as a story · 2 points
HN discussionhttps://nonint.com/2023/07/05/on-the-efficiency-of-human-int... https://nonint.com/2022/05/30/my-deep-learning-rig/
Comment linking /2022/05/30/my-deep-learning-rig/
on “Ask HN: Can you help me find a blog that has been linked here before?”Submitted Research Code as a story · 2 points
HN discussion
