Top all-time · #1672
Alex Irpan
Personal blog · alexirpan.comaiprogramming
Alex Irpan writes Sorta Insightful, a personal blog at alexirpan.com about ai and programming. Since 2016, 46 different Hacker News readers have linked it 53 times, most often “Deep reinforcement learning doesn't work yet”. It is still writing: the last post was on 2026-08-18, and you can follow it by feed.
In a world where everyone has opinions, one man...also has opinions
Most-linked posts?
- Deep reinforcement learning doesn't work yet21 people · 24 mentions · first linked 2018-02-14
- /2019/02/22/alphastar.html2 people · 3 mentions · first linked 2019-07-10
- On the Perils of Batch Norm2 people · 2 mentions · first linked 2017-04-26
- The Tragedies of Reality Are Coming for You2 people · 2 mentions · first linked 2024-07-13
- My AI Timelines Have Sped Up2 people · 2 mentions · first linked 2020-08-25
- /2019/10/29/openai-rubiks.html2 people · 2 mentions · first linked 2020-05-07
What Hacker News says?
AGI = Artificial General Intelligence, watch this for the main idea around the goal alignment problem: https://www.youtube.com/watch?v=EUjc1WuyPT8 They're explicitly not political, lesswrong is a website/community and rationality is about trying to think better by being aware of normal cognitive biases and correcting for them. Also trying to make better predictions and understand things better by applying Bayes' the…
Comment linking /2020/08/18/ai-timelines.html
on “Still alive”Submitted Deep reinforcement learning doesn't work yet as a story · 246 points
HN discussionRL can be massively disappointing, indeed. And I agree with you (and with the amazing post I already referenced [1]) that it is hard to get it to work at all. Sorry to hear you have been disappointed so much! Nonetheless, I would personally recommend even just learning the basics and fundamentals of RL. Beyond supervised, unsupervised, and the most-recent and well-deservedly hyped semi-supervised learning (generativ…
Comment linking /2018/02/14/rl-hard.html
on “Deep Reinforcement Learning: Zero to Hero”Submitted My AI Timelines Have Sped Up as a story · 95 points
HN discussionThanks a lot, and another great suggestion for improvement. I also found that the common advice is "tweak hyperparameters until you find the right combination". That can definitely help. But usually issues hide in different "corners", both of the problem space and its formulation, the algorithm itself (e.g., just different random seeds have big variance in performance), and more. As you mentioned, in real applicatio…
Comment linking /2018/02/14/rl-hard.html
on “Deep Reinforcement Learning: Zero to Hero”Submitted Solving Crew Battle Strategy with Math as a story · 69 points
HN discussionBoth are tricky subject and lot of cooking-like thinking (there are popular recipes, general rules, but in all practical cases to master it there is a lot of non-transferable experience and knowledge). Some pointers: * http://rishy.github.io/ml/2017/01/05/how-to-train-your-dnn/ * http://www.alexirpan.com/2017/06/27/hyperparam-spectral.html At deepsense.io we are developing Neptune to facilitate that process (compare…
Comment linking /2017/06/27/hyperparam-spectral.html
on “Hacker's guide to Neural Networks (2012)”Submitted A Gentle Introduction to Secure Computation as a story · 59 points
HN discussionWhile I agree to Karpathy and I also had a "wut? They call this RL? " reaction when RLHF was presented as an method of CHATGPT training, I'm a bit surprised by the insight he makes because this same method and insight have been gathered from "Learning from Human preference" [1] from none other than openAI, published in 2017. Sometimes judging a "good enough" policy is order of magnitudes more easier than formulating…
Comment linking /2018/02/14/rl-hard.html
on “RLHF is just barely RL”Submitted My AI Timelines Have Sped Up (Again) as a story · 50 points
HN discussionHere is a detailed post which covers the specifics: https://www.alexirpan.com/2019/02/22/alphastar.html
Comment linking /2019/02/22/alphastar.html
on “DeepMind’s StarCraft II agent will play anonymously on battle.net”Submitted Deep Reinforcement Learning Doesn't Work (yet) as a story · 45 points
HN discussionIt's disputable whether any such limitations currently applied to AlphaStar are really working as intended. There has been some controversy over it. I saw the matches and it did seem that some of the actions taken by AlphaStar would not be possible to do by a human. Human APM is limited not just by the mechanical speed of one's hands, but also by the mental models that exist in one's brain. Muscle memory can't do ev…
Comment linking /2019/02/22/alphastar.html
on “DeepMind’s StarCraft II agent will play anonymously on battle.net”Submitted Deep Reinforcement Learning Doesn't Work Yet as a story · 13 points
HN discussionEdit 2: Reading through the "supplementary data" of the 2019 paper, it definitely appears that the AlphaStar which reached grandmaster was not limited in the same ways as the 2017 paper would suggest. x/y positions of units are not determined visually, but fed directly from the API. So AlphaStar absolutely can just run Attack(Position: carrier_of_interest->pos.x) and not mis-click. It's "map" / "vision" is really ju…
Comment linking /2019/02/22/alphastar.html
on “How A.I. Conquered Poker”Submitted Late Takes on OpenAI O1 as a story · 4 points
HN discussionCurrent state-of-the-art in reinforcement learning can barely make a physical robot walk. In theory, with transfer learning, we will probably see better success over time but I'm looking forward to seeing results in practice. A 2018 article about the challenges of reinforcement learning: https://www.alexirpan.com/2018/02/14/rl-hard.html
Comment linking /2018/02/14/rl-hard.html
on “DeepMind says reinforcement learning is ‘enough’ to reach general AI”Submitted Deep Reinforcement Learning Doesn't Work Yet as a story · 3 points
HN discussionhttps://www.alexirpan.com/2018/02/14/rl-hard.html Reinforcment learning for the average person is a big waste of time. Probably for anyone atm
Comment linking /2018/02/14/rl-hard.html
on “Reinforcement Learning from scratch”Submitted On the Perils of Batch Norm as a story · 3 points
HN discussionAt the beginning of this post the given definition of value doesn't seem correct to me, because I think it should be the expected value of the sum of all the rewards from t to infinity and not of R_t, but I could be wrong. Citing D. Silver: The agent's job is to maximise cumulative rewards (highlight cumulative). I would suggest David Silver course for reinforcement learning, http://www0.cs.ucl.ac.uk/staff/d.silver/…
Comment linking /2018/02/14/rl-hard.html
on “Reinforcement Learning – Bandit Problems”Submitted Detailed Read-Through: Wasserstein GAN as a story · 3 points
HN discussionNot meant as a critique to this book, but the fact that RL-based approaches rarely work for optimal control problems (in, for example, robotics) came as a surprise to me, given the hype and focus on RL [1]. It turns out that model-based methods for optimal control (e.g. linear quadratic control) invented quite a long time ago dramatically outperform RL-based approaches in most tasks and require multiple orders of ma…
Comment linking /2018/02/14/rl-hard.html
on “Reinforcement Learning and Optimal Control”Submitted I'm Switching into AI Safety as a story · 2 points
HN discussionTwo months to advance the state of the art on a complex physics-based game with branching paths. Current approaches such as DQNs or god forbid DRL[1] barely reach the performance of my three year old cousin in atari game score maximization and are mostly non-transferable to new levels... Good luck. [1] https://www.alexirpan.com/2018/02/14/rl-hard.html
Comment linking /2018/02/14/rl-hard.html
on “OpenAI Retro Contest”Submitted The Tragedies of Reality Are Coming for You as a story · 2 points
HN discussion"Data", not so much, but you can explore the subject by searching for "reality gap in machine learning". To be honest there are not many references to it that I could find online, but this post by the Google AI blog starts with a very good summary (then proceeds to propose one way to overcome the difficulties of training on simulations, though of course the problem is still far from solved): https://ai.googleblog.co…
Comment linking /2018/02/14/rl-hard.html
on “Machine Learning Confronts the Elephant in the Room”Submitted An Overdue Post on AlphaStar, Part 2 as a story · 2 points
HN discussionAgh I typed up a response but it got deleted somehow. But basically, I view there as being 3 layers of understanding. For whatever level you're stuck at, there's different ways to get "unstuck". 1. Understand the task they're solving, how they did it, and their results. Anybody with a basic understanding of the domain should be able to do this by reading the abstract/intro and conclusion. If you find yourself having…
Comment linking /2017/02/22/wasserstein-gan.html
on “My Favorite Deep Learning Papers of 2017”Submitted Thoughts on ICLR 2018 and ICRA 2018 as a story · 2 points
HN discussion
