Top all-time · #11658
sidsite
Personal blog · sidsite.comaidataprogramming
sidsite is a personal blog at sidsite.com about ai, data and programming. Since 2019, 6 different Hacker News readers have linked it 49 times, most often “The Fourier transform is a neural network”. It is still writing: the last post was on 2026-10-22, and you can follow it by feed.
Too few sampled comments (4) to say how often HN recommends it.?
Why HN links it
HN readers link sidsite's autodiff tutorial as an introduction to reverse-mode automatic differentiation. Commenters call it a better introduction than others they have seen.
Summarised by Claude from a sample of 4 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
The site of Sid
Most-linked posts?
- The Fourier transform is a neural network3 people · 3 mentions · first linked 2021-04-29
- From scratch: reverse-mode automatic differentiation (in Python)2 people · 8 mentions · first linked 2020-06-11
What Hacker News says?
Hope you don't mind me plugging my blog post, that covers chain rule -> autodiff -> training of nn. https://sidsite.com/posts/autodiff/
Comment linking /posts/autodiff/
on “Lessons I wish I had learned before I started teaching differential equations [pdf] (1997)”Submitted The Fourier transform is a neural network as a story · 267 points
HN discussionOp here, IMO the "deepest" bit is [1] - the network learns the DFT in order to reconstruct the signal, and is not explicitly trained on the DFT values from the FFT. Admittedly, I should have mentioned that any linear transform can be considered to be a single layer neural network (if you want to see the world through a neural network lens), and will add this to the post at some point. In fact, I have a series of pos…
Comment linking /posts/fourier-nets/
on “The Fourier transform is a neural network”Submitted From scratch: reverse-mode automatic differentiation (in Python) as a story · 114 points
HN discussionYou can implement autograd as a library. Just take a look at this https://github.com/sradc/SmallPebble The first line of the description is: > SmallPebble is a minimal automatic differentiation and deep learning library written from scratch in Python, using NumPy/CuPy. The author of this library wrote a blog post, which is in my opinion the best introduction to reverse mode automatic differentiation that exists: htt…
Comment linking /posts/autodiff/
on “Fastest autograd in the West”Submitted Piano Tuning as a story · 101 points
HN discussionIt looks like it's doing only addition and multiplication. The rest is left as an exercise to the reader. Here's a much better introduction, that links you to an industrial-level autodiff library [1]. [1] https://sidsite.com/posts/autodiff/
Comment linking /posts/autodiff/
on “Reverse-mode algorithmic differentiation using effect handlers in OCaml 5”Submitted Transformer neural net learns to run Conway's Game of Life just from examples as a story · 69 points
HN discussionThere's an example here of using Numba to speed up data science code, (disclaimer, I'm the author): https://sidsite.com/posts/python-corrset-optimization/ > for most "normal" uses, you'd probably be fine with having your code written in numpy or pandas In my experience this is the case. Occasionally though, there might be an bottleneck, and Numba can be a good way to handle it. E.g. when implementing seam carving, I…
Comment linking /posts/python-corrset-optimization/
on “Profiling your Numba code”Submitted A toy example of Bayesian hyperparameter optimization on parallel cloud VMs as a story · 60 points
HN discussionThanks. Yes, I did. During the process of making this [1] more basic autodiff framework, I realised that it could become actually usable if the values the "Variables" contained were NumPy arrays. Then I just found myself chipping away it at. It was worth it for me, since I found it fun to work on, learned many things, and want to continue learning in this area. A limitation is that one only has to work out the local…
Comment linking /posts/autodiff/
on “Show HN: I wrote a minimal deep learning library from scratch in Python”Submitted Gridnotes – an infinite 2D text editor as a story · 4 points
HN discussionHere's what you do: you watch this video of Andrew Karpathy [1] called "Becoming a backprop ninja". Then you pick up a function that you like and implement this backprop (which is a different way of saying reverse mode automatic differentiation) using just numpy. If you use some numpy broadcasting, an np.sum, some for-loops, you'll start getting a good feel for what's going on. Then you can go and read this fabulous…
Comment linking /posts/autodiff/
on “Understanding Automatic Differentiation in 30 lines of Python”Submitted Reverse-mode automatic differentiation from scratch, in Python as a story · 4 points
HN discussionRelated; 4x improvement in accuracy for experiment 2 of this paper, (with gpt-3.5-turbo), by changing the prompt: https://sidsite.com/posts/reversal-curse/ Imo it's interesting to see what a dramatic effect prompt engineering can have.
Comment linking /posts/reversal-curse/
on “Is the reversal curse in LLMs real?”Submitted Extracting Copyrighted Text from GPT as a story · 3 points
HN discussionhttps://sidsite.com/ Reverse-mode automatic differentiation from scratch, in Python - https://sidsite.com/posts/autodiff/ Notes on training BERT from scratch on an 8GB consumer GPU - https://sidsite.com/posts/bert-from-scratch/ How the BPE tokenization algorithm used by large language models works - https://sidsite.com/posts/bpe/
Comment linking /posts/bpe/
on “Ask HN: Could you share your personal blog here?”Submitted How to implement semantic video search in 5 minutes using OpenAI's CLIP as a story · 3 points
HN discussionhttps://sidsite.com/ <- so hn recognises as link
Comment linking /
on “Ask HN: Could you share your personal blog here?”Submitted The apostrophe's in the wrong place show I'm human as a story · 2 points
HN discussionThis tutorial [0] is by far the best resource I have found for learning the intuition of reverse-mode automatic differentiation in Python [0] https://sidsite.com/posts/autodiff/?utm_source=pocket_mylist
Comment linking /posts/autodiff/?utm_source=pocket_mylist
on “The Simple Essence of Automatic Differentiation”Submitted Training a Simple Transformer Neural Net on Conway's Game of Life as a story · 2 points
HN discussionSubmitted The Fourier transform is a neural network as a story · 2 points
HN discussionSubmitted Bayesian hyperparam optimization parallelized on cloud VMs with Kubeflow example as a story · 2 points
HN discussionSubmitted Implementing semantic video search in 5 minutes using CLIP as a story · 2 points
HN discussionSubmitted On Being Shy by Jerome K. Jerome as a story · 2 points
HN discussionSubmitted Implementing Naive Bayes in Python as a story · 2 points
HN discussionSubmitted The hierarchical highlight journalling system as a story · 1 points
HN discussionSubmitted Training a Simple Transformer Neural Net on Conway's Game of Life as a story · 1 points
HN discussionSubmitted Demo: Gridnotes – an infinite 2D text editor as a story · 1 points
HN discussion