Top all-time · #2590
Brendan T. O'Connor
Personal blog · brenocon.com/blog · on HN as brendanodataai
Brendan T. O'Connor writes Brendan O'Connor, a personal blog at brenocon.com about data and ai. Since 2011, 29 different Hacker News readers have linked it 40 times, most often “Statistics vs. Machine Learning”. The last post was in 2015, but the archive is still widely read.
Most-linked posts?
- Statistics vs. Machine Learning6 people · 7 mentions · first linked 2011-05-26
- /blog/2009/09/dont-mawk-awk-the-fastest-and-most-elegant-big-data-munging-language/4 people · 6 mentions · first linked 2016-07-28
- "Numbers Everyone Should Know" from Jeff Dean4 people · 4 mentions · first linked 2025-11-15
- /watson_special_issue/2 people · 3 mentions · first linked 2017-08-11
- Don’t Mawk Awk (2009)2 people · 3 mentions · first linked 2012-11-14
- Statistics vs. Machine Learning, fight2 people · 2 mentions · first linked 2016-12-18
What Hacker News says?
If anyone is interested, this issue from IBM outlines most parts of Watson in great detail. http://brenocon.com/watson_special_issue/
Comment linking /watson_special_issue/
on “IBM is not doing "cognitive computing" with Watson (2016)”Submitted Statistics vs. Machine Learning as a story · 59 points
HN discussionawk is a language with multiple implementations, so we can only talk about performance of a particular awk implementation. Which is especially true for awk because mawk (based on a vm rather than being a traditional interpreter) is so much faster than other awks if you can live with its limitations [1]. If you're using gawk, setting LANG=C also helps performance because no utf-8 honoring needs to be done. Speaking o…
Comment linking /blog/2009/09/dont-mawk-awk-the-fastest-and-most-elegant-big-data-munging-language/
on “Awk in 20 Minutes (2015)”Submitted Don’t Mawk Awk (2009) as a story · 8 points
HN discussionThis is probably the most concise introduction to awk I've read. For more extensions[0] and advanced features like arrays of arrays and array sorting, there's also gawk. And for larger files there's performance driven mawk which can drastically increase processing speed[1]. [0] https://www.gnu.org/software/gawk/manual/html_node/Extension... [1] http://brenocon.com/blog/2009/09/dont-mawk-awk-the-fastest-a...
Comment linking /blog/2009/09/dont-mawk-awk-the-fastest-and-most-elegant-big-data-munging-language/
on “Awk in 20 Minutes”Submitted Statistics vs. Machine Learning, fight as a story · 4 points
HN discussion- nl · 2017-08-11
There's a lot more there than just search. See http://brenocon.com/watson_special_issue/
Comment linking /watson_special_issue/
on “A rising sentiment that IBM’s Watson can’t deliver on its promises” Submitted Statistics vs. Machine Learning, fight as a story · 4 points
HN discussion- nl · 2017-08-11
This is true. I've read all the Watson Special Edition papers numerous times[1]. The original Watson work was very good, and adding neural-network based models as alternative question parsers and answer rankers is a clear way forward. It's a pity IBM decided to use the Watson name for everthing they do in ML now. [1] http://brenocon.com/watson_special_issue/
Comment linking /watson_special_issue/
on “A rising sentiment that IBM’s Watson can’t deliver on its promises” Submitted Statistics vs. Machine Learning, fight (2008) as a story · 4 points
HN discussionFollowing up on blahedo's comment and the questions about what the heck their p-values mean -- This is a nice example that you can get statistical significance for small effects, if your sample is big enough. Their p-values are explained very badly, so I did my own analysis by transcribing their data from those plots. Let's take their weighting scheme for granted. I agree with some other commenters that the sums and…
Comment linking /Screen%20shot%202012-08-10%20at%202.03.07%20AM.png
on “The effect of typefaces on credibility”Submitted "Numbers Everyone Should Know" from Jeff Dean as a story · 3 points
HN discussionIf you bet on the right platform, code can live for a while. For example: this classic paper on compression http://web.stanford.edu/class/ee398a/handouts/papers/WittenA... was published with a C, Unix-based implementation (I put it here http://brenocon.com/WittenNealCleary-ArithmeticEncodin-cacm-... ) which compiles and runs fine (just lots of compiler warnings) on modern Linux and OS X, 18 years later.
Comment linking /WittenNealCleary-ArithmeticEncodin-cacm-87.shar
on “Going “Write-Only””Submitted Statistics vs. Machine Learning, fight (2008) as a story · 2 points
HN discussionMachine learning Statistics network, graphs model weights parameters learning fitting generalization test set performance supervised learning regression/classification unsupervised learning density estimation, clustering large grant = $1,000,000 large grant = $50,000 nice place to have a meeting: nice place to have a meeting: Snowbird, Utah, French Alps Las Vegas in August http://brenocon.com/blog/2008/12/statistics-…
Comment linking /blog/2008/12/statistics-vs-machine-learning-fight/
on “What is the difference between deep learning and usual machine learning?”- ot · 2012-02-02
Submitted Histograms — matplotlib vs. R as a story · 2 points
HN discussion Depends on the task...sometimes nawk ("one true awk") is faster than gawk. Mawk is almost always faster than either. An older, but good article on that: https://brenocon.com/blog/2009/09/dont-mawk-awk-the-fastest-...
Comment linking /blog/2009/09/dont-mawk-awk-the-fastest-and-most-elegant-big-data-munging-language/
on “Why GNU grep is fast (2010)”Submitted Powerset's natural language search system (2012) as a story · 1 points
HN discussionSee the table in this link for a humorous comparison by Tibshirani: https://brenocon.com/blog/2008/12/statistics-vs-machine-lear...
Comment linking /blog/2008/12/statistics-vs-machine-learning-fight/
on “Ideas in statistics that have powered AI”Submitted Numbers Everyone Should Know as a story · 1 points
HN discussionInteresting article! However, Deep Blue, Watson, and AlphaGo are very different from one another. I don't think anyone deemed beating humans at chess or jeopardy impossible at the time Deep Blue and Watson were built. On AlphaGo the point about generalizing AI is valid, yet I think the author doesn't fully appreciate the novelty of approach described in the AlphaGo paper. Their work advances the field, has more gene…
Comment linking /watson_special_issue/05%20automatic%20knowledge%20extration.pdf
on “Go, Marvin Minsky, and the Chasm That AI Hasn’t yet Crossed”Submitted Comparison of Data Analysis Software - 4 years of discussions in comments as a story · 1 points
HN discussionI put together some free online resources for linear algebra: http://brenocon.com/blog/2011/03/poor-mans-linear-algebra-te... Everyone I talked to then recommended the Strang textbook, and also Stanford EE263, both already pointed out by other commenters here.
Comment linking /blog/2011/03/poor-mans-linear-algebra-textbook/
on “Surprises from numerical linear algebra (2010)”Submitted Rough binomial confidence intervals as a story · 1 points
HN discussionHere is a nice 4-year old still active discussion on pros and cons of different data analysis technologies: http://brenocon.com/blog/2009/02/comparison-of-data-analysis...
Comment linking /blog/2009/02/comparison-of-data-analysis-packages-r-matlab-scipy-excel-sas-spss-stata/
on “Matlab, R, and Julia: Languages for data analysis”Here's a few things that really helped me out: http://mattmazur.com/2015/03/17/a-step-by-step-backpropagati... http://brenocon.com/blog/2013/10/tanh-is-a-rescaled-logistic... Sort of a high-level overview (some of this is particular to neural nets as they're implemented in JMP, a product made by SAS, I don't know how well they generalize): • Neural nets consist of an input layer, an output layer, and some number of…
Comment linking /blog/2013/10/tanh-is-a-rescaled-logistic-sigmoid-function/
on “Implementing a Neural Network from Scratch – An Introduction”First, I must say that I like the idea of the language. It has ADTs (case classes) and typeclasses (traits), a bit of type inference, can run on Android and isn't Java. It's just the baroque syntax that bothers me. It starts from this [1]. Then there's case classes, _, optional everything (forgetting = in a function def can make it mean something else), the currying syntax [2] (sometimes it needs an arbitrary _ to b…
Comment linking /scalacheat/
on “On Scala”> And performance of awk is crap. [...] I had replaced all the awk scripts in python and everything is a lot faster. My experience points exactly the other way: for data-processing tasks, especially streaming ones, even Gawk is a lot faster than Python (pre-3.11), and apparently I’m not the only one[1]. If you’re not satisfied with Gawk’s performance, though, try Nawk[2] or, even better, Mawk[3]. (And stick to POSIX…
Comment linking /blog/2009/09/dont-mawk-awk-the-fastest-and-most-elegant-big-data-munging-language/
on “Parsing Awk Is Tricky”Discussing performance only makes sense in the context of a particular awk implementation, like TFA is doing as well. If you‘re (stuck) on gawk, try setting LANG=C to prevent Unicode/multi-byte regexp execution, or switch to mawk (which according to [1] is much faster than cpython). [1]: https://brenocon.com/blog/2009/09/dont-mawk-awk-the-fastest-...
Comment linking /blog/2009/09/dont-mawk-awk-the-fastest-and-most-elegant-big-data-munging-language/
on “Parsing Awk Is Tricky”I'm not sure what the difference between statistics and machine-learning is supposed to be, seriously. The theoretical foundation of any machine learning algorithm is probability theory and statistics. Is machine learning implied to be a more "hands on" approach, to focus more on learning a set of techniques than on learning the theoretical underpinning of such techniques? Is it basically "statistical engineering"?…
Comment linking /blog/2008/12/statistics-vs-machine-learning-fight/
on “Statistics: Losing Ground to CS, Losing Image Among Students”
