Top all-time · #1974
OpenSource Connections
Company blog · opensourceconnections.com/blogdataprogramming
OpenSource Connections is the company blog at opensourceconnections.com, writing about data and programming. Since 2008, 36 different Hacker News readers have linked it 133 times, most often “What problem does BERT hope to solve for search relevance?”. No feed was found, so read it on the site itself.
Too few sampled comments (24) to say how often HN recommends it.?
Why HN links it
OpenSource Connections' blog covers search relevance, Solr and Elasticsearch comparisons, BM25 and vector search, and databases. HN readers link its search comparisons and relevance posts.
Summarised by Claude from a sample of 24 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
At OpenSource Connections we specialize in search & AI training and consulting including Elasticsearch, Solr and OpenSearch.
Most-linked posts?
- What problem does BERT hope to solve for search relevance?3 people · 3 mentions · first linked 2019-12-19
- BM25 – The Next Generation of Lucene Relevance3 people · 3 mentions · first linked 2016-03-12
- Solr vs. Elasticsearch Query DSLs2 people · 3 mentions · first linked 2016-06-30
- Lets Stop Saying 'Cognitive' Search2 people · 2 mentions · first linked 2019-05-28
- Vespa vs. Lucene: Initial Impressions2 people · 2 mentions · first linked 2017-10-06
- Stop Worrying about Solr vs. Elasticsearch Decisions2 people · 2 mentions · first linked 2019-02-28
- Understanding BERT and Search Relevance (2019)2 people · 2 mentions · first linked 2019-12-13
- Using HyperLogLog to detect voting rings (2013)2 people · 2 mentions · first linked 2015-01-23
- Hybrid vigor: a winning way to use hybrid search2 people · 2 mentions · first linked 2023-03-07
- Recommender Systems 101: Basket Analysis2 people · 2 mentions · first linked 2016-06-10
What Hacker News says?
Great blog post on this subject: https://opensourceconnections.com/blog/2019/02/28/stop-worry...
Comment linking /blog/2019/02/28/stop-worrying-solr-elasticsearch/
on “OpenSearch: AWS fork of Elasticsearch and Kibana”Submitted Falsehoods Programmers Believe About Search as a story · 337 points
HN discussionAs someone who works in search this is unsurprising, and happens all the time. Search is a two sided sales exercise both to show you good products and also to products that make the e-commerce business money. Search is the e-commerce businesses sales person - not a neutral broker. You don’t expect the blue shirt at Best Buy to be entirely neutral do you? It’s actually a pretty understudied microeconomics case https:…
Comment linking /blog/2017/07/04/optimizing-user-product-match-economies/
on “Amazon Changed Search Algorithm in Ways That Boost Its Own Products”Submitted How a HyperLogLog works as a story · 117 points
HN discussionDoes anyone know of the"sqlite connector" mentioned in this post ? https://opensourceconnections.com/blog/2013/05/06/does-found... It would be really cool to find it, if it's still out there.
Comment linking /blog/2013/05/06/does-foundationdb-beat-the-cap-conjecture-hackathon-on-friday/
on “FoundationDB: A distributed, unbundled, transactional key value store [pdf]”Submitted Codd's relational vision – has NoSQL come full circle? as a story · 107 points
HN discussionHere is their resiliency status: https://www.elastic.co/guide/en/elasticsearch/resiliency/cur... It depends on your use case. If you are already familiar with Solr and it is good enough for your use case, then use it. Solr and ES are about the same feature-wise. Scaling is easier for ES because it is built-in. Here is a good comparison of their APIs. part 1: http://opensourceconnections.com/blog/2015/12/15/solr-vs-e…
Comment linking /blog/2016/01/22/solr-vs-elasticsearch-relevance-part-two/
on “Things to know before using AWS’s Elasticsearch Service”Submitted Falsehoods Programmers Believe About Search (2019) as a story · 106 points
HN discussionShameless plug - I’m teaching an intense NLP training course over 4 half days, which covers transformers and KNN the latter 2 days. https://opensourceconnections.com/training/natural-language-...
Comment linking /training/natural-language-search/
on “Rethinking Attention with Performers”Submitted Thoughts on Algolia vs. Solr and Elasticsearch as a story · 101 points
HN discussionAnd thank you for considering my reply as "kind words". :) I would not have been so harsh if I knew you were going to read the reply! That first chapter though. So many points repeated. For those that want to read more: http://opensourceconnections.com/about-us/doug-turnbull/ or really anything at http://opensourceconnections.com/blog/
Comment linking /blog/
on “Search at Slack”Submitted BM25 – The Next Generation of Lucene Relevance as a story · 95 points
HN discussiontl;dr is that those are different approaches to weighting documents in order to return the most relevant ones for a query. For an intro to the problem space, see https://opensourceconnections.com/blog/2014/06/10/what-is-se... If you want a lot more detail, check out the book Relevant Search. https://www.manning.com/books/relevant-search
Comment linking /blog/2014/06/10/what-is-search-relevancy/
on “Xapiand: A fast, simple, modern search and storage engine”Submitted Opening up search is an ethical imperative as a story · 83 points
HN discussionOn “return generators instead of lists” I think it’s also important when you do this to return use a context manager to manage the generators lifetime. Otherwise you might be surprised when the underlying file or source is closed later in your code. I blogged about this: https://opensourceconnections.com/blog/2020/06/10/python-gen...
Comment linking /blog/2020/06/10/python-generators-and-context-managers/
on “Specific ways to write better Python (2017)”Submitted Understanding BERT and Search Relevance (2019) as a story · 78 points
HN discussionYes you’re right on exact matches. For basic keyword searches. It’s not uncommon for search users to want inexact equivalencies, synonyms or conceptually close search results. (Search for cold, get back rhinovirus). Every user and use case has different definitions/tolerances of “this term appeared on the page” It’s a complicated topic, without simple solutions. I wrote about it here. https://opensourceconnections.c…
Comment linking /blog/2018/12/07/synonyms-by-any-other-name-part-1/
on “Google may rank sites for queries that don't appear on the page at all”Submitted Learning to Rank 101 – Linear Models as a story · 77 points
HN discussionI would say this isn't machine learning, but relevance in general is an interesting topic to apply supervised learning. Of course the training data is the hard part, An article on the topic, https://opensourceconnections.com/blog/2017/08/03/search-as-... (disclaimer I wrote it...)
Comment linking /blog/2017/08/03/search-as-machine-learning-prob/
on “Machine Learning: Full-Text Search in JavaScript – Relevance Scoring (2015)”Submitted New Book: 'Relevant Search' as a story · 76 points
HN discussionI think a lot off preference towards humans is an issue of trust and a measure of your relationship with that person. Usually we get recommendations from friends and give them more thought and weight than some random ad we scroll by on Facebook. https://opensourceconnections.com/blog/2016/08/21/recommenda...
Comment linking /blog/2016/08/21/recommendations-systems-not-as-cool-as-friends/
on “Categories of deep recommendation systems”Submitted Using HyperLogLog to detect voting rings (2013) as a story · 72 points
HN discussionThis is a good opportunity to update my search reading list. Books * Lucene in Action - A bit out of date, but it has a lot of the core concepts of Lucene. 10 years ago, this was the go-to book even if you’re working with Solr or Elasticsearch, to understand the core data structures * Relevant Search (my book) - Introduction to optimizing the relevance of full text search engines. Getting a bit old now, but still re…
Comment linking /training
on “Ask HN: I want to dive into how to make search engines”Submitted Semantic Search with Latent Semantic Analysis as a story · 61 points
HN discussionIf you're into this kind of thing check out the #vectors-in-search channel at https://opensourceconnections.com/slack -- Us vector search nerds hang out there (and some of the Home Depot search nerds too) and share insights and ideas.
Comment linking /slack
on “Home Depot Finds DIY Success with Vector Search”Submitted Detecting voting rings using HyperLogLog counters (2013) as a story · 57 points
HN discussionIn 2015, they evaluated Apache Solr and Elasticsearch and decided to build their own (Firefly). They said, other solutions did not scale. So, instead of contributing to scaling (like Apple and Bloomberg and Cloudera did), they went the other way. Now, they seem to be doing it again (at least they are using Tika). In a meanwhile, Solr implemented most of the features they are describing in their architecture document…
Comment linking /blog/2015/10/16/bm25-the-next-generation-of-lucene-relevation/
on “Architecture of Nautilus, the new Dropbox search engine”Submitted Easy to Build Recommender Systems with Elastic Graph as a story · 14 points
HN discussionDon’t forget JMESPath, as built into AWS CLI: http://jmespath.org/ With language libraries or as jp CLI: http://jmespath.org/libraries.html Example AWS CLI tricks: https://gist.github.com/magnetikonline/6a382a4c4412bbb68e33e... https://opensourceconnections.com/blog/2015/07/27/advanced-a...
Comment linking /blog/2015/07/27/advanced-aws-cli-jmespath-query/
on “Jsonpath – a query language for JSON in Postgres [pdf]”Submitted Improving on Git Flow: Code Reviews as a story · 8 points
HN discussion
