Top all-time · #8689
Joshleeb's Blog
Personal blog · joshleeb.com/postsprogrammingdesign
Joshleeb's Blog is a personal blog at joshleeb.com about programming and design. Since 2018, 9 different Hacker News readers have linked it 28 times, most often “Gear Hashing for Content Defined Chunking”. It is still writing: the last post was on 2026-10-30, and you can follow it by feed.
Too few sampled comments (3) to say how often HN recommends it.?
Why HN links it
HN readers link these posts to explain content-defined chunking and gear hashing, the method behind CDC file transfer and FastCDC-style chunking. One commenter calls them a clear explanation of the idea.
Summarised by Claude from a sample of 3 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Most-linked posts?
- Gear Hashing for Content Defined Chunking3 people · 3 mentions · first linked 2023-03-09
- Intro to Content Defined Chunking3 people · 3 mentions · first linked 2023-03-04
- On Linking Bookmarks2 people · 2 mentions · first linked 2023-10-25
- Surprising Ctags Behaviour2 people · 2 mentions · first linked 2021-01-10
What Hacker News says?
If anyone else was left wondering about the details of how CDC actually generates chunks, I found these two blog posts explained the idea pretty clearly: https://joshleeb.com/posts/content-defined-chunking.html https://joshleeb.com/posts/gear-hashing.html
Comment linking /posts/gear-hashing.html
on “CDC File Transfer”Submitted Shaping ligatures in monospace fonts as a story · 84 points
HN discussionI wonder whether OpenAI legal can make the case for storing fuzzy hashes of the content, in the form of ssdeep[1] hashes or content-defined chunks[2] of said data, instead of the actual conversations themselves. After all, since the NYT has a very limited corpus of information, and supposedly people are generating infringing content using their APIs, said hashes can be used to compare whether such content has been g…
Comment linking /posts/content-defined-chunking.html
on “How we’re responding to The NYT’s data demands in order to protect user privacy”Submitted Time as a Grid as a story · 4 points
HN discussionYou may or may not be familiar with the content defined chunking family of algorithms like FastCDC. https://joshleeb.com/posts/chunking.html https://www.usenix.org/conference/atc16/technical-sessions/p... On the side I'm working on an extension to git-lfs to use fastcdc for both storage and transmission of artifacts to drastically reduce size and make git-lfs more practical for more things.
Comment linking /posts/chunking.html
on “We could save petabytes of cache storage with Zstandard and Pingora”Submitted Designing Kar as a story · 3 points
HN discussionSubmitted Scoped Tagging and Local Exports as a story · 3 points
HN discussionSubmitted Bookmark Lists and Scoped Tagging as a story · 3 points
HN discussionSubmitted Scoped Tagging of Bookmarks as a story · 3 points
HN discussionSubmitted Concept Pieces for a Graphical Editor as a story · 3 points
HN discussionSubmitted Gear Hashing for Content Defined Chunking as a story · 3 points
HN discussionSubmitted Traits and Trait Objects in Rust as a story · 3 points
HN discussionSubmitted Snowhash in Rust with Wasm as a story · 3 points
HN discussionSubmitted More Exploration for a Graphical Code Editor as a story · 2 points
HN discussionSubmitted On Linking Bookmarks as a story · 2 points
HN discussionSubmitted A Look at RapidCDC and QuickCDC as a story · 2 points
HN discussionSubmitted Improving on Gear Hashing with FastCDC as a story · 2 points
HN discussionSubmitted Surprising Ctags Behaviour as a story · 2 points
HN discussionSubmitted Surprising Ctags Behaviour as a story · 2 points
HN discussionSubmitted Integration Testing in Rust as a story · 2 points
HN discussionSubmitted Chatting with the Designer of MonoLisa as a story · 1 points
HN discussionSubmitted Status Update: February 2024 as a story · 1 points
HN discussionSubmitted On Linking Bookmarks as a story · 1 points
HN discussionSubmitted On Organizing Bookmarks as a story · 1 points
HN discussionSubmitted Gear Hashing for Content Defined Chunking as a story · 1 points
HN discussionSubmitted Intro to Content Defined Chunking as a story · 1 points
HN discussionSubmitted Intro to Content Defined Chunking as a story · 1 points
HN discussion