Top all-time · #2126
R
rtyley.github.io
Personal blog · rtyley.github.ioprogrammingsystems
rtyley.github.io is a personal blog at rtyley.github.io about programming and systems. Since 2013, 38 different Hacker News readers have linked it 38 times, most often “BFG Repo-Cleaner: an alternative to git-filter-branch”. No feed was found, so read it on the site itself.
No feed foundVisit the blog
38mentions?
38HN people?
2013–2025first – last linked?
rtyley.github.io :
Most-linked posts?
- BFG Repo-Cleaner: an alternative to git-filter-branch34 people · 34 mentions · first linked 2015-01-19
What Hacker News says?
When you need to do it in a big repo, BFG Repo-Cleaner is usually the way to go: https://rtyley.github.io/bfg-repo-cleaner/
Comment linking /bfg-repo-cleaner/
on “YouTube-dl's repository has been restored”Submitted BFG Repo-Cleaner: an alternative to git-filter-branch as a story · 35 points
HN discussionHave you tried this: https://rtyley.github.io/bfg-repo-cleaner/ I've used it in the past, and it's pretty straightforward and does the job well :)
Comment linking /bfg-repo-cleaner/
on “Oh shit, git: Getting myself out of bad situations”Submitted BFG repo-cleaner: a faster `git filter-branch`, written in Scala as a story · 2 points
HN discussionGit bfg should do the job, it works well for that and to remove files with secrets... or really anything you want to pretend never existed in your repo :) https://rtyley.github.io/bfg-repo-cleaner/
Comment linking /bfg-repo-cleaner/
on “Oh shit, git: Getting myself out of bad situations”Submitted BFG Repo-Cleaner – like git-filter-branch, but faster as a story · 2 points
HN discussionFor anyone wondering, if you want to remove a file or secret you've already committed, you can use BFG Repo-Cleaner to go through your commit history and completely remove any trace of it. https://rtyley.github.io/bfg-repo-cleaner/
Comment linking /bfg-repo-cleaner/
on “GitHub commit search: “remove password””Submitted BFG Repo-Cleaner – Removes large or troublesome blobs from Git as a story · 1 points
HN discussionIt's possible to remove that from the history of your repo, although it breaks any forks. https://rtyley.github.io/bfg-repo-cleaner/
Comment linking /bfg-repo-cleaner/
on “You know how HTTP GET requests are meant to be idempotent?”Submitted A fast alternative to git-filter-branch as a story · 1 points
HN discussionOh shit, I commited a binary file larger than 100 mb and now i can't push to github.com. Solution: https://rtyley.github.io/bfg-repo-cleaner/
Comment linking /bfg-repo-cleaner/
on “Some bad Git situations and how I got myself out of them”This is great! And how did I not know about this? My favorite days are when I learn something new. Even better are when I learn that I was wrong and change my mind to improve my thinking. So meta! Git notes would be ideal for annotating commits that contain commit hashes used as breadcrumbs to inform the developer (usually me months later) about context around previous work. These hashes might have changed due to a…
Comment linking /bfg-repo-cleaner/
on “Git Notes: Git's coolest, most unloved feature (2022)”Andrew, co-founder of Gitter here. Removing secrets was a lot of work - more than I expected - while we open-sourced the product. I agree with your sentiment though. Handling secrets in a codebase is not something that it currently easy or standardised. As an aside, BFG Repo Cleaner really helped a lot with cleaning things up: https://rtyley.github.io/bfg-repo-cleaner/
Comment linking /bfg-repo-cleaner/
on “Gitter is open source”Most development shops are using the traditional client-server model, or self-host using Gitlab. I personally would never allow a repo with two hashing algorithms to exist on my watch. If you have ever had to use a tool like BFG to prune large objects from a repo you'll see it's not that bad, but it does require users to re-clone. I would want to use the same process for SHA256 - that is let it be the default for ne…
Comment linking /bfg-repo-cleaner/
on “A new hash algorithm for Git”When I helped to take Zulip open-source in 2015, I wrote a simple script that scrubbed secrets from the commit history using git fast-export and git fast-import. We replaced all our secrets with xxxxxxx placeholders, replaced internal customer references with dummy names, deleted and renamed certain files, and even did some code replacements that caused certain commit diffs to become empty so those commits could be…
Comment linking /bfg-repo-cleaner/
on “The Python Package Index is now a GitHub secret scanning integrator”In principle, you should be able to 'rewrite history' in the same way you can already do with git. It is clunky to remove a file from all versions using git itself but easy using tools like bfg[0]. [0] https://rtyley.github.io/bfg-repo-cleaner
Comment linking /bfg-repo-cleaner
on “Dolt is Git for data”There's also BFG ( https://rtyley.github.io/bfg-repo-cleaner/ ) for people like me who are scared of filter-branch. As someone else noted, this is about small, frequently changing files, so you could remove old versions from the history to save space, and use LFS going forward.
Comment linking /bfg-repo-cleaner/
on “We shrunk our Javascript monorepo git size”https://rtyley.github.io/bfg-repo-cleaner/
Comment linking /bfg-repo-cleaner/
on “Improving large monorepo performance on GitHub”You can use BFG [1] or git-filter-branch to remove individual files from the history, though doing so will change the hash of every commit since the files were introduced. Whether it's worth the disruption or not is up to you. [1]: https://rtyley.github.io/bfg-repo-cleaner/
Comment linking /bfg-repo-cleaner/
on “Nvui: A NeoVim GUI written in C++ and Qt”I just personally use BFG to scrub files from history https://rtyley.github.io/bfg-repo-cleaner/
Comment linking /bfg-repo-cleaner/
on “Bit – A modernized Git CLI written in Go”Daily reminder: - Once it is on the internet - it is always there so Rotate the key/secrets FIRST. - Never think secrets are gone because of you have recommited . - Deleting a commit is not enough , use BFG Cleaner - https://rtyley.github.io/bfg-repo-cleaner/ , and force commit to change history. Edit- Forget to add most important thing - rotating the key.
Comment linking /bfg-repo-cleaner/
on “I scanned all of GitHub's "oops commits" for leaked secrets”> erhaps now is a good time to recommend the ever-popular BFG to anyone unaware: https://rtyley.github.io/bfg-repo-cleaner/ Thanks for sharing this tool to help cleanup the Git history. Please be aware that it will rewrite the history, which could be very impactful to existing branches, merge requests and local clones. A similar approach is described in the documentation: https://docs.gitlab.com/ee/user/project/repo…
Comment linking /bfg-repo-cleaner/
on “Starting October 19, storage limit will be enforced on all Gitlab Free accounts”Perhaps now is a good time to recommend the ever-popular BFG to anyone unaware: https://rtyley.github.io/bfg-repo-cleaner/ Also my team's biggest repo is a 2.5 GB checkout but gitlab (self-managed) reports it as 185MB "files" and 353 MB "storage" (no CI/CD artifacts).
Comment linking /bfg-repo-cleaner/
on “Starting October 19, storage limit will be enforced on all Gitlab Free accounts”SpongyCastle is a repackage of BouncyCastle specifically for the Android platform, an unfortunate necessity due to Android shipping with a cut-down version of Bouncy Castle (see https://code.google.com/p/android/issues/detail?id=3280 & http://rtyley.github.io/spongycastle/ ). ...I'm the packager of SpongyCastle (a task a well-trained monkey could do, I don't alter the code other than package-renaming it), and I conf…
Comment linking /spongycastle/
on “Android RNG Weakness Renders Bitcoin Wallets Insecure”easiest method (this article glosses over it): use BFG 2000: https://rtyley.github.io/bfg-repo-cleaner/
Comment linking /bfg-repo-cleaner/
on “On undoing, fixing, or removing commits in git”I noticed just today that Github has a number of counter-measures for absurd git repositories built in when you try to push something. For instance, I imported a huge (3GB, mostly due to large frequently updated files in the history) subversion repository to git and got failures due to individual commits exceeding 100MB. This was quite helpful to bring the size of my repository to a reasonable state. Tools like the…
Comment linking /bfg-repo-cleaner/
on “Measuring the many sizes of a Git repository”Good. Looks like a job for https://rtyley.github.io/bfg-repo-cleaner/ and a new API key + revoking that one.
Comment linking /bfg-repo-cleaner/
on “FuzzBeed – Randomly generated BuzzFeed”Article quotes someone making this claim: > we discovered that even if commit histories are rewritten, secrets can still be recovered…. we discovered we could recover the full contents of deleted commits from GitHub with only the commit’s SHA-1 ID. Do repo cleaning tools such as https://rtyley.github.io/bfg-repo-cleaner/ leave the original commit's SHA-1 ID intact?
Comment linking /bfg-repo-cleaner/
on “Characterizing secret leakage in public GitHub repositories”For the mentioned problem of removing sensitive data, BFG Repo-Cleaner does the trick: https://rtyley.github.io/bfg-repo-cleaner/
Comment linking /bfg-repo-cleaner/
on “Git, JSON and Markdown walk into bar”BFG Repo Cleaner is also a good tool which I don't see mentioned in the article: https://rtyley.github.io/bfg-repo-cleaner/
Comment linking /bfg-repo-cleaner/
on “How to Handle Big Git Repositories”
