Purge from file history

Description

Customer is saying

Recently, we had an engineer commit files with sensitive information, attempting to remediate such spillage proved too difficult and resulted in removal and re-creation of the project. Having a feature to purge a specific file from project history would have saved us hours. Using git-filter-branch and BFG (https://rtyley.github.io/bfg-repo-cleaner/) proved to be ineffective in this case.

Completely removing a file from history is feasible but hard. And definitely requires advanced git knowledge. We could simplify this greatly.

Proposal

• In the file browser, when viewing a file, when clicking on the Delete button, display a popup:
  • In this popup, indicate a message saying what deletion means, and provide a checkbox with the option "Also purge this file from git history".
• Behind the scenes, we leverage https://rtyley.github.io/bfg-repo-cleaner/ to do it for us.

@chriscool could you tell me if this is something we could do, and if that would be the right way of doing it? Thanks!

@regisF sorry for the late response.

Yeah, it's possible to use the BFG (or maybe git-filter-branch) and it is the right tool, but as it may rewrite the history completely, it could create a number of problems.

For example all the processes (CI, fetch, push, gc, etc) related to the current repo should probably be stopped when doing that. And it should perhaps invalidate the previous CI results and deployments for all the tips of the branches that have been changed by the tool.

Then as the clients access the repo after the purge, they will have problems if the branches have been rewritten, and if they try to merge their own branches or force the push, it could reintroduce the purged file into the repo. So you have to make it clear to the client what they should and shouldn't do, and check what they do.

Also the file and its previous versions might be completely lost (at least on the server side), so it might be a problem if many people have the right to purge the file, or if it is done by mistake (how can I get the file and its old versions back?)

So what's difficult is not doing it (using the BFG or git filter-branch), but managing the process and its big consequences.

@chriscool thank you for your input. It is definitely very valuable, I didn't think of all these consequences.

Since we can't really predict ALL the consequences this feature can have, what can we do about it? Provide clear instructions in our documentation on how to achieve this, but don't provide the feature in the UI?

Or brute force the purge with a super clear, scary warning (like when you want to rename a repository on github)?

added EE Starter label

While talking about renaming a repository, I think a way to possibly implement this feature would be to just create a new repo with the purged history under another name. This way people who want to switch to the purged history could first play with it while keeping the old repo in case things are going too badly with the new one.

This would not fix all the possible problems, like for example people could perhaps still merge or force push the purged file back into the new repo, but it would alleviate problems with processes, CI results and deployment invalidation, getting the file and its old versions back.

@chriscool that could be a good workaround actually.