Add foreign keys to various tables that point to the "projects" table
This adds foreign keys to various tables that have project_id
columns referring to the projects
table. All these foreign keys have a ON DELETE CASCADE
clause set, making it much easier and faster to remove data associated with a project (while also enforcing consistency). The MR includes a rather big migration to do all of this without requiring downtime and while making sure no orphaned data exists.
Some assocations are still removed by Rails. For example, LFS objects are still removed one by one as for every row we also need to remove data on the file system and there's no easy way of doing this in bulk. The same applies to CI artifacts and traces, which need to be migrated directory wise first (taken care of in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11641).
The EE version of this MR (to deal with EE code such as ElasticSearch) can be found here: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2223
Related issues/MRs:
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6292
- https://gitlab.com/gitlab-org/gitlab-ce/issues/27998
Migration Timings
Migration | Time on Staging |
---|---|
ProjectForeignKeysWithCascadingDeletes | 60 minutes at least |
CorrectProtectedBranchesForeignKeys | 1.6 seconds |
AddForeignKeyForMergeRequestDiffs | 60 seconds |
The migration ProjectForeignKeysWithCascadingDeletes
had to be run 3 times as
the first time it did not take care of orphans in the
protected_branch_push_access_levels
table, leading to it failing when it tried
to remove orphans from protected_branches
. The second time it failed because a
table had orphans again that were added after the last removal. The 3rd time it
took 30 minutes to complete.