Rename all forbidden paths again

What does this MR do?

This MR renames projects that might be using a forbidden name with their path CaMeLcAsEd.

Currently these projects would not be accessible in the GitLab web-interface (rendering a 404 instead). But they are still accessible trough workhorse or the api. So users could still be interacting with them.

Since this would touch a lot of rows in the database as well as the filesystem. We now store every path that was changed in redis. This gives us a log that we can use to rollback & redo the migration. This was added after discussion with @DouweM & @smcgivern (https://gitlab.com/gitlab-com/infrastructure/issues/1909)

To generate a bunch of invalid data I used this script.

Why was this MR needed?

The NamespaceValidator in 9.1, and first version of DynamicPathValidator (before https://gitlab.com/gitlab-org/gitlab-ce/commit/4345bb8c507a11af694617187dea14284f48fb96) were matching case-sensitive. That means that users might have created invalid paths

Affected rows

Type	Query	Rows
projects	SELECT count(*) FROM "projects" INNER JOIN "routes" ON "routes"."source_id" = "projects"."id" AND "routes"."source_type" = 'Project' WHERE ("routes"."path" ILIKE '%/badges' OR "routes"."path" ILIKE 'badges' OR "routes"."path" ILIKE '%/blame' OR "routes"."path" ILIKE 'blame' OR "routes"."path" ILIKE '%/blob' OR "routes"."path" ILIKE 'blob' OR "routes"."path" ILIKE '%/builds' OR "routes"."path" ILIKE 'builds' OR "routes"."path" ILIKE '%/commits' OR "routes"."path" ILIKE 'commits' OR "routes"."path" ILIKE '%/create' OR "routes"."path" ILIKE 'create' OR "routes"."path" ILIKE '%/create_dir' OR "routes"."path" ILIKE 'create_dir' OR "routes"."path" ILIKE '%/edit' OR "routes"."path" ILIKE 'edit' OR "routes"."path" ILIKE '%/environments/folders' OR "routes"."path" ILIKE 'environments/folders' OR "routes"."path" ILIKE '%/files' OR "routes"."path" ILIKE 'files' OR "routes"."path" ILIKE '%/find_file' OR "routes"."path" ILIKE 'find_file' OR "routes"."path" ILIKE '%/gitlab-lfs/objects' OR "routes"."path" ILIKE 'gitlab-lfs/objects' OR "routes"."path" ILIKE '%/info/lfs/objects' OR "routes"."path" ILIKE 'info/lfs/objects' OR "routes"."path" ILIKE '%/new' OR "routes"."path" ILIKE 'new' OR "routes"."path" ILIKE '%/preview' OR "routes"."path" ILIKE 'preview' OR "routes"."path" ILIKE '%/raw' OR "routes"."path" ILIKE 'raw' OR "routes"."path" ILIKE '%/refs' OR "routes"."path" ILIKE 'refs' OR "routes"."path" ILIKE '%/tree' OR "routes"."path" ILIKE 'tree' OR "routes"."path" ILIKE '%/update' OR "routes"."path" ILIKE 'update' OR "routes"."path" ILIKE '%/wikis' OR "routes"."path" ILIKE 'wikis')	108
Top Level Groups	SELECT count(*) FROM "namespaces" INNER JOIN "routes" ON "routes"."source_id" = "namespaces"."id" AND "routes"."source_type" = 'Namespace' WHERE "namespaces"."parent_id" IS NULL AND ("routes"."path" ILIKE '%/-' OR "routes"."path" ILIKE '-' OR "routes"."path" ILIKE '%/.well-known' OR "routes"."path" ILIKE '.well-known' OR "routes"."path" ILIKE '%/abuse_reports' OR "routes"."path" ILIKE 'abuse_reports' OR "routes"."path" ILIKE '%/admin' OR "routes"."path" ILIKE 'admin' OR "routes"."path" ILIKE '%/all' OR "routes"."path" ILIKE 'all' OR "routes"."path" ILIKE '%/api' OR "routes"."path" ILIKE 'api' OR "routes"."path" ILIKE '%/assets' OR "routes"."path" ILIKE 'assets' OR "routes"."path" ILIKE '%/autocomplete' OR "routes"."path" ILIKE 'autocomplete' OR "routes"."path" ILIKE '%/ci' OR "routes"."path" ILIKE 'ci' OR "routes"."path" ILIKE '%/dashboard' OR "routes"."path" ILIKE 'dashboard' OR "routes"."path" ILIKE '%/explore' OR "routes"."path" ILIKE 'explore' OR "routes"."path" ILIKE '%/files' OR "routes"."path" ILIKE 'files' OR "routes"."path" ILIKE '%/groups' OR "routes"."path" ILIKE 'groups' OR "routes"."path" ILIKE '%/health_check' OR "routes"."path" ILIKE 'health_check' OR "routes"."path" ILIKE '%/help' OR "routes"."path" ILIKE 'help' OR "routes"."path" ILIKE '%/hooks' OR "routes"."path" ILIKE 'hooks' OR "routes"."path" ILIKE '%/import' OR "routes"."path" ILIKE 'import' OR "routes"."path" ILIKE '%/invites' OR "routes"."path" ILIKE 'invites' OR "routes"."path" ILIKE '%/issues' OR "routes"."path" ILIKE 'issues' OR "routes"."path" ILIKE '%/jwt' OR "routes"."path" ILIKE 'jwt' OR "routes"."path" ILIKE '%/koding' OR "routes"."path" ILIKE 'koding' OR "routes"."path" ILIKE '%/member' OR "routes"."path" ILIKE 'member' OR "routes"."path" ILIKE '%/merge_requests' OR "routes"."path" ILIKE 'merge_requests' OR "routes"."path" ILIKE '%/new' OR "routes"."path" ILIKE 'new' OR "routes"."path" ILIKE '%/notes' OR "routes"."path" ILIKE 'notes' OR "routes"."path" ILIKE '%/notification_settings' OR "routes"."path" ILIKE 'notification_settings' OR "routes"."path" ILIKE '%/oauth' OR "routes"."path" ILIKE 'oauth' OR "routes"."path" ILIKE '%/profile' OR "routes"."path" ILIKE 'profile' OR "routes"."path" ILIKE '%/projects' OR "routes"."path" ILIKE 'projects' OR "routes"."path" ILIKE '%/public' OR "routes"."path" ILIKE 'public' OR "routes"."path" ILIKE '%/repository' OR "routes"."path" ILIKE 'repository' OR "routes"."path" ILIKE '%/robots.txt' OR "routes"."path" ILIKE 'robots.txt' OR "routes"."path" ILIKE '%/s' OR "routes"."path" ILIKE 's' OR "routes"."path" ILIKE '%/search' OR "routes"."path" ILIKE 'search' OR "routes"."path" ILIKE '%/sent_notifications' OR "routes"."path" ILIKE 'sent_notifications' OR "routes"."path" ILIKE '%/services' OR "routes"."path" ILIKE 'services' OR "routes"."path" ILIKE '%/snippets' OR "routes"."path" ILIKE 'snippets' OR "routes"."path" ILIKE '%/teams' OR "routes"."path" ILIKE 'teams' OR "routes"."path" ILIKE '%/u' OR "routes"."path" ILIKE 'u' OR "routes"."path" ILIKE '%/unicorn_test' OR "routes"."path" ILIKE 'unicorn_test' OR "routes"."path" ILIKE '%/unsubscribes' OR "routes"."path" ILIKE 'unsubscribes' OR "routes"."path" ILIKE '%/uploads' OR "routes"."path" ILIKE 'uploads' OR "routes"."path" ILIKE '%/users' OR "routes"."path" ILIKE 'users')	20
Child Groups	SELECT count(*) FROM "namespaces" INNER JOIN "routes" ON "routes"."source_id" = "namespaces"."id" AND "routes"."source_type" = 'Namespace' WHERE ("namespaces"."parent_id" IS NOT NULL) AND ("routes"."path" ILIKE '%/badges' OR "routes"."path" ILIKE 'badges' OR "routes"."path" ILIKE '%/blame' OR "routes"."path" ILIKE 'blame' OR "routes"."path" ILIKE '%/blob' OR "routes"."path" ILIKE 'blob' OR "routes"."path" ILIKE '%/builds' OR "routes"."path" ILIKE 'builds' OR "routes"."path" ILIKE '%/commits' OR "routes"."path" ILIKE 'commits' OR "routes"."path" ILIKE '%/create' OR "routes"."path" ILIKE 'create' OR "routes"."path" ILIKE '%/create_dir' OR "routes"."path" ILIKE 'create_dir' OR "routes"."path" ILIKE '%/edit' OR "routes"."path" ILIKE 'edit' OR "routes"."path" ILIKE '%/environments/folders' OR "routes"."path" ILIKE 'environments/folders' OR "routes"."path" ILIKE '%/files' OR "routes"."path" ILIKE 'files' OR "routes"."path" ILIKE '%/find_file' OR "routes"."path" ILIKE 'find_file' OR "routes"."path" ILIKE '%/gitlab-lfs/objects' OR "routes"."path" ILIKE 'gitlab-lfs/objects' OR "routes"."path" ILIKE '%/info/lfs/objects' OR "routes"."path" ILIKE 'info/lfs/objects' OR "routes"."path" ILIKE '%/new' OR "routes"."path" ILIKE 'new' OR "routes"."path" ILIKE '%/preview' OR "routes"."path" ILIKE 'preview' OR "routes"."path" ILIKE '%/raw' OR "routes"."path" ILIKE 'raw' OR "routes"."path" ILIKE '%/refs' OR "routes"."path" ILIKE 'refs' OR "routes"."path" ILIKE '%/tree' OR "routes"."path" ILIKE 'tree' OR "routes"."path" ILIKE '%/update' OR "routes"."path" ILIKE 'update' OR "routes"."path" ILIKE '%/wikis' OR "routes"."path" ILIKE 'wikis')	4

The queries here were timing out on production, but we've devised separate queries to determine the number of affected rows above (see https://gitlab.com/gitlab-com/infrastructure/issues/1873 for more info). To avoid these timeouts, we'll do the rename of each invalid word separately.

Closes #32625 (closed)