Merge branch 'cat-master-patch-51840' into 'master'

Geo database failover promotion pre-13.5 docs

See merge request gitlab-org/gitlab!62132

doc/administration/geo/disaster_recovery/index.md

@@ -98,6 +98,9 @@ Note the following when promoting a secondary:
 - If you encounter an `ActiveRecord::RecordInvalid: Validation failed: Name has already been taken`
   error message during this process, for more information, see this
   [troubleshooting advice](../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-node).
+- If you run into errors when using `--force` or `--skip-preflight-checks` before 13.5 during this process,
+  for more information, see this
+  [troubleshooting advice](../replication/troubleshooting.md#errors-when-using---skip-preflight-checks-or---force).
 #### Promoting a **secondary** node running on a single machine

doc/administration/geo/replication/troubleshooting.md

@@ -756,6 +756,30 @@ this command reports `ERROR - Replication is not up-to-date` even if
 replication is actually up-to-date. If replication and verification output
 shows that it is complete, you can add `--skip-preflight-checks` to make the command complete promotion. This bug was fixed in GitLab 13.8 and later.
+### Errors when using `--skip-preflight-checks` or `--force`
+
+Before GitLab 13.5, you could bump into one of the following errors when using
+`--skip-preflight-checks` or `--force`:
+
+```plaintext
+get_ctl_options': invalid option: --skip-preflight-checks (OptionParser::InvalidOption)
+
+get_ctl_options': invalid option: --force (OptionParser::InvalidOption)
+```
+
+This can happen with XFS or filesystems that list files in lexical order, because the
+load order of the Omnibus command files can be different than expected, and a global function would get redefined.
+More details can be found in [the related issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6076).
+
+The workaround is to manually run the preflight checks and promote the database, by running
+the following commands on the Geo secondary site:
+
+```shell
+sudo gitlab-ctl promotion-preflight-checks
+sudo /opt/gitlab/embedded/bin/gitlab-pg-ctl promote
+sudo gitlab-ctl reconfigure
+sudo gitlab-rake geo:set_secondary_as_primary
+
 ## Expired artifacts
 If you notice for some reason there are more artifacts on the Geo

doc/administration/geo/replication/version_specific_updates.md

@@ -82,6 +82,12 @@ paused fails. Do not pause replication before promoting a secondary. If the
 node is paused, be sure to resume before promoting. To avoid this issue,
 upgrade to GitLab 13.4 or later.
+WARNING:
+Promoting the database during a failover can fail on XFS and filesystems ordering files lexically,
+when using `--force` or `--skip-preflight-checks`, due to [an issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6076) fixed in 13.5.
+The [troubleshooting steps](troubleshooting.md#errors-when-using---skip-preflight-checks-or---force)
+contain a workaround if you run into errors during the failover.
+
 ## Updating to GitLab 13.2
 In GitLab 13.2, promoting a secondary node to a primary while the secondary is