Ian Baum authored 4 years ago and Valery Sizov committed 4 years ago

Implementation of PITR recovery before promotion

Closes gitlab#225173

See merge request gitlab-org/omnibus-gitlab!4636

(cherry picked from commit 9ae9fcf7)

691a1fa5 Geo: Implement PITR recovery before promotion of secondary node
1171279d Apply 1 suggestion(s) to 1 file(s)
402e18c6 Apply suggestion. Better lsn file check
fe0c04a3 Merge branch 'master' of gitlab.com:gitlab-org/omnibus-gitlab into pitr
e3f99908 Apply 1 suggestion(s) to 1 file(s)
1d5781dc Merge branch 'pitr' of gitlab.com:gitlab-org/omnibus-gitlab into pitr
7a34d0da Apply 1 suggestion(s) to 1 file(s)
3e9c8592 Apply suggestions
dbbccdee Merge branch 'pitr' of gitlab.com:gitlab-org/omnibus-gitlab into pitr
2ff5b783 Apply suggestions
8b9163ac Add suggestions
565f96c6 Apply suggestions
39b2465d Apply suggestions
c8f5d0ab Merge remote-tracking branch 'origin/master' into pitr
0ba306f4 Fix part of Geo::PromoteToPrimaryNode specs
90c50385 Make recovery_to_point_in_time tio return true
8dd17632 Merge remote-tracking branch 'origin/master' into pitr
1af3bf5a Separate Promote DB task
8a521139 Update specs for the command split
f068dfc3 Merge remote-tracking branch 'origin/master' into pitr
175a4efd Apply 1 suggestion(s) to 1 file(s)
b1af4039 Remove io/console

Robert Marshall authored 4 years ago and Ian Baum committed 4 years ago

- PGBouncer commands check for if the service is running using a status
  check. This does not always work, as the service can be running
  without listening. Changed to run a command that should always work so
  that it verifies that PGBouncer is both running and ready.

Related https://gitlab.com/gitlab-org/gitlab-orchestrator/-/issues/195



Signed-off-by: Robert Marshall <rmarshall@gitlab.com>

Ian Baum authored 4 years ago and Balasankar C committed 4 years ago

* Matching behavior in rails codebase
* This will be default behavior in 4.0

The query is missing a SELECT which causes the database to throw
invalid syntax errors when attempting to resume the replication.

task was relocated to gitlab:geo namespace

Aakriti Gupta authored 4 years ago and Ian Baum committed 4 years ago

- removing unused option confirm-removing-keys,
- improving description for skip-preflight-checks

It will allow promotion even if preflight checks
fail.

preflight checks

`get-postgresql-primary` command prints out the connection information
(IP address and port) of primary PostgreSQL node. This is useful to
identify the primary node in a clustered deployment, for example with
repmgr or Patroni.

This command requires a Consul agent on the same node that it is
running. It uses Consul DNS interface to lookup the SRV record for
PostgreSQL service.

- Update POC code to also call rake task that calls primary api to pause
- Add changelog

Relates to https://gitlab.com/gitlab-org/omnibus-gitlab/issues/35913

It will run preflight checks for promoting to
a primary node. `promote-to-primary-node` command
can include or optionally skip these checks.

Signed-off-by: Balasankar "Balu" C <balasankarc@autistici.org>

Signed-off-by: Balasankar "Balu" C <balasankar@gitlab.com>

Signed-off-by: Balasankar "Balu" C <balasankar@gitlab.com>

Signed-off-by: Balasankar "Balu" C <balasankar@gitlab.com>

Defaults to gitlabhq_production (backwards-compatible).

We needed this setting when a failover happen and the secondary wasn't
able to pull the latest segment from the current timeline. Setting to
latest allows the secondary to recovery in case this happens.

See https://www.postgresql.org/docs/9.6/static/continuous-archiving.html#BACKUP-TIMELINES
for more details.

Relates to gitlab-com/migration#407

We saw that on GitLab.com, turning off SSL compression while keeping SSL on
virtually eliminated replication lag while the site was operating. This
change won't affect a current Geo installation since this requires a change
in recovery.conf and a PostgreSQL restart on the secondary, but it will
take effect for new installations.

Closes https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3495

* Fix specs for pgbouncer
* Fix helper methods that weren't actually selecting specific attributes
* Update get_public_node_attributes to return an empty hash if the file
doesn't exist

chef_helper loads a lot of chef related testing libraries, and includes
a call to ohai to configure them.  This is not always needed when
testing library code/rake tasks.

When running examples directly from the command-line during development
we can see a speedup of around 1.5 seconds, by using the reduced
`spec_helper` rather than the full `chef_helper`.

Relates to:

* https://gitlab.com/gitlab-com/migration/issues/161
* https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3134