Geo: Post-migration services cause CI failures
In https://gitlab.com/gitlab-org/gitlab-ee/builds/11811087, we see the build fail in the ce-upstream
branch due to post-migrations:
$ [ "$SETUP_DB" != "true" ] || bundle exec rake geo:db:drop geo:db:create geo:db:schema:load geo:db:migrate
Database 'gitlabhq_geo_test' does not exist
WARNING: This version of GitLab depends on gitlab-shell 4.1.1, but you're running Unknown. Please update gitlab-shell.
rake aborted!
StandardError: An error has occurred, all later migrations canceled:
Mysql2::Error: Table 'gitlabhq_geo_test.ci_builds' doesn't exist: UPDATE ci_builds SET status = 'manual', allow_failure = true
WHERE ci_builds.when = 'manual' AND ci_builds.status = 'skipped';
As @ayufan points out, the config/initializers/0_post_deployment_migrations.rb
is run before geo.rake
has a chance to skip over the post migrations, which may cause the Rake task to run them earlier than expected. Note in the build trace the Geo DB does get set up at the end.
We worked around the problem by changing .gitlab-ci.yml
to use SKIP_POST_DEPLOYMENT_MIGRATIONS=1
, but this is a bit of a hack.
/cc: @vsizov