Skip to content

Test::Instance::Staging doesn't work for RC that aren't released yet

The Problem

As discussed in https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3588, we're not pushing QA images to Docker Hub until we publicly release an RC (the QA-Tag job in the Image-release stage in https://dev.gitlab.org/gitlab/omnibus-gitlab/pipelines/84363).

This is a problem for performing QA against staging since we want to run QA before a release is publicly available (for obvious reasons).

Solutions

  1. Build the QA image in the Images stage, in parallel to the building of the Docker image (e.g. https://dev.gitlab.org/gitlab/omnibus-gitlab/pipelines/84363).
  2. The current temporary Docker images (GitLab and QA images) naming is of the form 11.0.0-rc8.ee.0 which doesn't match with the GitLab version exposed at https://staging.gitlab.com/help or https://staging.gitlab.com/api/v4/version, which is of the form 11.0.0-rc8-ee. We should change the naming of these Docker images. => We shouldn't change this naming as it doesn't have any impact on the QA Docker image.
  3. We should make gitlab-qa RC-aware, meaning that if the release is an RC, it should use the QA image that is in the dev container registry, e.g. dev.gitlab.org:5005/gitlab/omnibus-gitlab/gitlab-ee-qa:11.0.0-rc8-ee instead of gitlab/gitlab-ee-qa:11.0.0-rc8-ee which doesn't exist on Docker Hub at this time.

Workaround

In the meantime, a workaround for 1. is to build and tag the QA image locally:

In gitlab-ee:

› git checkout v11.0.0-rc8-ee
› cd qa
› docker build -t gitlab/gitlab-ee-qa:11.0.0-rc8-ee .
› gitlab-qa Test::Instance::Any gitlab/gitlab-ee 11.0.0-rc8-ee https://staging.gitlab.com

Once 1. is done, we won't have to build and tag the QA image locally, we'll be able to use the dev image:

› docker login dev.gitlab.org:5005 --username remy --password-stdin $(cat ~/.gitlab-token-dev)
› gitlab-qa Test::Instance::Any dev.gitlab.org:5005/gitlab/omnibus-gitlab/gitlab-ee-qa 11.0.0-rc8-ee https://staging.gitlab.com

Once 3. is done we could just do gitlab-qa Test::Instance::Staging and that should work even for RC releases that aren't public yet.

/cc @balasankarc @meks

Edited by James Nutt