Overhaul Omnibus Configuration (!662) · Merge requests · multiple-projects / imported-project-7304d3d5d908a902

James Nutt requested to merge dj-custom-omnibus-configurations into master Feb 25, 2021

Update references to configuring Omnibus

Design Considerations

Detection of competing configurations. E.g.,

gitlab_rails['artifacts_enabled'] = true
gitlab_rails['artifacts_enabled'] = false #=> competing configuration!  Will raise RuntimeError

Removal of duplicate configurations

gitlab_rails['artifacts_enabled'] = true #=> From Packages configurator
gitlab_rails['artifacts_enabled'] = true #=> From ObjectStorage configurator

The final omnibus configuration will contain only one artifacts_enabled line.

Sidecar containers

Sometimes, we need to run containers around the GitLab instance in question. The prepare method is dedicated to this. If multiple configurators are specified that have prepare overridden, it will run each container, run tests, then teardown the containers.

Multiple GitLab Instances

These configurators can be re-used, however running multiple GitLab Instances requires a separate Scenario. For example, Geo Primary and Secondary.

Example Configurations (/etc/gitlab/gitlab.rb)

Usage: gitlab-qa [options] Scenario URL [[--] path] [rspec_options]  
       --no-teardown                Skip teardown of containers after the scenario completes.  
       --no-tests                   Orchestrates the docker containers but does not run the tests. Implies --no-teardown  
       --omnibus-config config1[,config2,...]  
                                    Use Omnibus Configuration package  
   -v, --version                    Show the version  
   -h, --help                       Show the usage

Default

$ exe/gitlab-qa Test::Instance::Image EE

# Generated by GitLab QA Omnibus Configurator at 2021-02-24 23:15:36
gitlab_rails['gitlab_default_theme'] = 10 # Light Red Theme
gitlab_rails['gitlab_disable_animations'] = true
gitlab_rails['application_settings_cache_seconds'] = 0

infers Default

Packages

$ exe/gitlab-qa Test::Instance::Image EE --omnibus-config packages

# Generated by GitLab QA Omnibus Configurator at 2021-03-10 16:04:24
# Gitlab::QA::Runtime::OmnibusConfiguration::Default
gitlab_rails['gitlab_default_theme'] = 10 # Light Red Theme
gitlab_rails['gitlab_disable_animations'] = true # Disable animations
gitlab_rails['application_settings_cache_seconds'] = 0 # Settings cache expiry
# Gitlab::QA::Runtime::OmnibusConfiguration::Packages
gitlab_rails['packages_enabled'] = true

infers Default and tacks on Packages

ObjectStorage

$ exe/gitlab-qa Test::Instance::Image EE --omnibus-config object_storage

# Generated by GitLab QA Omnibus Configurator at 2021-03-10 16:13:35
# Gitlab::QA::Runtime::OmnibusConfiguration::Default
gitlab_rails['gitlab_default_theme'] = 10 # Light Red Theme
gitlab_rails['gitlab_disable_animations'] = true # Disable animations
gitlab_rails['application_settings_cache_seconds'] = 0 # Settings cache expiry
# Gitlab::QA::Runtime::OmnibusConfiguration::ObjectStorage
gitlab_rails['artifacts_enabled'] = true
gitlab_rails['artifacts_storage_path'] = '/var/opt/gitlab/gitlab-rails/shared/artifacts'
gitlab_rails['artifacts_object_store_enabled'] = true
gitlab_rails['artifacts_object_store_remote_directory'] = 'artifacts-bucket'
gitlab_rails['artifacts_object_store_background_upload'] = false
gitlab_rails['artifacts_object_store_direct_upload'] = true
gitlab_rails['artifacts_object_store_proxy_download'] = true
gitlab_rails['artifacts_object_store_connection'] = {'provider'=>'AWS', 'aws_access_key_id'=>'AKIAIOSFODNN7EXAMPLE', 'aws_secret_access_key'=>'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', 'aws_signature_version'=>4, 'host'=>'minio-59b1fd30.test', 'endpoint'=>'http://minio-59b1fd30.test:9000', 'path_style'=>true}
gitlab_rails['external_diffs_enabled'] = true
gitlab_rails['external_diffs_storage_path'] = '/var/opt/gitlab/gitlab-rails/shared/external_diffs'
gitlab_rails['external_diffs_object_store_enabled'] = true
gitlab_rails['external_diffs_object_store_remote_directory'] = 'external_diffs-bucket'
gitlab_rails['external_diffs_object_store_background_upload'] = false
gitlab_rails['external_diffs_object_store_direct_upload'] = true
gitlab_rails['external_diffs_object_store_proxy_download'] = true
gitlab_rails['external_diffs_object_store_connection'] = {'provider'=>'AWS', 'aws_access_key_id'=>'AKIAIOSFODNN7EXAMPLE', 'aws_secret_access_key'=>'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', 'aws_signature_version'=>4, 'host'=>'minio-59b1fd30.test', 'endpoint'=>'http://minio-59b1fd30.test:9000', 'path_style'=>true}
gitlab_rails['lfs_enabled'] = true
gitlab_rails['lfs_storage_path'] = '/var/opt/gitlab/gitlab-rails/shared/lfs'
gitlab_rails['lfs_object_store_enabled'] = true
gitlab_rails['lfs_object_store_remote_directory'] = 'lfs-bucket'
gitlab_rails['lfs_object_store_background_upload'] = false
gitlab_rails['lfs_object_store_direct_upload'] = true
gitlab_rails['lfs_object_store_proxy_download'] = true
gitlab_rails['lfs_object_store_connection'] = {'provider'=>'AWS', 'aws_access_key_id'=>'AKIAIOSFODNN7EXAMPLE', 'aws_secret_access_key'=>'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', 'aws_signature_version'=>4, 'host'=>'minio-59b1fd30.test', 'endpoint'=>'http://minio-59b1fd30.test:9000', 'path_style'=>true}
gitlab_rails['uploads_enabled'] = true
gitlab_rails['uploads_storage_path'] = '/var/opt/gitlab/gitlab-rails/shared/uploads'
gitlab_rails['uploads_object_store_enabled'] = true
gitlab_rails['uploads_object_store_remote_directory'] = 'uploads-bucket'
gitlab_rails['uploads_object_store_background_upload'] = false
gitlab_rails['uploads_object_store_direct_upload'] = true
gitlab_rails['uploads_object_store_proxy_download'] = true
gitlab_rails['uploads_object_store_connection'] = {'provider'=>'AWS', 'aws_access_key_id'=>'AKIAIOSFODNN7EXAMPLE', 'aws_secret_access_key'=>'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', 'aws_signature_version'=>4, 'host'=>'minio-59b1fd30.test', 'endpoint'=>'http://minio-59b1fd30.test:9000', 'path_style'=>true}
gitlab_rails['packages_enabled'] = true
gitlab_rails['packages_storage_path'] = '/var/opt/gitlab/gitlab-rails/shared/packages'
gitlab_rails['packages_object_store_enabled'] = true
gitlab_rails['packages_object_store_remote_directory'] = 'packages-bucket'
gitlab_rails['packages_object_store_background_upload'] = false
gitlab_rails['packages_object_store_direct_upload'] = true
gitlab_rails['packages_object_store_proxy_download'] = true
gitlab_rails['packages_object_store_connection'] = {'provider'=>'AWS', 'aws_access_key_id'=>'AKIAIOSFODNN7EXAMPLE', 'aws_secret_access_key'=>'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', 'aws_signature_version'=>4, 'host'=>'minio-59b1fd30.test', 'endpoint'=>'http://minio-59b1fd30.test:9000', 'path_style'=>true}
gitlab_rails['dependency_proxy_enabled'] = true
gitlab_rails['dependency_proxy_storage_path'] = '/var/opt/gitlab/gitlab-rails/shared/dependency_proxy'
gitlab_rails['dependency_proxy_object_store_enabled'] = true
gitlab_rails['dependency_proxy_object_store_remote_directory'] = 'dependency_proxy-bucket'
gitlab_rails['dependency_proxy_object_store_background_upload'] = false
gitlab_rails['dependency_proxy_object_store_direct_upload'] = true
gitlab_rails['dependency_proxy_object_store_proxy_download'] = true
gitlab_rails['dependency_proxy_object_store_connection'] = {'provider'=>'AWS', 'aws_access_key_id'=>'AKIAIOSFODNN7EXAMPLE', 'aws_secret_access_key'=>'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', 'aws_signature_version'=>4, 'host'=>'minio-59b1fd30.test', 'endpoint'=>'http://minio-59b1fd30.test:9000', 'path_style'=>true}

combines Default and ObjectStorage
starts Minio sidecar container before tests run
tears down Minio sidecar container after tests complete

Packages and ObjectStorage

$ exe/gitlab-qa Test::Instance::Image EE --omnibus-config packages,object_storage (or --omnibus-config packages --omnibus-config object_storage)

# Generated by GitLab QA Omnibus Configurator at 2021-03-10 16:15:50
# Gitlab::QA::Runtime::OmnibusConfiguration::Default
gitlab_rails['gitlab_default_theme'] = 10 # Light Red Theme
gitlab_rails['gitlab_disable_animations'] = true # Disable animations
gitlab_rails['application_settings_cache_seconds'] = 0 # Settings cache expiry
# Gitlab::QA::Runtime::OmnibusConfiguration::Packages
gitlab_rails['packages_enabled'] = true
# Gitlab::QA::Runtime::OmnibusConfiguration::ObjectStorage
gitlab_rails['artifacts_enabled'] = true
gitlab_rails['artifacts_storage_path'] = '/var/opt/gitlab/gitlab-rails/shared/artifacts'
gitlab_rails['artifacts_object_store_enabled'] = true
gitlab_rails['artifacts_object_store_remote_directory'] = 'artifacts-bucket'
gitlab_rails['artifacts_object_store_background_upload'] = false
gitlab_rails['artifacts_object_store_direct_upload'] = true
gitlab_rails['artifacts_object_store_proxy_download'] = true
gitlab_rails['artifacts_object_store_connection'] = {'provider'=>'AWS', 'aws_access_key_id'=>'AKIAIOSFODNN7EXAMPLE', 'aws_secret_access_key'=>'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', 'aws_signature_version'=>4, 'host'=>'minio-c892afc6.test', 'endpoint'=>'http://minio-c892afc6.test:9000', 'path_style'=>true}
gitlab_rails['external_diffs_enabled'] = true
gitlab_rails['external_diffs_storage_path'] = '/var/opt/gitlab/gitlab-rails/shared/external_diffs'
gitlab_rails['external_diffs_object_store_enabled'] = true
gitlab_rails['external_diffs_object_store_remote_directory'] = 'external_diffs-bucket'
gitlab_rails['external_diffs_object_store_background_upload'] = false
gitlab_rails['external_diffs_object_store_direct_upload'] = true
gitlab_rails['external_diffs_object_store_proxy_download'] = true
gitlab_rails['external_diffs_object_store_connection'] = {'provider'=>'AWS', 'aws_access_key_id'=>'AKIAIOSFODNN7EXAMPLE', 'aws_secret_access_key'=>'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', 'aws_signature_version'=>4, 'host'=>'minio-c892afc6.test', 'endpoint'=>'http://minio-c892afc6.test:9000', 'path_style'=>true}
gitlab_rails['lfs_enabled'] = true
gitlab_rails['lfs_storage_path'] = '/var/opt/gitlab/gitlab-rails/shared/lfs'
gitlab_rails['lfs_object_store_enabled'] = true
gitlab_rails['lfs_object_store_remote_directory'] = 'lfs-bucket'
gitlab_rails['lfs_object_store_background_upload'] = false
gitlab_rails['lfs_object_store_direct_upload'] = true
gitlab_rails['lfs_object_store_proxy_download'] = true
gitlab_rails['lfs_object_store_connection'] = {'provider'=>'AWS', 'aws_access_key_id'=>'AKIAIOSFODNN7EXAMPLE', 'aws_secret_access_key'=>'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', 'aws_signature_version'=>4, 'host'=>'minio-c892afc6.test', 'endpoint'=>'http://minio-c892afc6.test:9000', 'path_style'=>true}
gitlab_rails['uploads_enabled'] = true
gitlab_rails['uploads_storage_path'] = '/var/opt/gitlab/gitlab-rails/shared/uploads'
gitlab_rails['uploads_object_store_enabled'] = true
gitlab_rails['uploads_object_store_remote_directory'] = 'uploads-bucket'
gitlab_rails['uploads_object_store_background_upload'] = false
gitlab_rails['uploads_object_store_direct_upload'] = true
gitlab_rails['uploads_object_store_proxy_download'] = true
gitlab_rails['uploads_object_store_connection'] = {'provider'=>'AWS', 'aws_access_key_id'=>'AKIAIOSFODNN7EXAMPLE', 'aws_secret_access_key'=>'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', 'aws_signature_version'=>4, 'host'=>'minio-c892afc6.test', 'endpoint'=>'http://minio-c892afc6.test:9000', 'path_style'=>true}
gitlab_rails['packages_storage_path'] = '/var/opt/gitlab/gitlab-rails/shared/packages'
gitlab_rails['packages_object_store_enabled'] = true
gitlab_rails['packages_object_store_remote_directory'] = 'packages-bucket'
gitlab_rails['packages_object_store_background_upload'] = false
gitlab_rails['packages_object_store_direct_upload'] = true
gitlab_rails['packages_object_store_proxy_download'] = true
gitlab_rails['packages_object_store_connection'] = {'provider'=>'AWS', 'aws_access_key_id'=>'AKIAIOSFODNN7EXAMPLE', 'aws_secret_access_key'=>'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', 'aws_signature_version'=>4, 'host'=>'minio-c892afc6.test', 'endpoint'=>'http://minio-c892afc6.test:9000', 'path_style'=>true}
gitlab_rails['dependency_proxy_enabled'] = true
gitlab_rails['dependency_proxy_storage_path'] = '/var/opt/gitlab/gitlab-rails/shared/dependency_proxy'
gitlab_rails['dependency_proxy_object_store_enabled'] = true
gitlab_rails['dependency_proxy_object_store_remote_directory'] = 'dependency_proxy-bucket'
gitlab_rails['dependency_proxy_object_store_background_upload'] = false
gitlab_rails['dependency_proxy_object_store_direct_upload'] = true
gitlab_rails['dependency_proxy_object_store_proxy_download'] = true
gitlab_rails['dependency_proxy_object_store_connection'] = {'provider'=>'AWS', 'aws_access_key_id'=>'AKIAIOSFODNN7EXAMPLE', 'aws_secret_access_key'=>'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', 'aws_signature_version'=>4, 'host'=>'minio-c892afc6.test', 'endpoint'=>'http://minio-c892afc6.test:9000', 'path_style'=>true}

combines Default, Packages and ObjectStorage (in the order of arguments, --omnibus-config packages,object_storage)
runs Minio sidecar container
removes duplicated gitlab_rails['packages_enabled'] = true from ObjectStorage omnibus configuration

Invalid

$ exe/gitlab-qa Test::Instance::Image EE --omnibus-config foo

/bin/zsh -c "bash -c '/Users/ddavison/.rvm/bin/rvm ruby-2.6.3@gitlab-qa do /Users/ddavison/.rvm/rubies/ruby-2.6.3/bin/ruby /Users/ddavison/workspace/org/gitlab-qa/exe/gitlab-qa '\"'\"'Test::Instance::Image'\"'\"' EE --no-teardown --omnibus-config foo'"

Did you mean?  Gitlab::QA::Runtime::OmnibusConfigurations::Packages
Invalid Omnibus Configuration `package`. (RuntimeError)
Possible configurations: object_storage,packages,default"

Prevents gitlab-qa from running as it cannot load a desired omnibus configuration
Lists possible configurations in case of a typo

Edited Mar 25, 2021 by James Nutt

Admin message