Separate bridge and DAG needs

Closes #14928 and #30675

See merge request gitlab-org/gitlab!17539

This reverts merge request !17539

Kamil Trzciński authored Oct 12, 2019 and Matija Čupić committed Oct 28, 2019

Use post compose validation:
- ensure that bridge has a valid number of entries
- use simplifiable to construct need type
- use aspect with opt to indicate allowed opts

This changes `Job`, `Bridge` and `Default`
to use `Inheritable` to inherit the configuration.

Shared example ensure that all fields are annotated
with `inherit:` and does have matching type when inherited.

Since the `Default:` is shared, it requires that
inheritable fields to be also added to `Job` and `Bridge`.

Refactors the nested array of string validations into separate
validators that can be reused.

Moves validation logic into the entry instead of having it in a separate
validator.

Flattens Entry::Commands elements so we can use YAML anchors as part of
the script definition.

* Allow user to specify `artifacts:expose_as` in CI config
* Save :has_exposed_artifacts in job metadata for queries
* Find exposed artifacts in build metadata model
* Expose API endpoint for frontend to fetch data

Fix unlrelated controller specs

Use default has_exposed_artifacts NULL

Avoid using a background migration to change NULL
to false. It's not needed.

Feedback from review

* add links to issue for follow up refactoring
* preload job artifacts and metadata associations
* merge DisallowedRegexInArrayValidator into existing
ArrayOfStringsValidator
* other minor changes

Rename params to match frontend code

Feedback from review

Feedback from review

Also fix specs to define the constants for the :entry class, not the
ancestor class Simplifiable

Don't have time to debug why specs are failing for these

There should be no cases where we need to inherit=true.

- Added Gitlab::Ci::Config::Entry::Rules and Gitlab::Ci::Config::Entry::Rules:Rule
  to handle lists of Rule objects to be evalauted for job inclusion
- Added `if:` and `changes:` as available Rules::Rule::Clause classes
- Added Rules handling logic to Seed::Build#included? with extra specs
- Use DisallowedKeysValidator to mutually exclude rules: from only:/except: on job config

- Added Gitlab::Ci::Config::Entry::Rules and Gitlab::Ci::Config::Entry::Rules:Rule
  to handle lists of Rule objects to be evalauted for job inclusion
- Added `if:` and `changes:` as available Rules::Rule::Clause classes
- Added Rules handling logic to Seed::Build#included? with extra specs
- Use DisallowedKeysValidator to mutually exclude rules: from only:/except: on job config

Since we are unsure what would be the behavior of `stage:`
when we work on DAG. Let's make `stage:` to be required
today with `needs:`.

Fabio Pitino authored Jul 02, 2019 and Marin Jankovski committed Jul 02, 2019

It keeps track of the memory being used when loading the YAML file
as well as the depth of nesting.
Track exception when YAML is too big

Fabio Pitino authored Jul 02, 2019 and Marin Jankovski committed Jul 02, 2019

It keeps track of the memory being used when loading the YAML file
as well as the depth of nesting.
Track exception when YAML is too big

Kamil Trzciński authored Jun 18, 2019 and Grzegorz Bizon committed Jun 18, 2019

This moves all existing `image/services/before_script/variables`
into `default:`. This allows us to easily add a default and
top-level entries. `default`: is keep backward compatible: to
be considered to be job if `default:script:` is specified. This
behavior should be removed.

All existing `image/services/before_script/variables` are properly
handled in root context.

Kamil Trzciński authored Jun 18, 2019 and Grzegorz Bizon committed Jun 18, 2019

This moves all existing `image/services/before_script/variables`
into `default:`. This allows us to easily add a default and
top-level entries. `default`: is keep backward compatible: to
be considered to be job if `default:script:` is specified. This
behavior should be removed.

All existing `image/services/before_script/variables` are properly
handled in root context.

Paul 🐻 authored Apr 05, 2019 and Grzegorz Bizon committed Apr 05, 2019

This fix is a followup to !24098 which introduced a validation of the
`include:` keyword of a gitlab-ci configuration file when triggered
from /ci/lint API calls.

However, there was a test case missing: the case of a single string as
value. I have added a test case for that which shows that the code was
not validating it correctly.

This commit fixes that to allow all `include:` valid inputs.

This brings support for untrusted regexp for 'only:refs:' when
enabled via feature flag: alllow_unsafe_ruby_regexp.

This is by default disabled, and should not be used in production

This brings support for untrusted regexp for 'only:refs:' when
enabled via feature flag: alllow_unsafe_ruby_regexp.

This is by default disabled, and should not be used in production

In order to implement https://gitlab.com/gitlab-org/gitlab-ee/issues/5276
we need several modifications on the CI config file.

We are adding a new ports section in the default Image object. Each
one of these ports will accept: number, protocol and name.

By default this new configuration will be only enabled in
the Web IDE config file.

In order to implement https://gitlab.com/gitlab-org/gitlab-ee/issues/10179
we need several modifications on the CI config file. We are
adding a new ports section in the default Image object.

Each of these ports will accept: number, protocol and name.

By default this new configuration will be only enabled in
the Web IDE config file.

This makes ref validation to use always `UntrustedRegexp`.
This also splits the existing RubySyntax into separate
class.

Introduce `default:` configuration entry setting that makes it possible
to configure a default value of an entry, what overrides class-level
`def self.default` value.

This commit refactors only/except policies so that these policies could
be self-contained. This also adds some changes to YAML configuration
library to provide more context to default entry value fabrication
process.

This decouples Ci::Config to provide a common interface for handling
user configuration files.