Add latest changes from gitlab-org/gitlab@master

doc/development/documentation/index.md

@@ -152,20 +152,6 @@ disqus_identifier: 'https://docs.gitlab.com/my-old-location/README.html'
 Note: it is necessary to include the file name in the `disqus_identifier` URL,
 even if it's `index.html` or `README.html`.
-## Branch naming
-
-If your contribution contains **only** documentation changes, you can speed up
-the CI process by following these branch naming conventions:
-
-| Branch name           | Valid example                |
-|:----------------------|:-----------------------------|
-| Starting with `docs/` | `docs/update-api-issues`     |
-| Starting with `docs-` | `docs-update-api-issues`     |
-| Ending in `-docs`     | `123-update-api-issues-docs` |
-
-If your branch name matches any of the above, it will run only the docs
-tests. If not, the whole application test suite will run (including docs tests).
-
 ## Merge requests for GitLab documentation
 Before getting started, make sure you read the introductory section
@@ -173,7 +159,6 @@ Before getting started, make sure you read the introductory section
 [documentation workflow](workflow.md).
 - Use the current [merge request description template](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/merge_request_templates/Documentation.md)
-- Use the correct [branch name](#branch-naming)
 - Label the MR `Documentation`
 - Assign the correct milestone (see note below)
@@ -283,10 +268,6 @@ Several [rspec tests](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/feat
 are run to ensure GitLab documentation renders and works correctly. In particular, that [main docs landing page](../../README.md) will work correctly from `/help`.
 For example, [GitLab.com's `/help`](https://gitlab.com/help).
-CAUTION: **Caution:**
-Because the rspec tests only run in a full pipeline, and not a special [docs-only pipeline](#branch-naming), it is possible
-to merge changes that will break `master` from a merge request with a successful docs-only pipeline run.
-
 ## Docs site architecture
 See the [Docs site architecture](site_architecture/index.md) page to learn
@@ -309,14 +290,9 @@ The live preview is currently enabled for the following projects:
 - <https://gitlab.com/gitlab-org/gitlab>
 - <https://gitlab.com/gitlab-org/gitlab-runner>
-If your branch contains only documentation changes, you can use
-[special branch names](#branch-naming) to avoid long-running pipelines.
-
-For [docs-only changes](#branch-naming), the review app is run automatically.
-For all other branches, you can use the manual `review-docs-deploy-manual` job
-in your merge request. You will need at least Maintainer permissions to be able
-to run it. In the mini pipeline graph, you should see a `>>` icon. Clicking it will
-reveal the `review-docs-deploy-manual` job. Click the play button to start the job.
+If your merge request has docs changes, you can use the manual `review-docs-deploy` job
+to deploy the docs review app for your merge request.
+You will need at least Maintainer permissions to be able to run it.
 ![Manual trigger a docs build](img/manual_build_docs.png)

doc/development/pipelines.md

+# Pipelines for the GitLab project
+
+Pipelines for `gitlab-org/gitlab` and `gitlab-org/gitlab-foss` (as well as the
+`dev` instance's mirrors) are configured in the usual
+[`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml)
+which itself includes files under
+[`.gitlab/ci/`](https://gitlab.com/gitlab-org/gitlab/tree/master/.gitlab/ci)
+for easier maintenance.
+
+We're striving to [dogfood](https://about.gitlab.com/handbook/engineering/#dogfooding)
+GitLab [CI/CD features and best-practices](../ci/yaml/README.md)
+as much as possible.
+
+## Stages
+
+The current stages are:
+
+- `prepare`: This stage includes jobs that prepare artifacts that are needed by
+  jobs in subsequent stages.
+- `quick-test`: This stage includes test jobs that should run first and fail the
+  pipeline early (currently used to run Geo tests when the branch name starts
+  with `geo-`, `geo/`, or ends with `-geo`).
+- `test`: This stage includes most of the tests, DB/migration jobs, and static analysis jobs.
+- `review-prepare`: This stage includes a job that build the CNG images that are
+  later used by the (Helm) Review App deployment (see
+  [Review Apps](testing_guide/review_apps.md) for details).
+- `review`: This stage includes jobs that deploy the GitLab and Docs Review Apps.
+- `qa`: This stage includes jobs that perform QA tasks against the Review App
+  that is deployed in the previous stage.
+- `post-test`: This stage includes jobs that build reports or gather data from
+  the previous stages' jobs (e.g. coverage, Knapsack metadata etc.).
+- `pages`: This stage includes a job that deploys the various reports as
+  GitLab pages (e.g. <https://gitlab-org.gitlab.io/gitlab/coverage-ruby/>,
+  <https://gitlab-org.gitlab.io/gitlab/coverage-javascript/>,
+  <https://gitlab-org.gitlab.io/gitlab/webpack-report/>).
+
+## Default image
+
+The default image is currently
+`dev.gitlab.org:5005/gitlab/gitlab-build-images:ruby-2.6.3-golang-1.11-git-2.22-chrome-73.0-node-12.x-yarn-1.16-postgresql-9.6-graphicsmagick-1.3.33`.
+It includes Ruby 2.6.3, Go 1.11, Git 2.22, Chrome 73, Node 12, Yarn 1.16,
+PostgreSQL 9.6, and Graphics Magick 1.3.33.
+
+The images used in our pipelines are configured in the
+[`gitlab-org/gitlab-build-images`](https://gitlab.com/gitlab-org/gitlab-build-images)
+project, which is push-mirrored to <https://dev.gitlab.org/gitlab/gitlab-build-images>
+for redundancy.
+
+The current version of the build images can be found in the
+["Used by GitLab CE/EE section"](https://gitlab.com/gitlab-org/gitlab-build-images/blob/master/.gitlab-ci.yml).
+
+## Default variables
+
+In addition to the [predefined variables](../ci/variables/predefined_variables.md),
+each pipeline includes the following [variables](../ci/variables/README.md):
+
+- `RAILS_ENV: "test"`
+- `NODE_ENV: "test"`
+- `SIMPLECOV: "true"`
+- `GIT_DEPTH: "20"`
+- `GIT_SUBMODULE_STRATEGY: "none"`
+- `GET_SOURCES_ATTEMPTS: "3"`
+- `KNAPSACK_RSPEC_SUITE_REPORT_PATH: knapsack/${CI_PROJECT_NAME}/rspec_report-master.json`
+- `EE_KNAPSACK_RSPEC_SUITE_REPORT_PATH: knapsack/${CI_PROJECT_NAME}/rspec_report-master-ee.json`
+- `FLAKY_RSPEC_SUITE_REPORT_PATH: rspec_flaky/report-suite.json`
+- `BUILD_ASSETS_IMAGE: "false"`
+- `ES_JAVA_OPTS: "-Xms256m -Xmx256m"`
+- `ELASTIC_URL: "http://elastic:changeme@docker.elastic.co-elasticsearch-elasticsearch:9200"`
+
+## Common job definitions
+
+Most of the jobs [extend from a few CI definitions](../ci/yaml/README.md#extends)
+that are scoped to a single
+[configuration parameter](../ci/yaml/README.md#configuration-parameters).
+
+These common definitions are:
+
+- `.default-tags`: Ensures a job has the `gitlab-org` tag to ensure it's using
+  our dedicated runners.
+- `.default-retry`: Allows a job to retry upon `unknown_failure`, `api_failure`,
+  `runner_system_failure`.
+- `.default-before_script`: Allows a job to use a default `before_script` definition
+  suitable for Ruby/Rails tasks that may need a database running (e.g. tests).
+- `.default-cache`: Allows a job to use a default `cache` definition suitable for
+  Ruby/Rails and frontend tasks.
+- `.default-only`: Restricts the cases where a job is created. This currently
+  includes `master`, `/^[\d-]+-stable(-ee)?$/` (stable branches),
+  `/^\d+-\d+-auto-deploy-\d+$/` (security branches), `merge_requests`, `tags`.
+  Note that jobs won't be created for branches with this default configuration.
+- `.only-review`: Only creates a job for the `gitlab-org` namespace and if
+  Kubernetes integration is available. Also, prevents a job from being created
+  for `master` and auto-deploy branches.
+- `.only-review-schedules`: Same as `.only-review` but also restrict a job to
+  only run for [schedules](../user/project/pipelines/schedules.md).
+- `.use-pg`: Allows a job to use the `postgres:9.6.14` and `redis:alpine` services.
+- `.use-pg-10`: Allows a job to use the `postgres:10.9` and `redis:alpine` services.
+- `.only-ee`: Only creates a job for the `gitlab` project.
+
+## Changes detection
+
+If a job extends from `.default-only` (and most of the jobs should), it can restrict
+the cases where it should be created
+[based on the changes](../ci/yaml/README.md#onlychangesexceptchanges)
+from a commit or MR by extending from the following CI definitions:
+
+- `.only-code-changes`: Allows a job to only be created upon code-related changes.
+- `.only-qa-changes`: Allows a job to only be created upon QA-related changes.
+- `.only-docs-changes`: Allows a job to only be created upon docs-related changes.
+- `.only-code-qa-changes`: Allows a job to only be created upon code-related or QA-related changes.
+
+**See <https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/global.gitlab-ci.yml>
+for the list of exact patterns.**
+
+## Directed acyclic graph
+
+We're using the [`needs:`](../ci/yaml/README.md#needs) keyword to
+execute jobs out of order for the following jobs:
+
+```mermaid
+graph RL;
+  A[setup-test-env];
+  B["gitlab:assets:compile<br/>(master only)"];
+  C[gitlab:assets:compile pull-cache];
+  D["cache gems<br/>(master and tags only)"];
+  E[review-build-cng];
+  F[build-qa-image];
+  G[review-deploy];
+  G2["schedule:review-deploy<br/>(master only)"];
+  H[karma];
+  I[jest];
+  J["compile-assets<br/>(master only)"];
+  K[compile-assets pull-cache];
+  L[webpack-dev-server];
+  M[coverage];
+  N[pages];
+  O[static-analysis];
+  P["package-and-qa-manual:master<br/>(master schedule only)"];
+  Q[package-and-qa];
+  R[package-and-qa-manual];
+
+subgraph "`prepare` stage"
+    A
+    F
+    J
+    K
+    end
+
+subgraph "`test` stage"
+    B --> |needs| A;
+    C --> |needs| A;
+    D --> |needs| A;
+    H -.-> |depends on| A;
+    H -.-> |depends on| J;
+    H -.-> |depends on| K;
+    I -.-> |depends on| A;
+    I -.-> |depends on| J;
+    I -.-> |depends on| K;
+    L -.-> |depends on| A;
+    L -.-> |depends on| J;
+    L -.-> |depends on| K;
+    downtime_check --> |needs and depends on| A;
+    db:* --> |needs| A;
+    gitlab:setup --> |needs| A;
+    O -.-> |depends on| A;
+    O -.-> |depends on| B;
+    O -.-> |depends on| C;
+    downtime_check --> |needs and depends on| A;
+    end
+
+subgraph "`review-prepare` stage"
+    E --> |needs| C;
+    X["schedule:review-build-cng<br/>(master schedule only)"] --> |needs| B;
+    end
+
+subgraph "`review` stage"
+    G --> |needs| E;
+    G2 --> |needs| E;
+    end
+
+subgraph "`qa` stage"
+    Q --> |needs| C;
+    Q --> |needs| F;
+    R --> |needs| C;
+    R --> |needs| F;
+    P --> |needs| B;
+    P --> |needs| F;
+    review-qa-smoke -.-> |depends on| G;
+    review-qa-all -.-> |depends on| G;
+    review-qa-performance -.-> |depends on| G;
+    X2["schedule:review-performance<br/>(master only)"] -.-> |depends on| G2;
+    dast -.-> |depends on| G;
+    end
+
+subgraph "`post-test` stage"
+    M
+    end
+
+subgraph "`pages` stage"
+    N -.-> |depends on| B;
+    N -.-> |depends on| H;
+    N -.-> |depends on| M;
+    end
+```
+
+## Test jobs
+
+Consult [GitLab tests in the Continuous Integration (CI) context](testing_guide/ci.md)
+for more information.
+
+## Review app jobs
+
+Consult the [Review Apps](testing_guide/review_apps.md) dedicated page for more information.
+
+---
+
+[Return to Development documentation](README.md)

doc/development/testing_guide/end_to_end/index.md

@@ -7,24 +7,6 @@ as expected across the entire software stack and architecture, including
 integration of all micro-services and components that are supposed to work
 together.
-## Branch naming
-
-If your contribution contains **only** changes under the
-[`qa/` folder](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa), you can
-speed up the CI process by following some branch naming conventions. You have
-three choices:
-
-| Branch name           | Valid example                |
-|:----------------------|:-----------------------------|
-| Starting with `qa/`   | `qa/new-oauth-login-test`     |
-| Starting with `qa-`   | `qa-new-oauth-login-test`     |
-| Ending in `-qa`       | `123-new-oauth-login-test-qa` |
-
-If your branch name matches any of the above, it will run only the QA-related
-jobs.
-If it does not, the whole application test suite will run (including QA-related
-jobs).
-
 ## How do we test GitLab?
 We use [Omnibus GitLab][omnibus-gitlab] to build GitLab packages and then we

lib/gitlab_danger.rb

@@ -21,7 +21,6 @@ class GitlabDanger
     single_codebase
     gitlab_ui_wg
     ce_ee_vue_templates
-    only_documentation
   ].freeze
   MESSAGE_PREFIX = '==>'.freeze

locale/gitlab.pot

@@ -4656,9 +4656,6 @@ msgstr ""
 msgid "Days"
 msgstr ""
-msgid "Days to merge"
-msgstr ""
-
 msgid "Debug"
 msgstr ""
@@ -5845,6 +5842,9 @@ msgstr ""
 msgid "Epics Roadmap"
 msgstr ""
+msgid "Epics and Issues"
+msgstr ""
+
 msgid "Epics let you manage your portfolio of projects more efficiently and with less effort"
 msgstr ""
@@ -5896,6 +5896,9 @@ msgstr ""
 msgid "Epics|Something went wrong while fetching group epics."
 msgstr ""
+msgid "Epics|Something went wrong while ordering item."
+msgstr ""
+
 msgid "Epics|Something went wrong while removing issue from epic."
 msgstr ""
@@ -11461,6 +11464,9 @@ msgstr ""
 msgid "ProductivityAnalytics|Ascending"
 msgstr ""
+msgid "ProductivityAnalytics|Days to merge"
+msgstr ""
+
 msgid "ProductivityAnalytics|Descending"
 msgstr ""
@@ -16376,9 +16382,6 @@ msgstr ""
 msgid "TransferProject|Transfer failed, please contact an admin."
 msgstr ""
-msgid "Tree"
-msgstr ""
-
 msgid "Tree view"
 msgstr ""

package.json

@@ -115,7 +115,7 @@
     "select2": "3.5.2-browserify",
     "sha1": "^1.1.1",
     "smooshpack": "^0.0.54",
-    "sortablejs": "^1.7.0",
+    "sortablejs": "^1.10.0",
     "sql.js": "^0.4.0",
     "stickyfilljs": "^2.0.5",
     "style-loader": "^0.23.1",
@@ -137,6 +137,7 @@
     "vue-router": "^3.0.2",
     "vue-template-compiler": "^2.6.10",
     "vue-virtual-scroll-list": "^1.3.1",
+    "vuedraggable": "^2.23.0",
     "vuex": "^3.1.0",
     "webpack": "^4.29.0",
     "webpack-bundle-analyzer": "^3.3.2",

spec/bin/changelog_spec.rb

@@ -69,6 +69,14 @@ describe 'bin/changelog' do
         end
       end
+      it 'parses --ee and -e' do
+        %w[--ee -e].each do |flag|
+          options = described_class.parse(%W[foo #{flag} security])
+
+          expect(options.ee).to eq true
+        end
+      end
+
       it 'parses -h' do
         expect do
           expect { described_class.parse(%w[foo -h bar]) }.to output.to_stdout

spec/frontend/jobs/store/utils_spec.js

@@ -28,6 +28,12 @@ describe('Jobs Store Utils', () => {
         content: [{ text: 'Pulling docker image postgres:9.6.14 ...', style: 'term-fg-l-green' }],
         sections: ['prepare-executor'],
       },
+      {
+        offset: 1005,
+        content: [],
+        sections: ['prepare-executor'],
+        section_duration: '10:00',
+      },
     ];
     let result;
@@ -58,6 +64,16 @@ describe('Jobs Store Utils', () => {
         expect(result[1].lines[1].content).toEqual(mockData[3].content);
       });
     });
+
+    describe('section duration', () => {
+      it('adds the section information to the header section', () => {
+        expect(result[1].section_duration).toEqual(mockData[4].section_duration);
+      });
+
+      it('does not add section duration as a line', () => {
+        expect(result[1].lines.includes(mockData[4])).toEqual(false);
+      });
+    });
   });
   describe('updateIncrementalTrace', () => {

spec/javascripts/jobs/components/log/duration_badge_spec.js

+import { shallowMount } from '@vue/test-utils';
+import DurationBadge from '~/jobs/components/log/duration_badge.vue';
+
+describe('Job Log Duration Badge', () => {
+  let wrapper;
+
+  const data = {
+    duration: '00:30:01',
+  };
+
+  const createComponent = (props = {}) => {
+    wrapper = shallowMount(DurationBadge, {
+      sync: false,
+      propsData: {
+        ...props,
+      },
+    });
+  };
+
+  beforeEach(() => {
+    createComponent(data);
+  });
+
+  afterEach(() => {
+    wrapper.destroy();
+  });
+
+  it('renders provided duration', () => {
+    expect(wrapper.text()).toBe(data.duration);
+  });
+});

spec/javascripts/jobs/components/log/line_header_spec.js

 import { mount } from '@vue/test-utils';
 import LineHeader from '~/jobs/components/log/line_header.vue';
 import LineNumber from '~/jobs/components/log/line_number.vue';
+import DurationBadge from '~/jobs/components/log/duration_badge.vue';
 describe('Job Log Header Line', () => {
   let wrapper;
@@ -81,4 +82,14 @@ describe('Job Log Header Line', () => {
       expect(wrapper.emitted().toggleLine.length).toBe(1);
     });
   });
+
+  describe('with duration', () => {
+    beforeEach(() => {
+      createComponent(Object.assign({}, data, { duration: '00:10' }));
+    });
+
+    it('renders the duration badge', () => {
+      expect(wrapper.contains(DurationBadge)).toBe(true);
+    });
+  });
 });

spec/services/ci/create_pipeline_service_spec.rb

@@ -287,51 +287,21 @@ describe Ci::CreatePipelineService do
               expect(pipeline.builds.find_by(name: 'rspec').interruptible).to be_falsy
             end
           end
-
-          context 'not defined, but an environment is' do
-            before do
-              config = YAML.dump(rspec: { script: 'echo', environment: { name: "review/$CI_COMMIT_REF_NAME" } })
-              stub_ci_pipeline_yaml_file(config)
-            end
-
-            it 'is not cancelable' do
-              pipeline = execute_service
-
-              expect(pipeline.builds.find_by(name: 'rspec').interruptible).to be_nil
-            end
-          end
-
-          context 'overriding the environment definition' do
-            before do
-              config = YAML.dump(rspec: { script: 'echo', environment: { name: "review/$CI_COMMIT_REF_NAME" }, interruptible: true })
-              stub_ci_pipeline_yaml_file(config)
-            end
-
-            it 'is cancelable' do
-              pipeline = execute_service
-
-              expect(pipeline.builds.find_by(name: 'rspec').interruptible).to be_truthy
-            end
-          end
         end
         context 'interruptible builds' do
           before do
-            Feature.enable(:ci_support_interruptible_pipelines)
             stub_ci_pipeline_yaml_file(YAML.dump(config))
           end
-          after do
-            Feature.disable(:ci_support_interruptible_pipelines)
-          end
-
           let(:config) do
             {
               stages: %w[stage1 stage2 stage3 stage4],
               build_1_1: {
                 stage: 'stage1',
-                script: 'echo'
+                script: 'echo',
+                interruptible: true
               },
               build_1_2: {
                 stage: 'stage1',
@@ -342,7 +312,8 @@ describe Ci::CreatePipelineService do
                 stage: 'stage2',
                 script: 'echo',
                 when: 'delayed',
-                start_in: '10 minutes'
+                start_in: '10 minutes',
+                interruptible: true
               },
               build_3_1: {
                 stage: 'stage3',
@@ -364,9 +335,9 @@ describe Ci::CreatePipelineService do
                 .pluck(:name, 'ci_builds_metadata.interruptible')
             expect(interruptible_status).to contain_exactly(
-              ['build_1_1', nil],
+              ['build_1_1', true],
               ['build_1_2', true],
-              ['build_2_1', nil],
+              ['build_2_1', true],
               ['build_3_1', false],
               ['build_4_1', nil]
             )

spec/views/help/index.html.haml_spec.rb

@@ -31,7 +31,7 @@ describe 'help/index' do
         render
         expect(rendered).to match '8.0.2'
-        expect(rendered).to have_link('8.0.2', href: %r{https://gitlab.com/gitlab-org/gitlab-(ce|ee)/-/tags/v8.0.2})
+        expect(rendered).to have_link('8.0.2', href: %r{https://gitlab.com/gitlab-org/(gitlab|gitlab-foss)/-/tags/v8.0.2})
       end
       it 'shows a link to the commit for pre-releases' do
@@ -40,7 +40,7 @@ describe 'help/index' do
         render
         expect(rendered).to match '8.0.2'
-        expect(rendered).to have_link('abcdefg', href: %r{https://gitlab.com/gitlab-org/gitlab-(ce|ee)/commits/abcdefg})
+        expect(rendered).to have_link('abcdefg', href: %r{https://gitlab.com/gitlab-org/(gitlab|gitlab-foss)/commits/abcdefg})
       end
     end
   end

yarn.lock

@@ -11122,10 +11122,15 @@ sort-keys@^2.0.0:
   dependencies:
     is-plain-obj "^1.0.0"
-sortablejs@^1.7.0:
-  version "1.7.0"
-  resolved "https://registry.yarnpkg.com/sortablejs/-/sortablejs-1.7.0.tgz#80a2b2370abd568e1cec8c271131ef30a904fa28"
-  integrity sha1-gKKyNwq9Vo4c7IwnETHvMKkE+ig=
+sortablejs@^1.10.0:
+  version "1.10.0"
+  resolved "https://registry.yarnpkg.com/sortablejs/-/sortablejs-1.10.0.tgz#0ebc054acff2486569194a2f975b2b145dd5e7d6"
+  integrity sha512-+e0YakK1BxgEZpf9l9UiFaiQ8ZOBn1p/4qkkXr8QDVmYyCrUDTyDRRGm0AgW4E4cD0wtgxJ6yzIRkSPUwqhuhg==
+
+sortablejs@^1.9.0:
+  version "1.9.0"
+  resolved "https://registry.yarnpkg.com/sortablejs/-/sortablejs-1.9.0.tgz#2d1e74ae6bac2cb4ad0622908f340848969eb88d"
+  integrity sha512-Ot6bYJ6PoqPmpsqQYXjn1+RKrY2NWQvQt/o4jfd/UYwVWndyO5EPO8YHbnm5HIykf8ENsm4JUrdAvolPT86yYA==
 source-list-map@^2.0.0:
   version "2.0.0"
@@ -12793,6 +12798,13 @@ vue@^2.6.10:
   resolved "https://registry.yarnpkg.com/vue/-/vue-2.6.10.tgz#a72b1a42a4d82a721ea438d1b6bf55e66195c637"
   integrity sha512-ImThpeNU9HbdZL3utgMCq0oiMzAkt1mcgy3/E6zWC/G6AaQoeuFdsl9nDhTDU3X1R6FK7nsIUuRACVcjI+A2GQ==
+vuedraggable@^2.23.0:
+  version "2.23.0"
+  resolved "https://registry.yarnpkg.com/vuedraggable/-/vuedraggable-2.23.0.tgz#1f4a5a601675a5dbf0d96ee61aebfffa43445262"
+  integrity sha512-RgdH16k43WNoxyRcv/OarB/DZh9SY5TYthk9TS4YiHXpelD1DytEG0phLAXiXx5EhsmdH8ltSWxklGa4g1WTCw==
+  dependencies:
+    sortablejs "^1.9.0"
+
 vuex@^3.1.0:
   version "3.1.0"
   resolved "https://registry.yarnpkg.com/vuex/-/vuex-3.1.0.tgz#634b81515cf0cfe976bd1ffe9601755e51f843b9"