Add latest changes from gitlab-org/gitlab@master

doc/administration/high_availability/consul.md

@@ -158,7 +158,7 @@ To fix this:
 ### Outage recovery
-If you lost enough server agents in the cluster to break quorum, then the cluster is considered failed, and it will not function without manual intervenetion.
+If you lost enough server agents in the cluster to break quorum, then the cluster is considered failed, and it will not function without manual intervention.
 #### Recreate from scratch

doc/administration/high_availability/gitlab.md

@@ -99,14 +99,14 @@ these additional steps before proceeding with GitLab installation.
 ## First GitLab application server
-As a final step, run the setup rake task **only on** the first GitLab application server.
-Do not run this on additional application servers.
-1. Initialize the database by running `sudo gitlab-rake gitlab:setup`.
-1. Run `sudo gitlab-ctl reconfigure` to compile the configuration.
-   CAUTION: **WARNING:** Only run this setup task on **NEW** GitLab instances because it
-   will wipe any existing data.
+On the first application server, run:
+```sh
+sudo gitlab-ctl reconfigure
+```
+This should compile the configuration and initialize the database. Do
+not run this on additional application servers until the next step.
 ## Extra configuration for additional GitLab application servers

doc/ci/variables/predefined_variables.md

@@ -47,6 +47,7 @@ future GitLab releases.**
 | `CI_ENVIRONMENT_NAME`                   | 8.15   | all    | The name of the environment for this job. Only present if [`environment:name`](../yaml/README.md#environmentname) is set. |
 | `CI_ENVIRONMENT_SLUG`                   | 8.15   | all    | A simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, etc. Only present if [`environment:name`](../yaml/README.md#environmentname) is set. |
 | `CI_ENVIRONMENT_URL`                    | 9.3    | all    | The URL of the environment for this job. Only present if [`environment:url`](../yaml/README.md#environmenturl) is set. |
+| `CI_DEFAULT_BRANCH`                     | 12.4   | all    | The name of the default branch for the project. |
 | `CI_JOB_ID`                             | 9.0    | all    | The unique id of the current job that GitLab CI uses internally |
 | `CI_JOB_MANUAL`                         | 8.12   | all    | The flag to indicate that job was manually started |
 | `CI_JOB_NAME`                           | 9.0    | 0.5    | The name of the job as defined in `.gitlab-ci.yml` |

doc/topics/autodevops/index.md

@@ -487,6 +487,9 @@ in the first place, and thus not realize that it needs to re-apply the old confi
 > Introduced in [GitLab Ultimate][ee] 10.4.
+This is an optional step, since it requires a [review app](#auto-review-apps).
+If that requirement is not met, the job will be silently skipped.
+
 Dynamic Application Security Testing (DAST) uses the
 popular open source tool [OWASP ZAProxy](https://github.com/zaproxy/zaproxy)
 to perform an analysis on the current code and checks for potential security
@@ -498,6 +501,29 @@ later download and check out.
 Any security warnings are also shown in the merge request widget. Read how
 [DAST works](../../user/application_security/dast/index.md).
+On your default branch, DAST scans an app deployed specifically for that purpose.
+The app is deleted after DAST has run.
+
+On feature branches, DAST scans the [review app](#auto-review-apps).
+
+#### Overriding the DAST target
+
+To use a custom target instead of the auto-deployed review apps,
+set a `DAST_WEBSITE` environment variable to the URL for DAST to scan.
+
+NOTE: **Note:**
+If [DAST Full Scan](../../user/application_security/dast/index.md#full-scan) is enabled, it is strongly advised **not**
+to set `DAST_WEBSITE` to any staging or production environment. DAST Full Scan
+actively attacks the target, which can take down the application and lead to
+data loss or corruption.
+
+#### Disabling Auto DAST
+
+DAST can be disabled:
+
+- On all branches by setting the `DAST_DISABLED` environment variable to `"true"`.
+- Only on the default branch by setting the `DAST_DISABLED_FOR_DEFAULT_BRANCH` environment variable to `"true"`.
+
 ### Auto Browser Performance Testing **(PREMIUM)**
 > Introduced in [GitLab Premium][ee] 10.4.

doc/user/application_security/dast/index.md

@@ -81,8 +81,15 @@ variables:
 There are two ways to define the URL to be scanned by DAST:
-- Set the `DAST_WEBSITE` [variable](../../../ci/yaml/README.md#variables).
-- Add it in an `environment_url.txt` file at the root of your project.
+1. Set the `DAST_WEBSITE` [variable](../../../ci/yaml/README.md#variables).
+
+1. Add it in an `environment_url.txt` file at the root of your project.
+    This is great for testing in dynamic environments. In order to run DAST against
+    an app that is dynamically created during a Gitlab CI pipeline, have the app
+    persist its domain in an `environment_url.txt` file, and DAST will
+    automatically parse that file to find its scan target.
+    You can see an [example](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml)
+    of this in our Auto DevOps CI YML.
 If both values are set, the `DAST_WEBSITE` value will take precedence.

spec/support/helpers/test_env.rb

@@ -100,7 +100,6 @@ module TestEnv
     clean_test_path
-    # Set up GitLab shell for test instance
     setup_gitlab_shell
     setup_gitaly
@@ -145,10 +144,7 @@ module TestEnv
   end
   def setup_gitlab_shell
-    component_timed_setup('GitLab Shell',
-      install_dir: Gitlab.config.gitlab_shell.path,
-      version: Gitlab::Shell.version_required,
-      task: 'gitlab:shell:install')
+    FileUtils.mkdir_p(Gitlab.config.gitlab_shell.path)
   end
   def setup_gitaly

spec/tasks/gitlab/shell_rake_spec.rb

@@ -14,8 +14,10 @@ describe 'gitlab:shell rake tasks' do
       storages = Gitlab::GitalyClient::StorageSettings.allow_disk_access do
         Gitlab.config.repositories.storages.values.map(&:legacy_disk_path)
       end
-      expect(Kernel).to receive(:system).with('bin/install', *storages).and_call_original
-      expect(Kernel).to receive(:system).with('bin/compile').and_call_original
+
+      expect_any_instance_of(Gitlab::TaskHelpers).to receive(:checkout_or_clone_version)
+      allow(Kernel).to receive(:system).with('bin/install', *storages).and_return(true)
+      allow(Kernel).to receive(:system).with('bin/compile').and_return(true)
       run_rake_task('gitlab:shell:install')
     end