Add latest changes from gitlab-org/gitlab@master

changelogs/unreleased/enable-customizable-cycle-analytics.yml

+---
+title: Enable customizable_cycle_analytics feature flag by default
+merge_request: 27418
+author:
+type: changed

changelogs/unreleased/feat-mr-diff-coverage-visualisation.yml

+---
+title: Add Cobertura XML coverage visualization to merge request diff view
+merge_request: 21791
+author: Fabio Huser
+type: added

changelogs/unreleased/osw-support-opt-in-cluster-in-bg-jobs-script.yml

+---
+title: Support sidekiq-cluster supervision through bin/background_jobs
+merge_request: 27042
+author:
+type: added

changelogs/unreleased/validate-subnets-field.yml

+---
+title: Validate that users selects at least two subnets in EKS Form
+merge_request: 26936
+author:
+type: fixed

config/routes/merge_requests.rb

@@ -14,6 +14,7 @@ resources :merge_requests, concerns: :awardable, except: [:new, :create, :show],
     post :rebase
     get :test_reports
     get :exposed_artifacts
+    get :coverage_reports
     scope constraints: ->(req) { req.format == :json }, as: :json do
       get :commits

danger/telemetry/Dangerfile

 # frozen_string_literal: true
 TELEMETRY_CHANGED_FILES_MESSAGE = <<~MSG
-This merge request adds or changes files that require a
-review from the Data team and Telemetry team @gitlab-org/growth/telemetry.
-The specific group is mentioned in order to send a notification to team members.
+This merge request adds or changes files for which a
+review from the Data team and Telemetry team is recommended.
+@gitlab-org/growth/telemetry group is mentioned in order to notify team members.
 MSG
 usage_data_changed_files = git.modified_files.grep(%r{usage_data})
@@ -12,7 +12,7 @@ if usage_data_changed_files.any?
   warn format(TELEMETRY_CHANGED_FILES_MESSAGE)
   USAGE_DATA_FILES_MESSAGE = <<~MSG
-  The following files require a review from the [Data team and Telemetry team](https://gitlab.com/groups/gitlab-org/growth/telemetry/-/group_members?with_inherited_permissions=exclude):
+  For the following files, a review from the [Data team and Telemetry team](https://gitlab.com/groups/gitlab-org/growth/telemetry/-/group_members?with_inherited_permissions=exclude) is recommended:
   MSG
   markdown(USAGE_DATA_FILES_MESSAGE + helper.markdown_list(usage_data_changed_files))

db/migrate/20200311141053_add_ci_pipeline_schedules_to_plan_limits.rb

+# frozen_string_literal: true
+
+class AddCiPipelineSchedulesToPlanLimits < ActiveRecord::Migration[6.0]
+  include Gitlab::Database::MigrationHelpers
+
+  disable_ddl_transaction!
+
+  DOWNTIME = false
+
+  def up
+    add_column_with_default(:plan_limits, :ci_pipeline_schedules, :integer, default: 0, allow_null: false)
+  end
+
+  def down
+    remove_column(:plan_limits, :ci_pipeline_schedules)
+  end
+end

db/migrate/20200311141943_insert_ci_pipeline_schedules_plan_limits.rb

+# frozen_string_literal: true
+
+class InsertCiPipelineSchedulesPlanLimits < ActiveRecord::Migration[6.0]
+  DOWNTIME = false
+
+  def change
+    return unless Gitlab.com?
+
+    create_or_update_plan_limit('ci_pipeline_schedules', 'free', 10)
+    create_or_update_plan_limit('ci_pipeline_schedules', 'bronze', 50)
+    create_or_update_plan_limit('ci_pipeline_schedules', 'silver', 50)
+    create_or_update_plan_limit('ci_pipeline_schedules', 'gold', 50)
+  end
+
+  def down
+    return unless Gitlab.com?
+
+    create_or_update_plan_limit('ci_pipeline_schedules', 'free', 0)
+    create_or_update_plan_limit('ci_pipeline_schedules', 'bronze', 0)
+    create_or_update_plan_limit('ci_pipeline_schedules', 'silver', 0)
+    create_or_update_plan_limit('ci_pipeline_schedules', 'gold', 0)
+  end
+end

db/schema.rb

@@ -3148,6 +3148,7 @@ ActiveRecord::Schema.define(version: 2020_03_12_163407) do
     t.integer "project_hooks", default: 0, null: false
     t.integer "group_hooks", default: 0, null: false
     t.integer "ci_project_subscriptions", default: 0, null: false
+    t.integer "ci_pipeline_schedules", default: 0, null: false
     t.index ["plan_id"], name: "index_plan_limits_on_plan_id", unique: true
   end

doc/administration/instance_limits.md

@@ -109,6 +109,29 @@ Plan.default.limits.update!(ci_project_subscriptions: 500)
 NOTE: **Note:** Set the limit to `0` to disable it.
+### Number of pipeline schedules
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29566) in GitLab 12.10.
+
+The total number of pipeline schedules can be limited per project. This limit is
+checked each time a new pipeline schedule is created. If a new pipeline schedule
+would cause the total number of pipeline schedules to exceed the limit, the
+pipeline schedule will not be created.
+
+On GitLab.com, different limits are [defined per plan](../user/gitlab_com/index.md#gitlab-cicd),
+and they affect all projects under that plan.
+
+On self-managed instances ([GitLab Starter](https://about.gitlab.com/pricing/#self-managed)
+or higher tiers), this limit is defined for the `default` plan that affects all
+projects. By default, there is no limit.
+
+To set this limit on a self-managed installation, run the following in the
+[GitLab Rails console](https://docs.gitlab.com/omnibus/maintenance/#starting-a-rails-console-session):
+
+```ruby
+Plan.default.limits.update!(ci_pipeline_schedules: 100)
+```
+
 ## Environment data on Deploy Boards
 [Deploy Boards](../user/project/deploy_boards.md) load information from Kubernetes about

doc/api/api_resources.md

@@ -120,6 +120,7 @@ The following API resources are available outside of project and group contexts
 | [Events](events.md)                                | `/events`, `/users/:id/events` (also available for projects)            |
 | [Feature flags](features.md)                       | `/features`                                                             |
 | [Geo Nodes](geo_nodes.md) **(PREMIUM ONLY)**       | `/geo_nodes`                                                            |
+| [Group Activity Analytics](group_activity_analytics.md) **(STARTER)**  | `/analytics/group_activity/{issues_count | merge_requests_count}`  |
 | [Import repository from GitHub](import.md)         | `/import/github`                                                        |
 | [Issues](issues.md)                                | `/issues` (also available for groups and projects)                      |
 | [Issues Statistics](issues_statistics.md)          | `/issues_statistics` (also available for groups and projects)           |

doc/api/graphql/reference/gitlab_schema.graphql

@@ -1988,8 +1988,7 @@ type Epic implements Noteable {
   descendantCounts: EpicDescendantCount
   """
-  Total weight of open and closed issues in the epic and its descendants.
-  Available only when feature flag `unfiltered_epic_aggregates` is enabled.
+  Total weight of open and closed issues in the epic and its descendants
   """
   descendantWeightSum: EpicDescendantWeights

doc/api/graphql/reference/gitlab_schema.json

@@ -5847,7 +5847,7 @@
             },
             {
               "name": "descendantWeightSum",
-              "description": "Total weight of open and closed issues in the epic and its descendants. Available only when feature flag `unfiltered_epic_aggregates` is enabled.",
+              "description": "Total weight of open and closed issues in the epic and its descendants",
               "args": [
               ],

doc/api/graphql/reference/index.md

@@ -327,7 +327,7 @@ Represents an epic.
 | `closedAt` | Time | Timestamp of the epic's closure |
 | `createdAt` | Time | Timestamp of the epic's creation |
 | `descendantCounts` | EpicDescendantCount | Number of open and closed descendant epics and issues |
-| `descendantWeightSum` | EpicDescendantWeights | Total weight of open and closed issues in the epic and its descendants. Available only when feature flag `unfiltered_epic_aggregates` is enabled. |
+| `descendantWeightSum` | EpicDescendantWeights | Total weight of open and closed issues in the epic and its descendants |
 | `description` | String | Description of the epic |
 | `downvotes` | Int! | Number of downvotes the epic has received |
 | `dueDate` | Time | Due date of the epic |

doc/api/group_activity_analytics.md

+# Group Activity Analytics API
+
+> **Note:** This feature was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26460) in GitLab 12.9.
+
+## Get count of recently created issues for group
+
+```plaintext
+GET /analytics/group_activity/issues_count
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `group_path` | string | yes | Group path |
+
+Example request:
+
+```shell
+curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/analytics/group_activity/issues_count?group_path=gitlab-org
+```
+
+Example response:
+
+```json
+[
+  { issues_count : 10 }
+]
+```
+
+## Get count of recently created merge requests for group
+
+```plaintext
+GET /analytics/group_activity/merge_requests_count
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `group_path` | string | yes | Group path |
+
+Example request:
+
+```shell
+curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/analytics/group_activity/merge_requests_count?group_path=gitlab-org
+```
+
+Example response:
+
+```json
+[
+  { merge_requests_count : 10 }
+]
+```

doc/ci/yaml/README.md

@@ -107,7 +107,7 @@ The following table lists available parameters for jobs:
 | [`when`](#when)                                    | When to run job. Also available: `when:manual` and `when:delayed`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
 | [`environment`](#environment)                      | Name of an environment to which the job deploys. Also available: `environment:name`, `environment:url`, `environment:on_stop`, `environment:auto_stop_in` and `environment:action`.                                                                                                                                                                                                                                                                                                                                                                                                                                |
 | [`cache`](#cache)                                  | List of files that should be cached between subsequent runs. Also available: `cache:paths`, `cache:key`, `cache:untracked`, and `cache:policy`.                                                                                                                                                                                                                                                                                                                                                                                                                                         |
-| [`artifacts`](#artifacts)                          | List of files and directories to attach to a job on success. Also available: `artifacts:paths`, `artifacts:expose_as`, `artifacts:name`, `artifacts:untracked`, `artifacts:when`, `artifacts:expire_in`, `artifacts:reports`, and `artifacts:reports:junit`.<br><br>In GitLab [Enterprise Edition](https://about.gitlab.com/pricing/), these are available: `artifacts:reports:codequality`, `artifacts:reports:sast`, `artifacts:reports:dependency_scanning`, `artifacts:reports:container_scanning`, `artifacts:reports:dast`, `artifacts:reports:license_management`, `artifacts:reports:performance` and `artifacts:reports:metrics`. |
+| [`artifacts`](#artifacts)                          | List of files and directories to attach to a job on success. Also available: `artifacts:paths`, `artifacts:expose_as`, `artifacts:name`, `artifacts:untracked`, `artifacts:when`, `artifacts:expire_in`, `artifacts:reports`, `artifacts:reports:junit`, and `artifacts:reports:cobertura`.<br><br>In GitLab [Enterprise Edition](https://about.gitlab.com/pricing/), these are available: `artifacts:reports:codequality`, `artifacts:reports:sast`, `artifacts:reports:dependency_scanning`, `artifacts:reports:container_scanning`, `artifacts:reports:dast`, `artifacts:reports:license_management`, `artifacts:reports:performance` and `artifacts:reports:metrics`. |
 | [`dependencies`](#dependencies)                    | Restrict which artifacts are passed to a specific job by providing a list of jobs to fetch artifacts from.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
 | [`coverage`](#coverage)                            | Code coverage settings for a given job.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
 | [`retry`](#retry)                                  | When and how many times a job can be auto-retried in case of a failure.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
@@ -2286,6 +2286,18 @@ There are a couple of limitations on top of the [original dotenv rules](https://
 - It doesn't support empty lines and comments (`#`) in dotenv file.
 - It doesn't support quote escape, spaces in a quote, a new line expansion in a quote, in dotenv file.
+##### `artifacts:reports:cobertura`
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/3708) in GitLab 12.9.
+> Requires [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 and above.
+
+The `cobertura` report collects [Cobertura coverage XML files](../../user/project/merge_requests/test_coverage_visualization.md).
+The collected Cobertura coverage reports will be uploaded to GitLab as an artifact
+and will be automatically shown in merge requests.
+
+Cobertura was originally developed for Java, but there are many
+third party ports for other languages like JavaScript, Python, Ruby, etc.
+
 ##### `artifacts:reports:codequality` **(STARTER)**
 > Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above.

doc/user/gitlab_com/index.md

@@ -76,6 +76,7 @@ Below are the current settings regarding [GitLab CI/CD](../../ci/README.md).
 | Artifacts [expiry time](../../ci/yaml/README.md#artifactsexpire_in)   | kept forever           | deleted after 30 days unless otherwise specified    |
 | Scheduled Pipeline Cron | `*/5 * * * *` | `19 * * * *` |
 | [Max jobs in active pipelines](../../administration/instance_limits.md#number-of-jobs-in-active-pipelines) | `500` for Free tier, unlimited otherwise | Unlimited
+| [Max pipeline schedules in projects](../../administration/instance_limits.md#number-of-pipeline-schedules) | `10` for Free tier, `50` for all paid tiers | Unlimited |
 ## Repository size limit

doc/user/project/merge_requests/index.md

@@ -102,6 +102,7 @@ or link to useful information directly in the merge request page:
 | [Multi-Project pipelines](../../../ci/multi_project_pipelines.md) **(PREMIUM)**                        | When you set up GitLab CI/CD across multiple projects, you can visualize the entire pipeline, including all cross-project interdependencies.                                                              |
 | [Pipelines for merge requests](../../../ci/merge_request_pipelines/index.md)                           | Customize a specific pipeline structure for merge requests in order to speed the cycle up by running only important jobs.                                                                                 |
 | [Pipeline Graphs](../../../ci/pipelines/index.md#visualizing-pipelines)                                      | View the status of pipelines within the merge request, including the deployment process.                                                                                                                  |
+| [Test Coverage visualization](test_coverage_visualization.md)                                          | See test coverage results for merge requests, within the file diff.                                                                                                                                       |
 ### Security Reports **(ULTIMATE)**

doc/user/project/merge_requests/test_coverage_visualization.md

+---
+type: reference, howto
+---
+
+# Test Coverage Visualization
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/3708) in GitLab 12.9.
+
+With the help of [GitLab CI/CD](../../../ci/README.md), you can collect the test
+coverage information of your favorite testing or coverage-analysis tool, and visualize
+this information inside the file diff view of your merge requests (MRs). This will allow you
+to see which lines are covered by tests, and which lines still require coverage, before the
+MR is merged.
+
+![Test Coverage Visualization Diff View](img/test_coverage_visualization_v12_9.png)
+
+## How test coverage visualization works
+
+Collecting the coverage information is done via GitLab CI/CD's
+[artifacts reports feature](../../../ci/yaml/README.md#artifactsreports).
+You can specify one or more coverage reports to collect, including wildcard paths.
+GitLab will then take the coverage information in all the files and combine it
+together.
+
+For the coverage analysis to work, you have to provide a properly formated
+[Cobertura XML](https://cobertura.github.io/cobertura/) report to
+[`artifacts:reports:cobertura`](../../../ci/yaml/README.md#artifactsreportscobertura).
+This format was originally developed for Java, but most coverage analysis frameworks
+for other languages have plugins to add support for it, like:
+
+- [simplecov-cobertura](https://rubygems.org/gems/simplecov-cobertura) (Ruby)
+- [gocover-cobertura](https://github.com/t-yuki/gocover-cobertura) (Golang)
+
+Other coverage analysis frameworks support the format out of the box, for example:
+
+- [Istanbul](https://istanbul.js.org/docs/advanced/alternative-reporters/#cobertura) (JavaScript)
+- [Coverage.py](https://coverage.readthedocs.io/en/coverage-5.0/cmd.html#xml-reporting) (Python)
+
+Once configured, if you create a merge request that triggers a pipeline which collects
+coverage reports, the coverage will be shown in the diff view. This includes reports
+from any job in any stage in the pipeline. The coverage will be displayed for each line:
+
+- `covered` (green): lines which have been checked at least once by tests
+- `no test coverage` (orange): lines which are loaded but never executed
+- no coverage information: lines which are non-instrumented or not loaded
+
+Hovering over the coverage bar will provide further information, such as the number
+of times the line was checked by tests.
+
+## Example test coverage configuration
+
+The following [`gitlab-ci.yml`](../../../ci/yaml/README.md) example uses [Mocha](https://mochajs.org/)
+JavaScript testing and [NYC](https://github.com/istanbuljs/nyc) coverage-tooling to
+generate the coverage artifact:
+
+```yaml
+test:
+  script:
+    - npm install
+    - npx nyc --reporter cobertura mocha
+  artifacts:
+    reports:
+      cobertura: coverage/cobertura-coverage.xml
+```
+
+## Enabling the feature
+
+This feature comes with the `:coverage_report_view` feature flag disabled by
+default. This feature is disabled due to some performance issues with very large
+data sets. When [the performance issue](https://gitlab.com/gitlab-org/gitlab/issues/37725)
+is resolved, the feature will be enabled by default.
+
+To enable this feature, ask a GitLab administrator with Rails console access to
+run the following command:
+
+```ruby
+Feature.enable(:coverage_report_view)
+```