Add latest changes from gitlab-org/gitlab@master

doc/user/project/clusters/index.md

@@ -27,7 +27,7 @@ Using the GitLab project Kubernetes integration, you can:
 - Use [Web terminals](#web-terminals).
 - Use [Deploy Boards](#deploy-boards-premium). **(PREMIUM)**
 - Use [Canary Deployments](#canary-deployments-premium). **(PREMIUM)**
-- View [Pod logs](#pod-logs-ultimate). **(ULTIMATE)**
+- View [Logs](#logs-ultimate). **(ULTIMATE)**
 - Run serverless workloads on [Kubernetes with Knative](serverless/index.md).
 ### Deploy Boards **(PREMIUM)**
@@ -48,11 +48,11 @@ the need to leave GitLab.
 [Read more about Canary Deployments](../canary_deployments.md)
-### Pod logs **(ULTIMATE)**
+### Logs **(ULTIMATE)**
 GitLab makes it easy to view the logs of running pods in connected Kubernetes clusters. By displaying the logs directly in GitLab, developers can avoid having to manage console tools or jump to a different interface.
-[Read more about Kubernetes pod logs](kubernetes_pod_logs.md)
+[Read more about Kubernetes logs](kubernetes_pod_logs.md)
 ### Kubernetes monitoring

doc/user/project/clusters/kubernetes_pod_logs.md

-# Kubernetes Pod Logs **(ULTIMATE)**
+# Kubernetes Logs **(ULTIMATE)**
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/4752) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.0.
@@ -11,17 +11,17 @@ Everything you need to build, test, deploy, and run your app at scale.
 ## Overview
-[Kubernetes](https://kubernetes.io) pod logs can be viewed directly within GitLab.
-![Pod logs](img/kubernetes_pod_logs_v12_8.png)
+[Kubernetes](https://kubernetes.io) logs can be viewed directly within GitLab.
+![Pod logs](img/kubernetes_pod_logs_v12_9.png)
 ## Requirements
-[Deploying to a Kubernetes environment](../deploy_boards.md#enabling-deploy-boards) is required in order to be able to use Pod Logs.
+[Deploying to a Kubernetes environment](../deploy_boards.md#enabling-deploy-boards) is required in order to be able to use Logs.
 ## Usage
-To access pod logs, you must have the right [permissions](../../permissions.md#project-members-permissions).
+To access logs, you must have the right [permissions](../../permissions.md#project-members-permissions).
 You can access them in two ways.
@@ -29,7 +29,7 @@ You can access them in two ways.
 > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22011) in GitLab 12.5.
-Go to **{cloud-gear}** **Operations > Pod logs** on the sidebar menu.
+Go to **{cloud-gear}** **Operations > Logs** on the sidebar menu.
 ![Sidebar menu](img/sidebar_menu_pod_logs_v12_5.png)

doc/user/project/integrations/prometheus.md

@@ -585,17 +585,17 @@ From each of the panels in the dashboard, you can access the context menu by cli
 The options are:
-- [View logs](#view-pod-logs-ultimate)
+- [View logs](#view-logs-ultimate)
 - [Download CSV](#downloading-data-as-csv)
 - [Generate link to chart](#embedding-gitlab-managed-kubernetes-metrics)
 - [Alerts](#setting-up-alerts-for-prometheus-metrics-ultimate)
-### View Pod Logs **(ULTIMATE)**
+### View Logs **(ULTIMATE)**
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/122013) in GitLab 12.8.
-If you have [Pod Logs](../clusters/kubernetes_pod_logs.md) enabled,
-you can navigate from the charts in the dashboard to view Pod Logs by
+If you have [Logs](../clusters/kubernetes_pod_logs.md) enabled,
+you can navigate from the charts in the dashboard to view Logs by
 clicking on the context menu in the upper-right corner.
 If you use the **Timeline zoom** function at the bottom of the chart, logs will narrow down to the time range you selected.
@@ -710,7 +710,7 @@ Prometheus server.
 > [Introduced][ce-29691] in GitLab 12.2.
-It is possible to display metrics charts within [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-gfm). The maximum number of embeds allowed in a GitLab Flavored Markdown field is 100.
+It is possible to display metrics charts within [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-gfm) fields such as issue or merge request descriptions. The maximum number of embedded charts allowed in a GitLab Flavored Markdown field is 100.
 This can be useful if you are sharing an application incident or performance
 metrics to others and want to have relevant information directly available.
@@ -748,6 +748,25 @@ It is also possible to embed either the default dashboard metrics or individual
 ![Embedded Metrics in issue templates](img/embed_metrics_issue_template.png)
+### Embedding Cluster Health Charts **(ULTIMATE)**
+
+> [Introduced](<https://gitlab.com/gitlab-org/gitlab/issues/40997>) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.9.
+
+[Cluster Health Metrics](../clusters/index.md#monitoring-your-kubernetes-cluster-ultimate) can also be embedded in [GitLab-flavored Markdown](../../markdown.md).
+
+To embed a metric chart, include a link to that chart in the form `https://<root_url>/<project>/-/cluster/<cluster_id>?<query_params>` anywhere that GitLab-flavored Markdown is supported. To generate and copy a link to the chart, follow the instructions in the [Cluster Health Metric documentation](../clusters/index.md#monitoring-your-kubernetes-cluster-ultimate).
+
+The following requirements must be met for the metric to unfurl:
+
+- The `<cluster_id>` must correspond to a real cluster.
+- Prometheus must be monitoring the cluster.
+- The user must be allowed access to the project cluster metrics.
+- The dashboards must be reporting data on the [Cluster Health Page](../clusters/index.md#monitoring-your-kubernetes-cluster-ultimate)
+
+ If the above requirements are met, then the metric will unfurl as seen below.
+
+![Embedded Cluster Metric in issue descriptions](img/prometheus_cluster_health_embed_v12_9.png)
+
 ### Embedding Grafana charts
 Grafana metrics can be embedded in [GitLab Flavored Markdown](../../markdown.md).

jest.config.js

@@ -82,7 +82,7 @@ module.exports = {
     '^.+\\.js$': 'babel-jest',
     '^.+\\.vue$': 'vue-jest',
   },
-  transformIgnorePatterns: ['node_modules/(?!(@gitlab/ui|bootstrap-vue)/)'],
+  transformIgnorePatterns: ['node_modules/(?!(@gitlab/ui|bootstrap-vue|three)/)'],
   timers: 'fake',
   testEnvironment: '<rootDir>/spec/frontend/environment.js',
   testEnvironmentOptions: {

lib/gitlab/ci/config/entry/inherit.rb

@@ -16,11 +16,11 @@ module Gitlab
             validates :config, allowed_keys: ALLOWED_KEYS
           end
-          entry :default, ::Gitlab::Config::Entry::Boolean,
+          entry :default, ::Gitlab::Ci::Config::Entry::Inherit::Default,
             description: 'Indicates whether to inherit `default:`.',
             default: true
-          entry :variables, ::Gitlab::Config::Entry::Boolean,
+          entry :variables, ::Gitlab::Ci::Config::Entry::Inherit::Variables,
             description: 'Indicates whether to inherit `variables:`.',
             default: true
         end

lib/gitlab/ci/config/entry/inherit/default.rb

+# frozen_string_literal: true
+
+module Gitlab
+  module Ci
+    class Config
+      module Entry
+        ##
+        # This class represents a default inherit entry
+        #
+        class Inherit
+          class Default < ::Gitlab::Config::Entry::Simplifiable
+            strategy :BooleanStrategy, if: -> (config) { [true, false].include?(config) }
+            strategy :ArrayStrategy, if: -> (config) { config.is_a?(Array) }
+
+            class BooleanStrategy < ::Gitlab::Config::Entry::Boolean
+              def inherit?(_key)
+                value
+              end
+            end
+
+            class ArrayStrategy < ::Gitlab::Config::Entry::Node
+              include ::Gitlab::Config::Entry::Validatable
+
+              ALLOWED_VALUES = ::Gitlab::Ci::Config::Entry::Default::ALLOWED_KEYS.map(&:to_s).freeze
+
+              validations do
+                validates :config, type: Array
+                validates :config, array_of_strings: true
+                validates :config, allowed_array_values: { in: ALLOWED_VALUES }
+              end
+
+              def inherit?(key)
+                value.include?(key.to_s)
+              end
+            end
+
+            class UnknownStrategy < ::Gitlab::Config::Entry::Node
+              def errors
+                ["#{location} should be a bool or array of strings"]
+              end
+
+              def inherit?(key)
+                false
+              end
+            end
+          end
+        end
+      end
+    end
+  end
+end

lib/gitlab/ci/config/entry/inherit/variables.rb

+# frozen_string_literal: true
+
+module Gitlab
+  module Ci
+    class Config
+      module Entry
+        ##
+        # This class represents a variables inherit entry
+        #
+        class Inherit
+          class Variables < ::Gitlab::Config::Entry::Simplifiable
+            strategy :BooleanStrategy, if: -> (config) { [true, false].include?(config) }
+            strategy :ArrayStrategy, if: -> (config) { config.is_a?(Array) }
+
+            class BooleanStrategy < ::Gitlab::Config::Entry::Boolean
+              def inherit?(_key)
+                value
+              end
+            end
+
+            class ArrayStrategy < ::Gitlab::Config::Entry::Node
+              include ::Gitlab::Config::Entry::Validatable
+
+              validations do
+                validates :config, type: Array
+                validates :config, array_of_strings: true
+              end
+
+              def inherit?(key)
+                value.include?(key.to_s)
+              end
+            end
+
+            class UnknownStrategy < ::Gitlab::Config::Entry::Node
+              def errors
+                ["#{location} should be a bool or array of strings"]
+              end
+
+              def inherit?(key)
+                false
+              end
+            end
+          end
+        end
+      end
+    end
+  end
+end

lib/gitlab/ci/config/entry/processable.rb

@@ -94,7 +94,7 @@ module Gitlab
           end
           def overwrite_entry(deps, key, current_entry)
-            return unless inherit_entry&.default_value
+            return unless inherit_entry&.default_entry&.inherit?(key)
             return unless deps.default_entry
             deps.default_entry[key] unless current_entry.specified?
@@ -111,11 +111,12 @@ module Gitlab
           end
           def root_and_job_variables_value
-            if inherit_entry&.variables_value
-              @root_variables_value.to_h.merge(variables_value.to_h) # rubocop:disable Gitlab/ModuleWithInstanceVariables
-            else
-              variables_value.to_h
+            root_variables = @root_variables_value.to_h # rubocop:disable Gitlab/ModuleWithInstanceVariables
+            root_variables = root_variables.select do |key, _|
+              inherit_entry&.variables_entry&.inherit?(key)
             end
+
+            root_variables.merge(variables_value.to_h)
           end
         end
       end

lib/gitlab/metrics/dashboard/finder.rb

@@ -29,9 +29,11 @@ module Gitlab
           #         Used by embedded dashboards.
           # @param options - y_label [String] Y-Axis label of
           #         a panel. Used by embedded dashboards.
-          # @param options - cluster [Cluster]
+          # @param options - cluster [Cluster]. Used by
+          #         embedded and un-embedded dashboards.
           # @param options - cluster_type [Symbol] The level of
-          #         cluster, one of [:admin, :project, :group]
+          #         cluster, one of [:admin, :project, :group]. Used by
+          #         embedded and un-embedded dashboards.
           # @param options - grafana_url [String] URL pointing
           #         to a grafana dashboard panel
           # @param options - prometheus_alert_id [Integer] ID of

lib/gitlab/metrics/dashboard/service_selector.rb

@@ -3,7 +3,8 @@
 # Responsible for determining which dashboard service should
 # be used to fetch or generate a dashboard hash.
 # The services can be considered in two categories - embeds
-# and dashboards. Embeds are all portions of dashboards.
+# and dashboards. Embed hashes are identical to dashboard hashes except
+# that they contain a subset of panels.
 module Gitlab
   module Metrics
     module Dashboard

lib/gitlab/middleware/go.rb

@@ -53,8 +53,9 @@ module Gitlab
         repository_url = if Gitlab::CurrentSettings.enabled_git_access_protocol == 'ssh'
                            shell = config.gitlab_shell
+                           user = "#{shell.ssh_user}@" unless shell.ssh_user.empty?
                            port = ":#{shell.ssh_port}" unless shell.ssh_port == 22
-                           "ssh://#{shell.ssh_user}@#{shell.ssh_host}#{port}/#{path}.git"
+                           "ssh://#{user}#{shell.ssh_host}#{port}/#{path}.git"
                          else
                            "#{project_url}.git"
                          end

lib/gitlab/sidekiq_middleware/server_metrics.rb

@@ -45,6 +45,8 @@ module Gitlab
           labels[:job_status] = job_succeeded ? "done" : "fail"
           @metrics[:sidekiq_jobs_cpu_seconds].observe(labels, job_thread_cputime)
           @metrics[:sidekiq_jobs_completion_seconds].observe(labels, monotonic_time)
+          @metrics[:sidekiq_jobs_db_seconds].observe(labels, ActiveRecord::LogSubscriber.runtime / 1000)
+          @metrics[:sidekiq_jobs_gitaly_seconds].observe(labels, Gitlab::GitalyClient.query_time)
         end
       end
@@ -54,6 +56,8 @@ module Gitlab
         {
           sidekiq_jobs_cpu_seconds:            ::Gitlab::Metrics.histogram(:sidekiq_jobs_cpu_seconds, 'Seconds of cpu time to run Sidekiq job', {}, SIDEKIQ_LATENCY_BUCKETS),
           sidekiq_jobs_completion_seconds:     ::Gitlab::Metrics.histogram(:sidekiq_jobs_completion_seconds, 'Seconds to complete Sidekiq job', {}, SIDEKIQ_LATENCY_BUCKETS),
+          sidekiq_jobs_db_seconds:             ::Gitlab::Metrics.histogram(:sidekiq_jobs_db_seconds, 'Seconds of database time to run Sidekiq job', {}, SIDEKIQ_LATENCY_BUCKETS),
+          sidekiq_jobs_gitaly_seconds:         ::Gitlab::Metrics.histogram(:sidekiq_jobs_gitaly_seconds, 'Seconds of Gitaly time to run Sidekiq job', {}, SIDEKIQ_LATENCY_BUCKETS),
           sidekiq_jobs_queue_duration_seconds: ::Gitlab::Metrics.histogram(:sidekiq_jobs_queue_duration_seconds, 'Duration in seconds that a Sidekiq job was queued before being executed', {}, SIDEKIQ_LATENCY_BUCKETS),
           sidekiq_jobs_failed_total:           ::Gitlab::Metrics.counter(:sidekiq_jobs_failed_total, 'Sidekiq jobs failed'),
           sidekiq_jobs_retried_total:          ::Gitlab::Metrics.counter(:sidekiq_jobs_retried_total, 'Sidekiq jobs retried'),

lib/sentry/client/issue.rb

@@ -75,7 +75,21 @@ module Sentry
         http_get(api_urls.issue_url(issue_id))[:body]
       end
-      def parse_gitlab_issue(plugin_issues)
+      def parse_gitlab_issue(issue)
+        parse_issue_annotations(issue) || parse_plugin_issue(issue)
+      end
+
+      def parse_issue_annotations(issue)
+        issue
+          .fetch('annotations', [])
+          .reject(&:blank?)
+          .map { |annotation| Nokogiri.make(annotation) }
+          .find { |html| html['href']&.starts_with?(Gitlab.config.gitlab.url) }
+          .try(:[], 'href')
+      end
+
+      def parse_plugin_issue(issue)
+        plugin_issues = issue.fetch('pluginIssues', nil)
         return unless plugin_issues
         gitlab_plugin = plugin_issues.detect { |item| item['id'] == 'gitlab' }
@@ -145,7 +159,7 @@ module Sentry
           short_id: issue.fetch('shortId', nil),
           status: issue.fetch('status', nil),
           frequency: issue.dig('stats', '24h'),
-          gitlab_issue: parse_gitlab_issue(issue.fetch('pluginIssues', nil)),
+          gitlab_issue: parse_gitlab_issue(issue),
           project_id: issue.dig('project', 'id'),
           project_name: issue.dig('project', 'name'),
           project_slug: issue.dig('project', 'slug'),

qa/qa/resource/deploy_key.rb

@@ -6,7 +6,7 @@ module QA
       attr_accessor :title, :key
       attribute :md5_fingerprint do
-        Page::Project::Settings::Repository.perform do |setting|
+        Page::Project::Settings::CICD.perform do |setting|
           setting.expand_deploy_keys do |key|
             key.find_md5_fingerprint(title)
           end
@@ -25,7 +25,7 @@ module QA
         Page::Project::Menu.perform(&:go_to_repository_settings)
-        Page::Project::Settings::Repository.perform do |setting|
+        Page::Project::Settings::CICD.perform do |setting|
           setting.expand_deploy_keys do |page|
             page.fill_key_title(title)
             page.fill_key_value(key)

qa/qa/specs/features/browser_ui/6_release/deploy_key/add_deploy_key_spec.rb

@@ -17,7 +17,7 @@ module QA
         expect(deploy_key.md5_fingerprint).to eq key.md5_fingerprint
-        Page::Project::Settings::Repository.perform do |setting|
+        Page::Project::Settings::CICD.perform do |setting|
           setting.expand_deploy_keys do |keys|
             expect(keys).to have_key(deploy_key_title, key.md5_fingerprint)
           end

spec/controllers/projects/clusters_controller_spec.rb

@@ -23,7 +23,6 @@ describe Projects::ClustersController do
     describe 'functionality' do
       context 'when project has one or more clusters' do
-        let(:project) { create(:project) }
         let!(:enabled_cluster) { create(:cluster, :provided_by_gcp, projects: [project]) }
         let!(:disabled_cluster) { create(:cluster, :disabled, :provided_by_gcp, :production_environment, projects: [project]) }
@@ -53,8 +52,6 @@ describe Projects::ClustersController do
       end
       context 'when project does not have a cluster' do
-        let(:project) { create(:project) }
-
         it 'returns an empty state page' do
           go

spec/features/projects/container_registry_spec.rb

@@ -126,6 +126,7 @@ describe 'Container Registry', :js do
       describe 'image repo details' do
         before do
+          stub_container_registry_tags(repository: %r{my/image}, tags: ('1'..'20').to_a, with_manifest: true)
           visit_container_registry_details 'my/image'
         end
@@ -140,12 +141,18 @@ describe 'Container Registry', :js do
         it 'user removes a specific tag from container repository' do
           service = double('service')
           expect(service).to receive(:execute).with(container_repository) { { status: :success } }
-          expect(Projects::ContainerRepository::DeleteTagsService).to receive(:new).with(container_repository.project, user, tags: ['latest']) { service }
-          click_on(class: 'js-delete-registry')
+          expect(Projects::ContainerRepository::DeleteTagsService).to receive(:new).with(container_repository.project, user, tags: ['1']) { service }
+          first('.js-delete-registry').click
           expect(find('.modal .modal-title')).to have_content _('Remove tag')
           find('.modal .modal-footer .btn-danger').click
         end
+
+        it('pagination navigate to the second page') do
+          pagination = find('.gl-pagination')
+          pagination.click_link('2')
+          expect(page).to have_content '20'
+        end
       end
     end
   end

spec/features/projects/snippets/create_snippet_spec.rb

@@ -95,6 +95,29 @@ shared_examples_for 'snippet editor' do
       link = find('a.no-attachment-icon img[alt="banana_sample"]')['src']
       expect(link).to match(%r{/#{Regexp.escape(project.full_path)}/uploads/\h{32}/banana_sample\.gif\z})
     end
+
+    context 'when the git operation fails' do
+      let(:error) { 'This is a git error' }
+
+      before do
+        allow_next_instance_of(Snippets::CreateService) do |instance|
+          allow(instance).to receive(:create_commit).and_raise(StandardError, error)
+        end
+
+        fill_form
+
+        click_button('Create snippet')
+        wait_for_requests
+      end
+
+      it 'displays the error' do
+        expect(page).to have_content(error)
+      end
+
+      it 'renders new page' do
+        expect(page).to have_content('New Snippet')
+      end
+    end
   end
   context 'when a user is not authenticated' do

spec/features/snippets/user_creates_snippet_spec.rb

@@ -78,6 +78,29 @@ shared_examples_for 'snippet editor' do
     expect(reqs.first.status_code).to eq(200)
   end
+  context 'when the git operation fails' do
+    let(:error) { 'This is a git error' }
+
+    before do
+      allow_next_instance_of(Snippets::CreateService) do |instance|
+        allow(instance).to receive(:create_commit).and_raise(StandardError, error)
+      end
+
+      fill_form
+
+      click_button('Create snippet')
+      wait_for_requests
+    end
+
+    it 'displays the error' do
+      expect(page).to have_content(error)
+    end
+
+    it 'renders new page' do
+      expect(page).to have_content('New Snippet')
+    end
+  end
+
   it 'validation fails for the first time' do
     fill_in 'personal_snippet_title', with: 'My Snippet Title'
     click_button('Create snippet')