Add latest changes from gitlab-org/gitlab@master

doc/development/pipelines.md

@@ -100,6 +100,7 @@ and included in `rules` definitions via [YAML anchors](../ci/yaml/README.md#anch
 | `if-master-refs`                                             | Matches if the current branch is `master`. | |
 | `if-master-or-tag`                                           | Matches if the pipeline is for the `master` branch or for a tag. | |
 | `if-merge-request`                                           | Matches if the pipeline is for a merge request. | |
+| `if-nightly-master-schedule`                                 | Matches if the pipeline is for a `master` scheduled pipeline with `$NIGHTLY` set. | |
 | `if-dot-com-gitlab-org-schedule`                             | Limits jobs creation to scheduled pipelines for the `gitlab-org` group on GitLab.com. | |
 | `if-dot-com-gitlab-org-master`                               | Limits jobs creation to the `master` branch for the `gitlab-org` group on GitLab.com. | |
 | `if-dot-com-gitlab-org-merge-request`                        | Limits jobs creation to merge requests for the `gitlab-org` group on GitLab.com. | |

doc/install/aws/index.md

@@ -53,8 +53,8 @@ Here's a list of the AWS services we will use, with links to pricing information
   [Amazon EBS pricing](https://aws.amazon.com/ebs/pricing/).
 - **S3**: We will use S3 to store backups, artifacts, LFS objects, etc. See the
   [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/).
-- **ALB**: An Application Load Balancer will be used to route requests to the
-  GitLab instance. See the [Amazon ELB pricing](https://aws.amazon.com/elasticloadbalancing/pricing/).
+- **ELB**: A Classic Load Balancer will be used to route requests to the
+  GitLab instances. See the [Amazon ELB pricing](https://aws.amazon.com/elasticloadbalancing/pricing/).
 - **RDS**: An Amazon Relational Database Service using PostgreSQL will be used
   to provide a High Availability database configuration. See the
   [Amazon RDS pricing](https://aws.amazon.com/rds/postgresql/pricing/).
@@ -291,27 +291,30 @@ and add a custom TCP rule for port `6379` accessible within itself.
 ## Load Balancer
-On the EC2 dashboard, look for Load Balancer on the left column:
+On the EC2 dashboard, look for Load Balancer in the left navigation bar:
 1. Click the **Create Load Balancer** button.
-   1. Choose the Application Load Balancer.
-   1. Give it a name (`gitlab-loadbalancer`) and set the scheme to "internet-facing".
-   1. In the "Listeners" section, make sure it has HTTP and HTTPS.
-   1. In the "Availability Zones" section, select the `gitlab-vpc` we have created
-      and associate the **public subnets**.
-1. Click **Configure Security Settings** to go to the next section to
-   select the TLS certificate. When done, go to the next step.
-1. In the "Security Groups" section, create a new one by giving it a name
-   (`gitlab-loadbalancer-sec-group`) and allow both HTTP ad HTTPS traffic
+   1. Choose the **Classic Load Balancer**.
+   1. Give it a name (`gitlab-loadbalancer`) and for the **Create LB Inside** option, select `gitlab-vpc` from the dropdown menu.
+   1. In the **Listeners** section, set HTTP port 80, HTTPS port 443, and TCP port 22 for both load balancer and instance protocols and ports.
+   1. In the **Select Subnets** section, select both public subnets from the list.
+1. Click **Assign Security Groups** and select **Create a new security group**, give it a name
+   (`gitlab-loadbalancer-sec-group`) and description, and allow both HTTP and HTTPS traffic
    from anywhere (`0.0.0.0/0, ::/0`).
-1. In the next step, configure the routing and select an existing target group
-   (`gitlab-public`). The Load Balancer Health will allow us to indicate where to
-   ping and what makes up a healthy or unhealthy instance.
-1. Leave the "Register Targets" section as is, and finally review the settings
-   and create the ELB.
+1. Click **Configure Security Settings** and select an SSL/TLS certificate from ACM or upload a certificate to IAM.
+1. Click **Configure Health Check** and set up a health check for your EC2 instances.
+   1. For **Ping Protocol**, select HTTP.
+   1. For **Ping Port**, enter 80.
+   1. For **Ping Path**, enter `/explore`. (We use `/explore` as it's a public endpoint that does
+   not require authorization.)
+   1. Keep the default **Advanced Details** or adjust them according to your needs.
+1. For now, don't click **Add EC2 Instances**, as we don't have any instances to add yet. Come back
+to your load balancer after creating your GitLab instances and add them.
+1. Click **Add Tags** and add any tags you need.
+1. Click **Review and Create**, review all your settings, and click **Create** if you're happy.
 After the Load Balancer is up and running, you can revisit your Security
-Groups to refine the access only through the ELB and any other requirement
+Groups to refine the access only through the ELB and any other requirements
 you might have.
 ## Deploying GitLab inside an auto scaling group

doc/user/group/saml_sso/index.md

@@ -64,7 +64,7 @@ We intend to add a similar SSO requirement for [Git and API activity](https://gi
 #### Group-managed accounts
-[Introduced in GitLab 12.1](https://gitlab.com/groups/gitlab-org/-/epics/709).
+> [Introduced in GitLab 12.1](https://gitlab.com/groups/gitlab-org/-/epics/709).
 When SSO is being enforced, groups can enable an additional level of protection by enforcing the creation of dedicated user accounts to access the group.
@@ -95,6 +95,14 @@ To access the Credentials inventory of a group, navigate to **{shield}** **Secur
 This feature is similar to the [Credentials inventory for self-managed instances](../../admin_area/credentials_inventory.md).
+##### Outer forks restriction for Group-managed accounts
+
+> [Introduced in GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/issues/34648)
+
+Groups with enabled group-managed accounts can allow or disallow forking of projects outside of root group
+by using separate toggle. If forking is disallowed any project of given root group or its subgroups can be forked to
+a subgroup of the same root group only.
+
 #### Assertions
 When using group-managed accounts, the following user details need to be passed to GitLab as SAML

doc/user/project/import/index.md

@@ -49,9 +49,9 @@ Docker pulls and pushes and re-run any CI pipelines to retrieve any build artifa
 ## Migrating between two self-hosted GitLab instances
-The best method for migrating a project from one GitLab instance to another,
+The best method for migrating from one GitLab instance to another,
 perhaps from an old server to a new server for example, is to
-[back up the project](../../../raketasks/backup_restore.md),
+[back up the instance](../../../raketasks/backup_restore.md),
 then restore it on the new server.
 In the event of merging two GitLab instances together (for example, both instances have existing data on them and one can't be wiped),

lib/api/projects.rb

@@ -267,18 +267,16 @@ module API
       post ':id/fork' do
         Gitlab::QueryLimiting.whitelist('https://gitlab.com/gitlab-org/gitlab-foss/issues/42284')
+        not_found! unless can?(current_user, :fork_project, user_project)
+
         fork_params = declared_params(include_missing: false)
-        namespace_id = fork_params[:namespace]
-        if namespace_id.present?
-          fork_params[:namespace] = find_namespace(namespace_id)
-          unless fork_params[:namespace] && can?(current_user, :create_projects, fork_params[:namespace])
-            not_found!('Target Namespace')
-          end
-        end
-        forked_project = ::Projects::ForkService.new(user_project, current_user, fork_params).execute
+        fork_params[:namespace] = find_namespace!(fork_params[:namespace]) if fork_params[:namespace].present?
+        service = ::Projects::ForkService.new(user_project, current_user, fork_params)
+        not_found!('Target Namespace') unless service.valid_fork_target?
+        forked_project = service.execute
         if forked_project.errors.any?
           conflict!(forked_project.errors.messages)

lib/gitlab/object_hierarchy.rb

@@ -51,7 +51,7 @@ module Gitlab
     # and all their ancestors (recursively).
     #
     # Passing an `upto` will stop the recursion once the specified parent_id is
-    # reached. So all ancestors *lower* than the specified acestor will be
+    # reached. So all ancestors *lower* than the specified ancestor will be
     # included.
     #
     # Passing a `hierarchy_order` with either `:asc` or `:desc` will cause the

spec/controllers/projects/forks_controller_spec.rb

@@ -209,6 +209,17 @@ describe Projects::ForksController do
         expect(response).to redirect_to(namespace_project_import_path(user.namespace, project))
       end
+      context 'when target namespace is not valid for forking' do
+        let(:params) { super().merge(namespace_key: another_group.id) }
+        let(:another_group) { create :group }
+
+        it 'responds with :not_found' do
+          subject
+
+          expect(response).to have_gitlab_http_status(:not_found)
+        end
+      end
+
       context 'continue params' do
         let(:params) do
           {

spec/finders/fork_targets_finder_spec.rb

+# frozen_string_literal: true
+
+require 'spec_helper'
+
+describe ForkTargetsFinder do
+  subject(:finder) { described_class.new(project, user) }
+
+  let(:project) { create(:project, namespace: create(:group)) }
+  let(:user) { create(:user) }
+  let!(:maintained_group) do
+    create(:group).tap { |g| g.add_maintainer(user) }
+  end
+  let!(:owned_group) do
+    create(:group).tap { |g| g.add_owner(user) }
+  end
+  let!(:developer_group) do
+    create(:group).tap { |g| g.add_developer(user) }
+  end
+  let!(:reporter_group) do
+    create(:group).tap { |g| g.add_reporter(user) }
+  end
+  let!(:guest_group) do
+    create(:group).tap { |g| g.add_guest(user) }
+  end
+
+  before do
+    project.namespace.add_owner(user)
+  end
+
+  describe '#execute' do
+    it 'returns all user manageable namespaces except project namespace' do
+      expect(finder.execute).to match_array([user.namespace, maintained_group, owned_group])
+    end
+  end
+end

spec/models/namespace_spec.rb

@@ -983,6 +983,24 @@ describe Namespace do
           expect(virtual_domain.lookup_paths).not_to be_empty
         end
       end
+
+      it 'preloads project_feature and route' do
+        project2 = create(:project, namespace: namespace)
+        project3 = create(:project, namespace: namespace)
+
+        project.mark_pages_as_deployed
+        project2.mark_pages_as_deployed
+        project3.mark_pages_as_deployed
+
+        virtual_domain = namespace.pages_virtual_domain
+
+        queries = ActiveRecord::QueryRecorder.new { virtual_domain.lookup_paths }
+
+        # 1 to load projects
+        # 1 to preload project features
+        # 1 to load routes
+        expect(queries.count).to eq(3)
+      end
     end
   end

spec/requests/api/projects_spec.rb

@@ -2698,20 +2698,14 @@ describe API::Projects do
       create(:project, :repository, creator: user, namespace: user.namespace)
     end
-    let(:group) { create(:group) }
-    let(:group2) do
-      group = create(:group, name: 'group2_name')
-      group.add_maintainer(user2)
-      group
-    end
-
-    let(:group3) do
-      group = create(:group, name: 'group3_name', parent: group2)
-      group.add_owner(user2)
-      group
-    end
+    let(:group) { create(:group, :public) }
+    let(:group2) { create(:group, name: 'group2_name') }
+    let(:group3) { create(:group, name: 'group3_name', parent: group2) }
     before do
+      group.add_guest(user2)
+      group2.add_maintainer(user2)
+      group3.add_owner(user2)
       project.add_reporter(user2)
       project2.add_reporter(user2)
     end
@@ -2720,7 +2714,7 @@ describe API::Projects do
       it 'forks if user has sufficient access to project' do
         post api("/projects/#{project.id}/fork", user2)
-        expect(response).to have_gitlab_http_status(201)
+        expect(response).to have_gitlab_http_status(:created)
         expect(json_response['name']).to eq(project.name)
         expect(json_response['path']).to eq(project.path)
         expect(json_response['owner']['id']).to eq(user2.id)
@@ -2733,7 +2727,7 @@ describe API::Projects do
       it 'forks if user is admin' do
         post api("/projects/#{project.id}/fork", admin)
-        expect(response).to have_gitlab_http_status(201)
+        expect(response).to have_gitlab_http_status(:created)
         expect(json_response['name']).to eq(project.name)
         expect(json_response['path']).to eq(project.path)
         expect(json_response['owner']['id']).to eq(admin.id)
@@ -2747,14 +2741,17 @@ describe API::Projects do
         new_user = create(:user)
         post api("/projects/#{project.id}/fork", new_user)
-        expect(response).to have_gitlab_http_status(404)
+        expect(response).to have_gitlab_http_status(:not_found)
         expect(json_response['message']).to eq('404 Project Not Found')
       end
       it 'fails if forked project exists in the user namespace' do
-        post api("/projects/#{project.id}/fork", user)
-        expect(response).to have_gitlab_http_status(409)
+        new_project = create(:project, name: project.name, path: project.path)
+        new_project.add_reporter(user)
+
+        post api("/projects/#{new_project.id}/fork", user)
+        expect(response).to have_gitlab_http_status(:conflict)
         expect(json_response['message']['name']).to eq(['has already been taken'])
         expect(json_response['message']['path']).to eq(['has already been taken'])
       end
@@ -2762,48 +2759,48 @@ describe API::Projects do
       it 'fails if project to fork from does not exist' do
         post api('/projects/424242/fork', user)
-        expect(response).to have_gitlab_http_status(404)
+        expect(response).to have_gitlab_http_status(:not_found)
         expect(json_response['message']).to eq('404 Project Not Found')
       end
       it 'forks with explicit own user namespace id' do
         post api("/projects/#{project.id}/fork", user2), params: { namespace: user2.namespace.id }
-        expect(response).to have_gitlab_http_status(201)
+        expect(response).to have_gitlab_http_status(:created)
         expect(json_response['owner']['id']).to eq(user2.id)
       end
       it 'forks with explicit own user name as namespace' do
         post api("/projects/#{project.id}/fork", user2), params: { namespace: user2.username }
-        expect(response).to have_gitlab_http_status(201)
+        expect(response).to have_gitlab_http_status(:created)
         expect(json_response['owner']['id']).to eq(user2.id)
       end
       it 'forks to another user when admin' do
         post api("/projects/#{project.id}/fork", admin), params: { namespace: user2.username }
-        expect(response).to have_gitlab_http_status(201)
+        expect(response).to have_gitlab_http_status(:created)
         expect(json_response['owner']['id']).to eq(user2.id)
       end
       it 'fails if trying to fork to another user when not admin' do
         post api("/projects/#{project.id}/fork", user2), params: { namespace: admin.namespace.id }
-        expect(response).to have_gitlab_http_status(404)
+        expect(response).to have_gitlab_http_status(:not_found)
       end
       it 'fails if trying to fork to non-existent namespace' do
         post api("/projects/#{project.id}/fork", user2), params: { namespace: 42424242 }
-        expect(response).to have_gitlab_http_status(404)
-        expect(json_response['message']).to eq('404 Target Namespace Not Found')
+        expect(response).to have_gitlab_http_status(:not_found)
+        expect(json_response['message']).to eq('404 Namespace Not Found')
       end
       it 'forks to owned group' do
         post api("/projects/#{project.id}/fork", user2), params: { namespace: group2.name }
-        expect(response).to have_gitlab_http_status(201)
+        expect(response).to have_gitlab_http_status(:created)
         expect(json_response['namespace']['name']).to eq(group2.name)
       end
@@ -2811,7 +2808,7 @@ describe API::Projects do
         full_path = "#{group2.path}/#{group3.path}"
         post api("/projects/#{project.id}/fork", user2), params: { namespace: full_path }
-        expect(response).to have_gitlab_http_status(201)
+        expect(response).to have_gitlab_http_status(:created)
         expect(json_response['namespace']['name']).to eq(group3.name)
         expect(json_response['namespace']['full_path']).to eq(full_path)
       end
@@ -2819,20 +2816,21 @@ describe API::Projects do
       it 'fails to fork to not owned group' do
         post api("/projects/#{project.id}/fork", user2), params: { namespace: group.name }
-        expect(response).to have_gitlab_http_status(404)
+        expect(response).to have_gitlab_http_status(:not_found)
+        expect(json_response['message']).to eq("404 Target Namespace Not Found")
       end
       it 'forks to not owned group when admin' do
         post api("/projects/#{project.id}/fork", admin), params: { namespace: group.name }
-        expect(response).to have_gitlab_http_status(201)
+        expect(response).to have_gitlab_http_status(:created)
         expect(json_response['namespace']['name']).to eq(group.name)
       end
       it 'accepts a path for the target project' do
         post api("/projects/#{project.id}/fork", user2), params: { path: 'foobar' }
-        expect(response).to have_gitlab_http_status(201)
+        expect(response).to have_gitlab_http_status(:created)
         expect(json_response['name']).to eq(project.name)
         expect(json_response['path']).to eq('foobar')
         expect(json_response['owner']['id']).to eq(user2.id)
@@ -2846,14 +2844,14 @@ describe API::Projects do
         post api("/projects/#{project.id}/fork", user2), params: { path: 'foobar' }
         post api("/projects/#{project2.id}/fork", user2), params: { path: 'foobar' }
-        expect(response).to have_gitlab_http_status(409)
+        expect(response).to have_gitlab_http_status(:conflict)
         expect(json_response['message']['path']).to eq(['has already been taken'])
       end
       it 'accepts a name for the target project' do
         post api("/projects/#{project.id}/fork", user2), params: { name: 'My Random Project' }
-        expect(response).to have_gitlab_http_status(201)
+        expect(response).to have_gitlab_http_status(:created)
         expect(json_response['name']).to eq('My Random Project')
         expect(json_response['path']).to eq(project.path)
         expect(json_response['owner']['id']).to eq(user2.id)
@@ -2867,7 +2865,7 @@ describe API::Projects do
         post api("/projects/#{project.id}/fork", user2), params: { name: 'My Random Project' }
         post api("/projects/#{project2.id}/fork", user2), params: { name: 'My Random Project' }
-        expect(response).to have_gitlab_http_status(409)
+        expect(response).to have_gitlab_http_status(:conflict)
         expect(json_response['message']['name']).to eq(['has already been taken'])
       end
     end
@@ -2876,7 +2874,7 @@ describe API::Projects do
       it 'returns authentication error' do
         post api("/projects/#{project.id}/fork")
-        expect(response).to have_gitlab_http_status(401)
+        expect(response).to have_gitlab_http_status(:unauthorized)
         expect(json_response['message']).to eq('401 Unauthorized')
       end
     end
@@ -2890,8 +2888,7 @@ describe API::Projects do
       it 'denies project to be forked' do
         post api("/projects/#{project.id}/fork", admin)
-        expect(response).to have_gitlab_http_status(409)
-        expect(json_response['message']['forked_from_project_id']).to eq(['is forbidden'])
+        expect(response).to have_gitlab_http_status(:not_found)
       end
     end
   end

spec/services/projects/fork_service_spec.rb

@@ -275,6 +275,7 @@ describe Projects::ForkService do
       context 'fork project for group when user not owner' do
         it 'group developer fails to fork project into the group' do
           to_project = fork_project(@project, @developer, @opts)
+
           expect(to_project.errors[:namespace]).to eq(['is not valid'])
         end
       end
@@ -336,7 +337,9 @@ describe Projects::ForkService do
   context 'when linking fork to an existing project' do
     let(:fork_from_project) { create(:project, :public) }
     let(:fork_to_project) { create(:project, :public) }
-    let(:user) { create(:user) }
+    let(:user) do
+      create(:user).tap { |u| fork_to_project.add_maintainer(u) }
+    end
     subject { described_class.new(fork_from_project, user) }
@@ -387,4 +390,54 @@ describe Projects::ForkService do
       end
     end
   end
+
+  describe '#valid_fork_targets' do
+    let(:finder_mock) { instance_double('ForkTargetsFinder', execute: ['finder_return_value']) }
+    let(:current_user) { instance_double('User') }
+    let(:project) { instance_double('Project') }
+
+    before do
+      allow(ForkTargetsFinder).to receive(:new).with(project, current_user).and_return(finder_mock)
+    end
+
+    it 'returns whatever finder returns' do
+      expect(described_class.new(project, current_user).valid_fork_targets).to eq ['finder_return_value']
+    end
+  end
+
+  describe '#valid_fork_target?' do
+    subject { described_class.new(project, user, params).valid_fork_target? }
+
+    let(:project) { Project.new }
+    let(:params) { {} }
+
+    context 'when current user is an admin' do
+      let(:user) { build(:user, :admin) }
+
+      it { is_expected.to be_truthy }
+    end
+
+    context 'when current_user is not an admin' do
+      let(:user) { create(:user) }
+
+      let(:finder_mock) { instance_double('ForkTargetsFinder', execute: [user.namespace]) }
+      let(:project) { create(:project) }
+
+      before do
+        allow(ForkTargetsFinder).to receive(:new).with(project, user).and_return(finder_mock)
+      end
+
+      context 'when target namespace is in valid fork targets' do
+        let(:params) { { namespace: user.namespace } }
+
+        it { is_expected.to be_truthy }
+      end
+
+      context 'when target namespace is not in valid fork targets' do
+        let(:params) { { namespace: create(:group) } }
+
+        it { is_expected.to be_falsey }
+      end
+    end
+  end
 end