Add latest changes from gitlab-org/gitlab@master

app/helpers/projects_helper.rb

@@ -573,7 +573,7 @@ module ProjectsHelper
       lfsHelpPath: help_page_path('workflow/lfs/manage_large_binaries_with_git_lfs'),
       pagesAvailable: Gitlab.config.pages.enabled,
       pagesAccessControlEnabled: Gitlab.config.pages.access_control,
-      pagesHelpPath: help_page_path('user/project/pages/introduction', anchor: 'gitlab-pages-access-control-core-only')
+      pagesHelpPath: help_page_path('user/project/pages/introduction', anchor: 'gitlab-pages-access-control-core')
     }
   end

app/models/list.rb

@@ -50,7 +50,7 @@ class List < ApplicationRecord
       list_ids = items.map { |i| i[:list_id] }
       user_ids = items.map { |i| i[:user_id] }
-      ListUserPreference.where(list_id: list_ids, user_id: user_ids).find_each do |preference|
+      ListUserPreference.where(list_id: list_ids.uniq, user_id: user_ids.uniq).find_each do |preference|
         loader.call({ list_id: preference.list_id, user_id: preference.user_id }, preference)
       end
     end

changelogs/unreleased/fix-service-desk-forward.yml

+---
+title: Do not strip forwarded message body when creating an issue from Service Desk
+  email.
+merge_request: 18196
+author:
+type: fixed

doc/administration/auth/how_to_configure_ldap_gitlab_ce/index.md

@@ -18,12 +18,12 @@ The main reason organizations choose to utilize a LDAP server is to keep the ent
 There are many commercial and open source [directory servers](https://en.wikipedia.org/wiki/Directory_service#LDAP_implementations) that support the LDAP protocol. Deciding on the right directory server highly depends on the existing IT environment in which the server will be integrated with.
-For example, [Active Directory](https://technet.microsoft.com/en-us/library/hh831484(v=ws.11).aspx) is generally favored in a primarily Windows environment, as this allows quick integration with existing services. Other popular directory services include:
-- [Oracle Internet Directory](http://www.oracle.com/technetwork/middleware/id-mgmt/overview/index-082035.html)
+For example, [Active Directory](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/hh831484(v=ws.11)) is generally favored in a primarily Windows environment, as this allows quick integration with existing services. Other popular directory services include:
+- [Oracle Internet Directory](https://www.oracle.com/middleware/technologies/internet-directory.html)
 - [OpenLDAP](http://www.openldap.org/)
 - [389 Directory](http://directory.fedoraproject.org/)
-- [OpenDJ](https://forgerock.org/opendj/)
+- [OpenDJ (Renamed to Foregerock Directory Services)](https://www.forgerock.com/platform/directory-services)
 - [ApacheDS](https://directory.apache.org/)
 > GitLab uses the [Net::LDAP](https://rubygems.org/gems/net-ldap) library under the hood. This means it supports all [IETF](https://tools.ietf.org/html/rfc2251) compliant LDAPv3 servers.
@@ -32,9 +32,9 @@ For example, [Active Directory](https://technet.microsoft.com/en-us/library/hh83
 We won't cover the installation and configuration of Windows Server or Active Directory Domain Services in this tutorial. There are a number of resources online to guide you through this process:
-- Install Windows Server 2012  - (`technet.microsoft.com`) - [Installing Windows Server 2012](https://technet.microsoft.com/en-us/library/jj134246(v=ws.11).aspx)
-- Install Active Directory Domain Services (AD DS) (`technet.microsoft.com`)- [Install Active Directory Domain Services](https://technet.microsoft.com/windows-server-docs/identity/ad-ds/deploy/install-active-directory-domain-services--level-100-#BKMK_PS)
+- Install Windows Server 2012  - (`technet.microsoft.com`) - [Installing Windows Server 2012](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/jj134246(v=ws.11))
+- Install Active Directory Domain Services (AD DS) (`technet.microsoft.com`)- [Install Active Directory Domain Services](https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/deploy/install-active-directory-domain-services--level-100-#BKMK_PS)
 > **Shortcut:** You can quickly install AD DS via PowerShell using
 `Install-WindowsFeature AD-Domain-Services -IncludeManagementTools`
@@ -97,7 +97,7 @@ People Ops US     GitLab.org/GitLab INT/Global Groups/People Ops US
 Global Admins     GitLab.org/GitLab INT/Global Groups/Global Admins
 ```
-> See [more information](https://technet.microsoft.com/en-us/library/ff730967.aspx) on searching Active Directory with Windows PowerShell from [The Scripting Guys](https://technet.microsoft.com/en-us/scriptcenter/dd901334.aspx)
+> See [more information](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-powershell-1.0/ff730967(v=technet.10)) on searching Active Directory with Windows PowerShell from [The Scripting Guys](https://devblogs.microsoft.com/scripting/)
 ## GitLab LDAP configuration

doc/administration/auth/how_to_configure_ldap_gitlab_ee/index.md

@@ -110,7 +110,7 @@ gitlab_rails['ldap_servers'] = {
 Integration of GitLab with Active Directory (LDAP) reduces the complexity of user management.
 It has the advantage of improving user permission controls, whilst easing the deployment of GitLab into an existing [IT environment](https://www.techopedia.com/definition/29199/it-infrastructure). GitLab EE offers advanced group management and multiple LDAP servers.
-With the assistance of the [GitLab Support](https://about.gitlab.com/support) team, setting up GitLab with an existing AD/LDAP solution will be a smooth and painless process.
+With the assistance of the [GitLab Support](https://about.gitlab.com/support/) team, setting up GitLab with an existing AD/LDAP solution will be a smooth and painless process.
 <!-- ## Troubleshooting

doc/administration/auth/ldap-ee.md

@@ -415,7 +415,7 @@ main: # 'main' is the GitLab 'provider ID' of this LDAP server
 [^1]: In Active Directory, a user is marked as disabled/blocked if the user
       account control attribute (`userAccountControl:1.2.840.113556.1.4.803`)
-      has bit 2 set. See <https://ctogonewild.com/2009/09/03/bitmask-searches-in-ldap/>
+      has bit 2 set. See <https://ctovswild.com/2009/09/03/bitmask-searches-in-ldap/>
       for more information.
 ### User DN has changed

doc/administration/auth/ldap.md

@@ -408,7 +408,7 @@ group, you can use the following syntax:
 ```
 Find more information about this "LDAP_MATCHING_RULE_IN_CHAIN" filter at
-<https://docs.microsoft.com/en-us/windows/desktop/ADSI/search-filter-syntax>. Support for
+<https://docs.microsoft.com/en-us/windows/win32/adsi/search-filter-syntax>. Support for
 nested members in the user filter should not be confused with
 [group sync nested groups support](ldap-ee.md#supported-ldap-group-typesattributes). **(STARTER ONLY)**

doc/administration/housekeeping.md

 # Housekeeping
-> [Introduced][ce-2371] in GitLab 8.4.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/3041) in GitLab 8.4.
 ## Automatic housekeeping
@@ -23,16 +23,12 @@ For example in the following scenario a `git repack -d` will be executed:
 When the `pushes_since_gc` value is 50 a `repack -A -d --pack-kept-objects` will run, similarly when
 the `pushes_since_gc` value is 200 a `git gc` will be run.
-- `git gc` ([man page][man-gc]) runs a number of housekeeping tasks,
+- `git gc` ([man page](https://mirrors.edge.kernel.org/pub/software/scm/git/docs/git-gc.html)) runs a number of housekeeping tasks,
   such as compressing filerevisions (to reduce disk space and increase performance)
   and removing unreachable objects which may have been created from prior invocations of
   `git add`.
-- `git repack` ([man page][man-repack]) re-organize existing packs into a single, more efficient pack.
+- `git repack` ([man page](https://mirrors.edge.kernel.org/pub/software/scm/git/docs/git-repack.html)) re-organize existing packs into a single, more efficient pack.
 You can find this option under your project's **Settings > General > Advanced**.
 ![Housekeeping settings](img/housekeeping_settings.png)
-
-[ce-2371]: https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/2371 "Housekeeping merge request"
-[man-gc]: https://www.kernel.org/pub/software/scm/git/docs/git-gc.html "git gc man page"
-[man-repack]: https://www.kernel.org/pub/software/scm/git/docs/git-repack.html

doc/administration/index.md

@@ -97,7 +97,7 @@ Learn how to install, configure, update, and maintain your GitLab instance.
 ### GitLab platform integrations
-- [Mattermost](https://docs.gitlab.com/omnibus/gitlab-mattermost/): Integrate with [Mattermost](https://about.mattermost.com/), an open source, private cloud workplace for web messaging.
+- [Mattermost](https://docs.gitlab.com/omnibus/gitlab-mattermost/): Integrate with [Mattermost](https://mattermost.com), an open source, private cloud workplace for web messaging.
 - [PlantUML](integration/plantuml.md): Create simple diagrams in AsciiDoc and Markdown documents
   created in snippets, wikis, and repos.
 - [Web terminals](integration/terminal.md): Provide terminal access to your applications deployed to Kubernetes from within GitLab's CI/CD [environments](../ci/environments.md#web-terminals).

doc/administration/pages/index.md

@@ -305,7 +305,7 @@ Pages access control is disabled by default. To enable it:
    ```
 1. [Reconfigure GitLab][reconfigure].
-1. Users can now configure it in their [projects' settings](../../user/project/pages/introduction.md#gitlab-pages-access-control-core-only).
+1. Users can now configure it in their [projects' settings](../../user/project/pages/introduction.md#gitlab-pages-access-control-core).
 ### Running behind a proxy

doc/administration/pages/source.md

@@ -411,7 +411,7 @@ Pages access control is disabled by default. To enable it:
      -auth-server <URL of the GitLab instance>
    ```
-1. Users can now configure it in their [projects' settings](../../user/project/pages/introduction.md#gitlab-pages-access-control-core-only).
+1. Users can now configure it in their [projects' settings](../../user/project/pages/introduction.md#gitlab-pages-access-control-core).
 ## Change storage path

doc/administration/troubleshooting/debug.md

@@ -211,5 +211,5 @@ The output in `/tmp/unicorn.txt` may help diagnose the root cause.
 ## More information
-- [Debugging Stuck Ruby Processes](https://blog.newrelic.com/2013/04/29/debugging-stuck-ruby-processes-what-to-do-before-you-kill-9/)
+- [Debugging Stuck Ruby Processes](https://blog.newrelic.com/engineering/debugging-stuck-ruby-processes-what-to-do-before-you-kill-9/)
 - [Cheatsheet of using gdb and ruby processes](gdb-stuck-ruby.txt)

doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md

@@ -701,7 +701,8 @@ MergeRequests::RebaseService.new(m.target_project, u).execute(m)
 ### Cancel stuck pending pipelines
-See <https://gitlab.com/gitlab-com/support-forum/issues/2449#note_41929707>.
+For more information, see the [confidential issue](../../user/project/issues/confidential_issues.md)
+`https://gitlab.com/gitlab-com/support-forum/issues/2449#note_41929707`.
 ```ruby
 Ci::Pipeline.where(project_id: p.id).where(status: 'pending').count

doc/administration/troubleshooting/kubernetes_cheat_sheet.md

@@ -191,8 +191,8 @@ and they will assist you with any issues you are having.
 ## Installation of minimal GitLab config via Minukube on macOS
-This section is based on [Developing for Kubernetes with Minikube](https://gitlab.com/gitlab-org/charts/gitlab/blob/master/doc/minikube/index.md)
-and [Helm](https://gitlab.com/gitlab-org/charts/gitlab/blob/master/doc/helm/index.md). Refer
+This section is based on [Developing for Kubernetes with Minikube](https://docs.gitlab.com/charts/development/minikube/index.html)
+and [Helm](https://docs.gitlab.com/charts/installation/tools.html#helm). Refer
 to those documents for details.
 - Install Kubectl via Homebrew:

doc/api/README.md

@@ -329,7 +329,7 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab
 ### Pagination Link header
-[Link headers](http://www.w3.org/wiki/LinkHeader) are sent back with each
+[Link headers](https://www.w3.org/wiki/LinkHeader) are sent back with each
 response. They have `rel` set to prev/next/first/last and contain the relevant
 URL. Please use these links instead of generating your own URLs.
@@ -563,7 +563,7 @@ The correct encoding for the query parameter would be:
 ## Clients
 There are many unofficial GitLab API Clients for most of the popular
-programming languages. Visit the [GitLab website] for a complete list.
+programming languages. Visit the [GitLab website](https://about.gitlab.com/partners/#api-clients) for a complete list.
 ## Rate limits
@@ -572,7 +572,6 @@ For administrator documentation on rate limit settings, see
 specifically used by GitLab.com, see
 [GitLab.com-specific rate limits](../user/gitlab_com/index.md#gitlabcom-specific-rate-limits).
-[GitLab website]: https://about.gitlab.com/applications/#api-clients "Clients using the GitLab API"
 [lib-api-url]: https://gitlab.com/gitlab-org/gitlab-foss/tree/master/lib/api/api.rb
 [ce-3749]: https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/3749
 [ce-5951]: https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/5951

doc/api/graphql/index.md

@@ -42,7 +42,7 @@ A first iteration of a GraphQL API includes the following queries
 ### Multiplex queries
 GitLab supports batching queries into a single request using
-[apollo-link-batch-http](https://www.apollographql.com/docs/link/links/batch-http). More
+[apollo-link-batch-http](https://www.apollographql.com/docs/link/links/batch-http/). More
 info about multiplexed queries is also available for
 [graphql-ruby](https://graphql-ruby.org/queries/multiplex.html) the
 library GitLab uses on the backend.

doc/api/projects.md

@@ -2023,7 +2023,7 @@ Read more in the [Project members](members.md) documentation.
 ## Start the pull mirroring process for a Project **(STARTER)**
-> Introduced in [GitLab Starter](https://about.gitlab.com/pricing) 10.3.
+> Introduced in [GitLab Starter](https://about.gitlab.com/pricing/) 10.3.
 ```
 POST /projects/:id/mirror/pull

doc/api/releases/index.md

@@ -300,18 +300,18 @@ Create a Release. You need push access to the repository to create a Release.
 POST /projects/:id/releases
 ```
-| Attribute          | Type            | Required | Description                                                                                                                      |
-| -------------------| --------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------- |
-| `id`               | integer/string  | yes      | The ID or [URL-encoded path of the project](../README.md#namespaced-path-encoding).                                              |
-| `name`             | string          | yes      | The release name.                                                                                                                |
-| `tag_name`         | string          | yes      | The tag where the release will be created from.                                                                                  |
-| `description`      | string          | yes      | The description of the release. You can use [markdown](../../user/markdown.md).                                                  |
-| `ref`              | string          | no       | If `tag_name` doesn't exist, the release will be created from `ref`. It can be a commit SHA, another tag name, or a branch name. |
-| `milestones`       | array of string | no       | The title of each milestone the release is associated with.                                                                      |
-| `assets:links`     | array of hash   | no       | An array of assets links.                                                                                                        |
-| `assets:links:name`| string          | required by: `assets:links` | The name of the link.                                                                                         |
-| `assets:links:url` | string          | required by: `assets:links` | The url of the link.                                                                                          |
-| `released_at`      | datetime        | no       | The date when the release will be/was ready. Defaults to the current time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). |
+| Attribute          | Type            | Required                    | Description                                                                                                                      |
+| -------------------| --------------- | --------                    | -------------------------------------------------------------------------------------------------------------------------------- |
+| `id`               | integer/string  | yes                         | The ID or [URL-encoded path of the project](../README.md#namespaced-path-encoding).                                              |
+| `name`             | string          | yes                         | The release name.                                                                                                                |
+| `tag_name`         | string          | yes                         | The tag where the release will be created from.                                                                                  |
+| `description`      | string          | yes                         | The description of the release. You can use [markdown](../../user/markdown.md).                                                  |
+| `ref`              | string          | yes, if `tag_name` doesn't exist | If `tag_name` doesn't exist, the release will be created from `ref`. It can be a commit SHA, another tag name, or a branch name. |
+| `milestones`       | array of string | no                          | The title of each milestone the release is associated with.                                                                      |
+| `assets:links`     | array of hash   | no                          | An array of assets links.                                                                                                        |
+| `assets:links:name`| string          | required by: `assets:links` | The name of the link.                                                                                                            |
+| `assets:links:url` | string          | required by: `assets:links` | The url of the link.                                                                                                             |
+| `released_at`      | datetime        | no                          | The date when the release will be/was ready. Defaults to the current time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). |
 Example request:

doc/api/users.md

@@ -596,7 +596,7 @@ PUT /user/status
 | Attribute | Type   | Required | Description |
 | --------- | ------ | -------- | ----------- |
-| `emoji`   | string | no     | The name of the emoji to use as status, if omitted `speech_balloon` is used. Emoji name can be one of the specified names in the [Gemojione index][gemojione-index]. |
+| `emoji`   | string | no     | The name of the emoji to use as status, if omitted `speech_balloon` is used. Emoji name can be one of the specified names in the [Gemojione index](https://github.com/bonusly/gemojione/blob/master/config/index.json). |
 | `message` | string | no     | The message to set as a status. It can also contain emoji codes. |
 When both parameters `emoji` and `message` are empty, the status will be cleared.
@@ -1363,5 +1363,3 @@ Example response:
 ```
 Please note that `last_activity_at` is deprecated, please use `last_activity_on`.
-
-[gemojione-index]: https://github.com/jonathanwiesel/gemojione/blob/master/config/index.json

doc/development/migration_style_guide.md

@@ -286,6 +286,48 @@ For a small table (such as an empty one or one with less than `1,000` records),
 use `add_column` and `change_column_default` in a single-transaction migration,
 combining it with other operations that don't require `disable_ddl_transaction!`.
+## Changing the column default
+
+One might think that changing a default column with `change_column_default` is an
+expensive and disruptive operation for larger tables, but in reality it's not.
+
+Take the following migration as an example:
+
+```ruby
+class DefaultRequestAccessGroups < ActiveRecord::Migration[5.2]
+  include Gitlab::Database::MigrationHelpers
+
+  DOWNTIME = false
+
+  def up
+    change_column_default :namespaces, :request_access_enabled, true
+  end
+
+  def down
+    change_column_default :namespaces, :request_access_enabled, false
+  end
+end
+```
+
+Migration above changes the default column value of one of our largest
+tables: `namespaces`. This can be translated to:
+
+```sql
+ALTER TABLE namespaces
+ALTER COLUMN request_access_enabled
+DEFAULT false
+```
+
+In this particular case, the default value exists and we're just changing the metadata for
+`request_access_enabled` column, which does not imply a rewrite of all the existing records
+in the `namespaces` table. Only when creating a new column with a default, all the records are going be rewritten.
+
+NOTE: **Note:**  A faster [ALTER TABLE ADD COLUMN with a non-null default](https://www.depesz.com/2018/04/04/waiting-for-postgresql-11-fast-alter-table-add-column-with-a-non-null-default/)
+was introduced on PostgresSQL 11.0, removing the need of rewritting the table when a new column with a default value is added.
+
+For the reasons mentioned above, it's safe to use `change_column_default` in a single-transaction migration
+without requiring `disable_ddl_transaction!`.
+
 ## Updating an existing column
 To update an existing column to a particular value, you can use