To finish migrating untracked files to uploads for installations that were affected by https://gitlab.com/gitlab-org/gitlab-ce/issues/42881

Or just to delete the temp table if it is empty and left behind.

Artifacts are in the middle of being migrated from ci_builds to
ci_job_artifacts. The expiration date is currently visible in both of
these tables and the test for whether an expired artifact is present
for a job is complex as it requires checking both the of the tables.

Add two new indexes, one on ci_builds.artifacts_expire_at and one on
ci_job_artifacts.expire_at to enable finding expired artifacts
efficiently.

And until the migration is finished, replace the SQL for finding
expired and non-expired artifacts with a hand-crafted UNION ALL based
query instead of using OR. This overcomes a database optimizer
limitation that prevents it from using these indexes.

When the migration is finished the next version should remove this
query and replace it with a much simpler query on just
ci_job_artifacts. See
https://gitlab.com/gitlab-org/gitlab-ce/issues/42561 for followup.

Since populating the fork networks was scheduled multiple times
because of bugs that needed to be fixed:

- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15595/
  Creating fork networks for projects that were missing the root of
  the fork network.

- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15709
  The API allowed creating forked_project_links without
  fork_network_members.

Scheduling this migration multiple times would case it to run
concurrently. Which in turn would try to insert the same data into
`fork_network_members` causing duplicate key errors.

This avoids running the migration multiple times.

Fixes #32282.

EE seems to have had an outdated schema at some point, leading to some
environments not having the right columns in place. This adjusts the
migration for `issues.closed_at` so it takes care of those cases,
ensuring data can be migrated properly.

Fixes https://gitlab.com/gitlab-org/gitlab-ee/issues/4803

In the event of Sidekiq jobs getting lost there may be some rows left to
migrate. This migration ensures any remaining jobs are completed and
that all data has been migrated.

This fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/41595

Search query is especially slow if a user searches a generic string
which matches many records, in such case search can take tens of
seconds or time out. To speed up the search query, we search only for
first 1000 records, if there is >1000 matching records we just display
"1000+" instead of precise total count supposing that with such amount
the exact count is not so important for the user.

Because for issues even limited search was not fast enough, 2-phase
approach is used for issues: first we use simpler/faster query to get
all public issues, if this exceeds the limit, we just return the limit.
If the amount of matching results is lower than limit, we re-run more
complex search query (which includes also confidential issues).
Re-running the complex query should be fast enough in such case because the
amount of matching issues is lower than limit.

Because exact total_count is now limited, this patch also switches to
to "prev/next" pagination.

Related #40540

Get rid of a Rails 5 deprecation warning in db/migrate/20170425112128_create_pipeline_schedules_table.rb

Signed-off-by: Rémy Coutable <remy@rymai.me>

[10.3] Migrate `can_push` column from `keys` to `deploy_keys_project`

See merge request gitlab/gitlabhq!2276

(cherry picked from commit f6ca52d31bac350a23938e0aebf717c767b4710c)

1f2bd3c0 Backport to 10.3

Previously, the last push widget would only show when the branch never had
a merge request associated with it - even merged or closed ones. Now the
widget will disregard merge requests that are merged or closed.

For each MR diff an extra 'SELECT COUNT()' is executed
to get number of commits for the diff. Overall time to get counts for
all MR diffs may be quite expensive. To speed up loading of MR info,
information about number of commits is stored in a MR diff's extra column.

Closes #38068

Originally from branch 'fix-authorized-keys-enabled-default-2738' via merge request https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2240

Removed background migrations which were intended to fix state after using Gitlab
without a default having been set

Squashed commits:
Locally, if Spring was not restarted, `current_application_settings` was still cached, which prevented the migration from editing the file. This will also ensure that any app server somehow hitting old cache data will properly default this setting regardless.
Retroactively fix migration
  This allows us to identify customers who ran the broken migration. Their `authorized_keys_enabled` column does not have a default at this point.
  We will fix the column after we fix the `authorized_keys` file.
Fix authorized_keys file if needed
Add default to authorized_keys_enabled setting
  Reminder: The original migration was fixed retroactively a few commits ago, so people who did not ever run GitLab 9.3.0 already have a column that defaults to true and disallows nulls. I have tested on PostgreSQL and MySQL that it is safe to run this migration regardless.
  Affected customers who did run 9.3.0 are the ones who need this migration to fix the authorized_keys_enabled column.
  The reason for the retroactive fix plus this migration is that it allows us to run a migration in between to fix the authorized_keys file only for those who ran 9.3.0.
Tweaks to address feedback
Extract work into background migration
Move batch-add-logic to background migration
  Do the work synchronously to avoid multiple workers attempting to add batches of keys at the same time.
  Also, make the delete portion wait until after adding is done.
Do read and delete work in background migration
Fix Rubocop offenses
Add changelog entry
Inform the user of actions taken or not taken
Prevent unnecessary `select`s and `remove_key`s
Add logs for action taken
Fix optimization
Reuse `Gitlab::ShellAdapter`
Guarantee the earliest key
Fix migration spec for MySQL

Originally branch 'mk-toggle-writing-to-auth-keys-1631'

See merge request !2004

Squashed commits:
Add authorized_keys_enabled to Application Settings
Ensure default settings are exposed in UI
Without this change, `authorized_keys_enabled` is unchecked when it is nil, even if it should be checked by default.
Add “Speed up SSH operations” documentation
Clarify the reasons for disabling writes
Add "How to go back" section
Tweak copy
Update Application Setting screenshot

Add find key by base64 key or fingerprint to the internal API

See merge request !250

Squashed changes:
Add unique index to fingerprint
Add new index to schema
Add internal api to get ssh key by fingerprint
Change API endpoint to authorized_keys
Add InsecureKeyFingerprint that calculates the fingerprint without shelling out
Add require for gitlab key fingerprint
Remove uniqueness of fingerprint index
Remove unique option from migration
Fix spec style in fingerprint test
Fix rubocop complain
Extract insecure key fingerprint to separate file
Change migration to support building index concurrently
Remove those hideous tabs

This already existed in EE with a different timestamp, so rename for CE and make
it idempotent.

When a project uses fast-forward merging strategy user has
to rebase MRs to target branch before it can be merged.
Now user can do rebase in UI by clicking 'Rebase' button
instead of doing rebase locally.

This feature was already present in EE, this is only backport
of the feature to CE. Couple of changes:
* removed rebase license check
* renamed migration (changed timestamp)

Closes #40301

This change is required because otherwise if a user is created with a
value for `projects_limit` that matches the DB default, it gets
overwritten by `current_application_settings.default_projects_limit`. By
removing the default we once again can allow a user to be created with a
limit of 10 projects without the risk that it'll change to 10000