- Add new service that transfers milestones
from a group to a project
- Include new service in Projects transfer service
- Include FromUnion module in Milestone model
to use in transfer service
- Add specs for new milestones service
- Add specs for transferring milestones in
project transfer service

During a project import, `LfsLinkService` attempts to link `LfsObjects`
that have not already been associated with a project.  It's possible for
a large repo to have thousands of OIDs, which can cause long database
query and parsing times.  By processing a batch of 1000 at a time, we
can reduce that time at the expense of a few more SQL queries.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/66274

Luke Duncalfe authored 5 years ago and Mayra Cabrera committed 5 years ago

Notes call `#after_note_created` and `#after_note_destroyed` on their
noteable in callbacks, so the noteable can perform tasks particular to
them, like cache expiry.

This is in preparation of the EE-specific class
`DesignManagement::Design` clearing its `user_notes_count` cache when
its note are created or destroyed.

Refactoring Rspec behaviour testing of a counter caching service into a
shared example.

https://gitlab.com/gitlab-org/gitlab-ee/issues/13353

Adding, destroying and toggling emoji previously lacked services and
instead were performed through methods called on Awardable models.

This led to inconsistencies where relevant todos would be marked as done
only when emoji were awarded through our controllers, but not through
the API. Todos could also be marked as done when an emoji was being
removed.

Behaviour changes

- Awarding emoji through the API will now mark a relevant Todo as done
- Toggling an emoji off (destroying it) through our controllers will no
  longer mark a relevant Todo as done

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/63372

- Adds UI to configure in group and project settings
- Removes notification configuration for users when
disabled at group or project level

Add Gitlab::VisibilityLevelChecker that verifies
selected project visibility level (or overridden param)
is not restricted when creating or importing a project

Bob Van Landuyt :neckbeard: authored 5 years ago and Douwe Maan committed 5 years ago

**Prevention of running 2 simultaneous updates**

Instead of using `RemoteMirror#update_status` and raise an error if
it's already running to prevent the same mirror being updated at the
same time we now use `Gitlab::ExclusiveLease` for that.

When we fail to obtain a lease in 3 tries, 30 seconds apart, we bail
and reschedule. We'll reschedule faster for the protected branches.

If the mirror already ran since it was scheduled, the job will be
skipped.

**Error handling: Remote side**

When an update fails because of a `Gitlab::Git::CommandError`, we
won't track this error in sentry, this could be on the remote side:
for example when branches have diverged.

In this case, we'll try 3 times scheduled 1 or 5 minutes apart.

In between, the mirror is marked as "to_retry", the error would be
visible to the user when they visit the settings page.

After 3 tries we'll mark the mirror as failed and notify the user.

We won't track this error in sentry, as it's not likely we can help
it.

The next event that would trigger a new refresh.

**Error handling: our side**

If an unexpected error occurs, we mark the mirror as failed, but we'd
still retry the job based on the regular sidekiq retries with
backoff. Same as we used to

The error would be reported in sentry, since its likely we need to do
something about it.

David Wilkins authored 5 years ago and Ash McKenzie committed 5 years ago

- Closes #60024

- Change PrometheusClient.new to accept a base url instead of an
  already created RestClient

- Use Gitlab::HTTP in PrometheusClient instead of creating RestClient
  in PrometheusService

- Move http_options from PrometheusService to
  PrometheusClient (follow_redirects: false)

- ensure that base urls don't have the trailing slash

- Created a `PrometheusClient#url` method that might not be strictly
  required

- Change rescued exceptions from RestClient::* to
  HTTParty::ResponseError where possible and StandardError for the
  rest

https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21679 moved the
deletion of registry tags outside of a transaction, but introduced an
issue where Sidekiq would attempt to contact the container registry
during project destruction even if it were disabled.

Relates to:

* https://gitlab.com/charts/gitlab/issues/1455
* https://gitlab.com/gitlab-org/gitlab-ce/issues/45941

This MR adds new application setting to network section
`allow_local_requests_from_system_hooks`. Prior to this change
system hooks were allowed to do local network requests by default
and we are adding an ability for admins to control it.

We should just ignore these errors and move along with the deletion
since the repositories are going to be trashed anyway.

Closes https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/31164

This means we have one less Net::HTTP monkeypatch.

sham_rack cannot handle IPv6 addresses which means it breaks Net::HTTP
connections because it monkey-patches Net::HTTP

These are not required because MySQL is not
supported anymore

Pages access level currently depends on project visibilty
which is ignored by factory, this commit fixes that

When the SQL query cache is active, the SELECT query for finding
projects to apply service templates returns the same values. This causes
an infinite loop because even though bulk INSERT queries are made, the
cached results never reflect that progress.  To fix this, we call
`Project.uncached` around the query to ensure new data is retrieved.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/63595

updated documentation for Geo

Consider the scenario:

1. The default visibility level is set to internal
2. A user attempts to create a private project within a private group

Previously this would always fail because default_value_for would
overwrite the private visibility setting, no matter what
visibility_level were specified. This was happening because
default_value_for was confused by the default value of 0 specified by
the database schema.

default_value_for attempts to assign the default value in the block by
checking whether the attribute has changed. The problem is that since
the default value by the database was 0, and the user requested 0, this
appeared as though no changes were made. As a result, default_value_for
would always overwrite the user's preference.

To fix this, we remove the use of default_value_for and only set the
visibility level to the default application setting when no preference
has been given at creation time.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/63158

Enable Get and Update of ci_default_git_depth for
Project API.

Renaming Project#default_git_depth to :ci_default_git_depth
to give more context through the API usage.

Add API documentation

If the origin project has no default_git_depth set (i.e. nil) set the
fork's default_git_depth to 0

Krasimir Angelov authored 5 years ago and Fabio Pitino committed 5 years ago

Introduce default_git_depth in project's CI/CD settings and set it to
50. Use it if there is no GIT_DEPTH variable specified. Apply this
default only to newly created projects and keep it nil for old ones
in order to not break pipelines that rely on non-shallow clones.

default_git_depth can be updated from CI/CD Settings in the UI, must be
either nil or integer between 0 and 1000 (incl).

Inherit default_git_depth from the origin project when forking projects.

MR pipelines are run on a MR ref (refs/merge-requests/:iid/merge) and it
contains unique commit (i.e. merge commit) which doesn't exist in the
other branch/tags refs. We need to add it cause otherwise it may break
pipelines for old projects that have already enabled Pipelines for merge
results and have git depth 0.

Document new default_git_depth project CI/CD setting

Delete Todos for guest users when project visibility
level is updated to private.

We've already migrated all the legacy artifacts to the new realm,
which is ci_job_artifacts table.
It's time to remove the old code base that is no longer used.

Douwe Maan authored 5 years ago and Oswaldo Ferreir committed 5 years ago

Gitlab::HTTP now resolves the hostname only once, verifies the IP is not
blocked, and then uses the same IP to perform the actual request, while
passing the original hostname in the `Host` header and SSL SNI field.

GitDeduplicationService performs idempotent operations on deduplicated
projects.

Immediate configuration is not ideal for group and instance
level clusters as projects that may never be deployed would
still have Kubernetes namespaces and service accounts created
for them.

As of https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/25586
we now create only the resources that are required for the
project being deployed, at the time of deployment.

Pull mirrors would run the `Projects::AfterImportService`, which would
force a `git gc` each time it finished. This is overkill and not
necessary now that we have refs packed more frequently
(https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27826).

Closes https://gitlab.com/gitlab-org/gitlab-ee/issues/11556

Project imports were failing with `undefined method each_with_object for
String` because the import was attempting to parse the LFS Batch API and
failing due to the fact that the Content-Type wasn't a supported format
(e.g. application/vnd.git-lfs+json instead of application/json). We now
parse the body as JSON.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/61624

Fixes #60425

PackRefs is not an expensive gitaly call - we want to
call it more often (than as part of full `gc`) because
it helps to keep number of refs files small - too many
refs file may be a problem for deployments with
slow storage.

Francisco Javier López authored 5 years ago and James Lopez committed 5 years ago

In order to make `LfsImportService` more reusable, we
need to extract the logic inside `ImportService` and
encapsulate it into the service.

This new table will be used to store the external_dashboard_url which
allows users to add a link to their external dashboards (ex Grafana)
to the Metrics dashboard.

Probably useful as we often move these files to "new" files.

Imre (Admin) authored 5 years ago and Andreas Brandl committed 5 years ago

Move Contribution Analytics related spec in spec/features/groups/group_page_with_external_authorization_service_spec to EE

This reverts merge request !26823

Imre (Admin) authored 5 years ago and Andreas Brandl committed 5 years ago

Move Contribution Analytics related spec in spec/features/groups/group_page_with_external_authorization_service_spec to EE