Detect if pipeline runs for a GitHub pull request

When using a mirror for CI/CD only we register a pull_request
webhook. When a pull_request webhook is received, if the
source branch SHA matches the actual head of the branch in the
repository we create immediately a new pipeline for the
external pull request. Otherwise we store the
pull request info for when the push webhook is received.

When using "only/except: external_pull_requests" we can detect
if the pipeline has a open pull request on GitHub and create or
not the job based on that.

Manoj M J authored 5 years ago and Mayra Cabrera committed 5 years ago

This change limits the number of emails
for new access requests notifications to
10 most recently active owners/maintainers

The flag defaulted to true, so there's no change unless users turned it
off. Given there's a lack of issues regarding object pools, this should
be OK.

Adam Hegyi authored 5 years ago and Mayra Cabrera committed 5 years ago

This change lays the foundation for customizable cycle analytics stages.
The main reason for the change is to extract the event definitions to
separate objects (start_event, end_event) so that it could be easily
customized later on.

- Adds UI to configure in group and project settings
- Removes notification configuration for users when
disabled at group or project level

Bob Van Landuyt :neckbeard: authored 5 years ago and Douwe Maan committed 5 years ago

**Prevention of running 2 simultaneous updates**

Instead of using `RemoteMirror#update_status` and raise an error if
it's already running to prevent the same mirror being updated at the
same time we now use `Gitlab::ExclusiveLease` for that.

When we fail to obtain a lease in 3 tries, 30 seconds apart, we bail
and reschedule. We'll reschedule faster for the protected branches.

If the mirror already ran since it was scheduled, the job will be
skipped.

**Error handling: Remote side**

When an update fails because of a `Gitlab::Git::CommandError`, we
won't track this error in sentry, this could be on the remote side:
for example when branches have diverged.

In this case, we'll try 3 times scheduled 1 or 5 minutes apart.

In between, the mirror is marked as "to_retry", the error would be
visible to the user when they visit the settings page.

After 3 tries we'll mark the mirror as failed and notify the user.

We won't track this error in sentry, as it's not likely we can help
it.

The next event that would trigger a new refresh.

**Error handling: our side**

If an unexpected error occurs, we mark the mirror as failed, but we'd
still retry the job based on the regular sidekiq retries with
backoff. Same as we used to

The error would be reported in sentry, since its likely we need to do
something about it.

This commit reduces I/O load and memory utilization during PostReceive
for the common case when no project hooks or services are set up.

We saw a Gitaly N+1 issue in `CommitDelta` when many tags or branches
are pushed. We can reduce this overhead in the common case because we
observe that most new projects do not have any Web hooks or services,
especially when they are first created. Previously, `BaseHooksService`
unconditionally iterated through the last 20 commits of each ref to
build the `push_data` structure. The `push_data` structured was used in
numerous places:

1. Building the push payload in `EventCreateService`
2. Creating a CI pipeline
3. Executing project Web or system hooks
4. Executing project services
5. As the return value of `BaseHooksService#execute`
6. `BranchHooksService#invalidated_file_types`

We only need to generate the full `push_data` for items 3, 4, and 6.

Item 1: `EventCreateService` only needs the last commit and doesn't
actually need the commit deltas.

Item 2: In addition, `Ci::CreatePipelineService` only needed a subset of
the parameters.

Item 5: The return value of `BaseHooksService#execute` also wasn't being
used anywhere.

Item 6: This is only used when pushing to the default branch, so if
many tags are pushed we can save significant I/O here.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/65878

Fic

As decided in https://gitlab.com/gitlab-org/gitlab-ce/issues/53593

Tiger Watson authored 5 years ago and Thong Kuah committed 5 years ago

Kubernetes deployments on new clusters will now have
a separate namespace per project environment, instead
of sharing a single namespace for the project.

Behaviour of existing clusters is unchanged.

All new functionality is controlled by the
:kubernetes_namespace_per_environment feature flag,
which is safe to enable/disable at any time.

Add spec for cases where URI.join does not work as expected

All cluster resources are now created on demand when
a deployment job starts.

* Reword Project#latest_successful_build_for to
Project#latest_successful_build_for_ref
* Reword Ci::Pipeline#latest_successful_for to
Ci::Pipeline#latest_successful_build_for_ref

These are not required because MySQL is not
supported anymore

- Fix `O(n)` complexity of `append_or_update_attribute`,
  we append objects to an array and re-save project
- Remove the usage of `keys.include?` as it performs `O(n)`
  search, instead use `.has_key?`
- Remove the usage of `.keys.first` as it performs a copy
  of all keys, instead use `.first.first`

Adds specs for testing the new behavior of specifying a pipeline when
POSTing a status.

Jan Provaznik authored 5 years ago and James Lopez committed 5 years ago

This pagination is not used anywhere so there is no reason
to keep it. It seems the usage of offset_id was probably
removed in 90c60138

Now we have terminals for instance and group clusters we can remove the
FF now. Deploying to group clusters has been working without complaints
too.

Tiger Watson authored 5 years ago and Thong Kuah committed 5 years ago

Project-level clusters that made use of this legacy
behaviour have been migrated to unmanaged clusters,
so we are now free to remove this fallback.

James Fargher authored 5 years ago and Thong Kuah committed 5 years ago

We are deprecating this service in favor of instance wide clusters.
Therefore we removed some code that is not anymore needed for a
readonly cluster and also we added some flags to allow for this
deprecation. These flags are to be removed in the next release when
we finally completelly remove KubernetesService.

updated documentation for Geo

This backports to CE changes that allow the recording of the
repository_type in the table lfs_objects_projects.

This is in order to allow future pruning of unreferenced LFS objects,
as we will need to know which repository to look in for the LFS pointer
file.

The EE MR that contains the original code and a full explanation of the
changes is
https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/13894

EE Issue https://gitlab.com/gitlab-org/gitlab-ee/issues/9490

Note that there was a lot of CE code changed in the EE MR because we
want to allow the wiki repository to also use LFS. See
https://gitlab.com/gitlab-org/gitlab-ce/issues/43721. As the wiki is
an unlicensed feature, a full backport is required to enable this.

Consider the scenario:

1. The default visibility level is set to internal
2. A user attempts to create a private project within a private group

Previously this would always fail because default_value_for would
overwrite the private visibility setting, no matter what
visibility_level were specified. This was happening because
default_value_for was confused by the default value of 0 specified by
the database schema.

default_value_for attempts to assign the default value in the block by
checking whether the attribute has changed. The problem is that since
the default value by the database was 0, and the user requested 0, this
appeared as though no changes were made. As a result, default_value_for
would always overwrite the user's preference.

To fix this, we remove the use of default_value_for and only set the
visibility level to the default application setting when no preference
has been given at creation time.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/63158

Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Update documentation to reflect removal

Fix milestone titles being leaked using search API
when users cannot read milestones

Francisco Javier López authored 5 years ago and James Lopez committed 5 years ago

Removed the conditions added to
Project.with_feature_available_for_user, and moved to the
IssuableFinder. Now, we ensure that, in the projects retrieved
in the Finder, the user has enough access for the feature.

This reverts commit a5378665.

This reverts merge request !26481

Imre (Admin) authored 5 years ago and Andreas Brandl committed 5 years ago

Move Contribution Analytics related spec in spec/features/groups/group_page_with_external_authorization_service_spec to EE