Detect if pipeline runs for a GitHub pull request

When using a mirror for CI/CD only we register a pull_request
webhook. When a pull_request webhook is received, if the
source branch SHA matches the actual head of the branch in the
repository we create immediately a new pipeline for the
external pull request. Otherwise we store the
pull request info for when the push webhook is received.

When using "only/except: external_pull_requests" we can detect
if the pipeline has a open pull request on GitHub and create or
not the job based on that.

We migrated all logic to `PipelineProcessWorker`
and this worker become redundant.

This implements the support for `needs:` keyword
as part of GitLab CI. That makes some of the jobs
to be run out of order.

This allows the chaos endpoints to be invoked in Sidekiq so that this
environment can be tested for resilience.

- Add two new ActiveRecord models:
  - RootNamespaceStoragestatistics will persist root namespace statistics
  - NamespaceAggregationSchedule will save information when a new update
to the namespace statistics needs to be scheduled
- Inject into UpdateProjectStatistics concern a new callback that will
call an async job to insert a new row onto NamespaceAggregationSchedule
table
- When a new row is inserted a new job is scheduled. This job will
update call an specific service to update the statistics and after that
it will delete thee aggregated scheduled row
- The RefresherServices makes heavy use of arel to build composable
queries to update Namespace::RootStorageStatistics attributes.
- Add an extra worker to traverse pending rows on
NAmespace::AggregationSchedule table and schedule a worker for each one
of this rows.
- Add an extra worker to traverse pending rows on
NAmespace::AggregationSchedule table and schedule a worker for each one
of this rows

vshushlin authored 5 years ago and Nick Thomas committed 5 years ago

Add index for pages domain ssl auto renewal
Add PagesDomain.needs_ssl_renewal scope
Add cron worker for ssl renewal
Add worker for ssl renewal
Add pages ssl renewal worker queues settings

As we have a central domain for auto merge process today,
we should use a single worker for any auto merge process.

vshushlin authored 5 years ago and Nick Thomas committed 5 years ago

Domain will be removed by verification worker after 1 week
of being disabled

Add endpoint to delete/uninstall a cluster application

+ to monitor progress of uninstallation pod