Fabio Pitino authored 2 years ago and Marius Bobin committed 2 years ago

Allow merge request pipeline creation to take in input
Git push options in order to support `ci.skip`.

Changelog: other
EE: true

RepositoryRemoveRemoteWorker's logic has been removed in 14.2 but
the worker itself needs to stay until 15.0 to drain the job queue.
It can be removed now.

Extract ServicePing payload computation and persistance
out of SubmitService into GitlabServicePingWorker
in order to pre compute SerivcePing data for preview
for instance admins even if those instances does not submit
this data later on to GitLab

Changelog: performance

Thong Kuah authored 2 years ago and Bob Van Landuyt :neckbeard: committed 2 years ago

The idea is that we want to distinguish which jobs ultimately comes from
a Cronjob

João Pereira authored 2 years ago and David Fernandez committed 2 years ago

Adds a new container_registry_migration_phase2_capacity_40
feature flag to enable a capacity of 40 concurrent migrations.

The Guard worker will now log the repository id and the repository path
to ease debugging.

Use correct gitlab_schema when creating records in test setup.

https://gitlab.com/gitlab-org/gitlab/-/issues/362821

https://gitlab.com/gitlab-org/gitlab/-/issues/362821#note_951807289

Updates the enqueuery next_repository query to
use LIMIT 2 to force the postgresql query planner
to use the index rather than a Seq Scan

This aims to add end-to-end integration testing for batched background
migrations.

Sean Arnold authored 2 years ago and Alexis Kalderimis committed 2 years ago

- Removes code to automatically create PromtheusAlerts.
This includes when the Prometheus integration is created.
- Also removes updates to the Prometheus Cluster. As adding and editing alerts is removed this is no longer needed.

Changelog: removed

This commit removes a worker used to remove artifacts for
a given job. Prefer using ExpireBuildArtifactsWorker which
is more performant.

when those are read from application settings. We need to convert values
to seconds first.

This sets the data_consistency for the Ci::ArchiveTraceWorker to
sticky. Allowing the read queries before any write to be handled by a
replica.

The job is scheduled from the Ci::BuildFinishedWorker with some delay
built in. Which means this job can take the 0.8s wait in the
beginning, and is very likely to find a replica that is up to date
with the WAL locations at the time of scheduling.

To increase the effect of this, this also preloads some associations
in the worker. That way, we can take advantage of loading them from a
replica rather than doing it lazily from the primary.

David Fernandez authored 2 years ago and Bob Van Landuyt :neckbeard: committed 2 years ago

In loop mode, the Enqueuer will loop on migration starts until either:
* it fills the capacity
* it hits the loop deadline
* the loop mode feature flag is disabled

This makes sure that the ProjectImportScheduleWorker uses replicas for
it's reads in the beginning of the job.

We need to allow retries for the `delayed` strategy, because that will
raise an error if it cannot find an up-to-date replica in 0.8s the
first time.

If the job can't find an up-to-date replica in the retry, it will use
the primary instead.

* Encapsulate the redis logic for inactive projects deletion feature
inside InactiveProjectsDeletionWarningTracker.
* Make the workers idempotent.
* Update the feature_category to the correct owners.

Steve Abrams authored 2 years ago and Adam Hegyi committed 2 years ago

Add application settings to define timeouts for
long running container registry migration
pre-imports and imports.

Changelog: added

Administrators of large GitLab instances often find that over time
projects become inactive and are no longer used.  These projects take
up unnecessary disk space. This feature adds an automated way to
identify these projects, warn the maintainers ahead of time, and then
delete the projects if they are still inactive.

Changelog: added

Max Woolf authored 2 years ago and Terri Chu committed 2 years ago

Instance administrators can no longer allow
expired SSH keys to work.

Changelog: removed
EE: true

When closing issues errors out, it can stop the execution of
`PostMergeService` which can result to other operations not being
executed.

To ensure that we still run the rest of logic in `PostMergeService`
when that happens, we close issues asynchronously.

This is behind the `async_mr_close_issue` feature flag as a new
worker is being introduced.

The deployment hooks worker can run against
a runner's job request API call which lacks user,
project or group identities in certain cases.

In such cases it is difficult to locate the exact
log that was responsible for firing a webhook or
integration event related to a deployment because
the volume of overall deployments per minute is
pretty high and no distinguishable fields are
present to filter it any further.

This change adds ID values to the logging
that allows us to track the deployment by ID
or its associated project by ID. The expectation
is that these IDs are always printed regardless
of the scope of the runners.

Changelog: added

Steve Abrams authored 2 years ago and Bob Van Landuyt :neckbeard: committed 2 years ago

Adds container_registry_migration_phase2_enqueue_twice ff
to allow the Enqueuer to be called twice each time a
container repository finishes the import path

This reverts MR
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86669.

David Fernandez authored 2 years ago and Bob Van Landuyt :neckbeard: committed 2 years ago

to take the repository id into account. This way, if two executions in
parallel select different repositories, both can continue their
execution.

Previously RSpec only warned in the following cases:

   expect(foo).to raise_error

   expect(foo).not_to raise_error(SomeSpecifcError)

and suggested to correct potential false positives with:

   expect(foo).to raise_error(SomeSpecifcError)

   expect(foo).not_to raise_error

This commit corrects all warnings and make RSpec fail instead of warn in
such situations preventing future false positives.

Update Projects::RecordTargetPlatformsWorker to handle detection of
projects built for the Android platform. It executes a new service
(Projects::AndroidTargetPlatformDetectorService) to determine if a given
project that uses Java or Kotlin is an Android project by checking if
the file AndroidManifest.xml is present in the project repository.

Add the flag file_extraction_pipeline! to the
ProjectAttributesPipeline in bulk import to keep re-enqueuing the
pipeline until the exported file is generated and download from the
source instance

Changelog: fixed

We have jobs to check the consistency between Namespace/Project
tables and their mirror tables.
With this MR, we will detect them and fix them automatically

Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/359855

Alan (Maciej) Paruszewski authored 2 years ago and Dmitry Gruzd committed 2 years ago

Changelog: removed
EE: true

Markus Koller authored 2 years ago and Etienne Baqué committed 2 years ago

- Add a `#log_exception` helper to format exceptions.
- Also log the integration record ID.
- Log through the integration model in `Jira::Requests::Base` instead
  of including `Integrations::Loggable`.

Besides the formatting changes, there are two behaviour changes in
`ProjectServiceWorker`:

- The log message now goes to `integrations_json` instead of
  `exceptions_json`.
- We stop logging errors when the integration record can't be found,
  and instead just exit silently.

Changelog: changed

Changelog: other

Will extract into `master`.

Changelog: other