Revert "Merge branch 'sh-optimize-projects-api' into 'master'"

Closes #60315

See merge request gitlab-org/gitlab-ce!27195

(cherry picked from commit 3b163a75)

bcfd04a2 Revert "Merge branch 'sh-optimize-projects-api' into 'master'"

This reverts merge request !26823

Imre (Admin) authored 5 years ago and Andreas Brandl committed 5 years ago

Move Contribution Analytics related spec in spec/features/groups/group_page_with_external_authorization_service_spec to EE

Thong Kuah authored 5 years ago and James Lopez committed 5 years ago

- rewords examples starting with 'should'
- rewords examples starting with 'it'

Note: I had to manually fixup "onlies" to "only"

Signed-off-by: Rémy Coutable <remy@rymai.me>

Adds `# frozen_string_literal: true` to spec/models ruby files

Previously when a user requested a list of projects,
`Project#public_or_visible_to_user` would search all authorized projects
and public/internal projects as well. However, when a user requests a
specific `visibility_level` (e.g. private), that should reduce the
search space, and we shouldn't need to load public/internal projects.

Improves https://gitlab.com/gitlab-org/gitlab-ce/issues/59329

This changes the repository type from a binary `wiki?` to a type. So
we can have more than 2 repository types.

Now everywhere we called `.wiki?` and expected a boolean, we check
that type.

We already check max boards in the create service. This is not used.

- Includes instance methods on Group model to detect when a group has
AutoDevOps explicitly/implicitly enabled/disabled.
- Includes migration to add a new column to namespaces table
- Add UI necessary modifications
- Add service and controller to update auto devops
related instances
- Updates project and groups auto devops badges

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/52447

Moved to HashedStorage namespace, and added them to the
`:hashed_storage` queue namespace

We are adding sidekiq workers and service classes to allow to rollback
a hashed storage migration. There are some refactoring involved as well
as part of the code can be reused by both the migration and the rollback
logic.

The API get projects/:id/traffic/fetches allows user with write
access to the repository to get the number of clones for the
last 30 days.

Pipeline status was showing failed on projects when CI was disabled.

This checks to see if the feature is enabled before showing the status.

When hashed storage is in use, it's helpful to have the project
name associated with the request.

Closes https://gitlab.com/gitlab-org/gitaly/issues/1394

Group guests will only be displayed merge requests to
projects they have a access level to, higher than Reporter.

Visible projects will still display the merge requests to Guests

After the import has finished, we flush (delete) the InternalId records
related to the project at hand. This means we're starting over with
tracking correct internal id values, a new record will be created
automatically when the next internal id is generated.

The GitHub importer assigns iid values by using supplied values from
GitHub. We skip tracking internal id values during the import in favor
of higher concurrency. Deleting any InternalId records after the import
has finished is an extra measure to guarantee consistency.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/54270.

Specs were reviewed and improved to better cover the current behavior.
There was some standardization done as well to facilitate the
implementation of the rollback functionality.

StorageMigratorWorker was extracted to HashedStorage namespace were
RollbackerWorker will live one as well.

Signed-off-by: Rémy Coutable <remy@rymai.me>

Group guests will only be displayed merge requests to
projects they have a access level to, higher than Reporter.

Visible projects will still display the merge requests to Guests

This refactors the code used for checking if a user has exceeded the
personal projects limit. As part of this refactor the method has been
renamed from Project#check_limit to "check_personal_projects_limit", as
this name makes it much more clear what the purpose of the method is.
Standalone unit tests have also been added, as before we only had a
single generic validation test that did not cover all cases.

The old implementation of the refactored method also included a `rescue`
statement. This code would only run when a project creator was not set.
The error that would be added wasn't super useful, especially since
there would already be errors for the creator not being present. As none
of the other code in the "check_personal_projects_limit" raises, it has
been removed.

GitLab Pages supports projects hosted under a subgroup, but not subgroup
websites. That means that only the highest-level group supports

i.e.:
You created a group for your engineering department called
`engineering`, a subgroup for all your documentation websites called
`docs`,and a project within this subgroup is called `workflows`.
Your project URL is `https://gitlab.com/engineering/docs/workflows/`.
Once you enable GitLab Pages for this project, the site will live under
`https://engineering.gitlab.io/docs/workflows`.

Previously the nullification wasn't done, as the only caller would later
destroy the model all together. In
https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/9070 this change
was made. This commit is basically a backport before that MR is merged.

The original intention of `get_build` was as a workaround not to violate
`CodeReuse/ActiveRecord`. `find_by_id` does the exact same thing but
does not violate the rubocop rule.

In the previous code, we locked the project during the migration
scheduling step, which works fine for small setups, but can be
problematic in really big installations.

We now moved the logic to inside the worker, so we minimize the time a
project will be read-only. We also make sure we only do that if
reference counter is `0` (no current operation is in progress).

project.latest_successful_build_for! is being tested inside of `describe
explicit. In doing so some duplication was generated but not
`#latest_successful_build_for!` has full coverage unlike before.

Move `create_pipeline` & `create_build` as helper methods for this spec
to reduce duplication.

Inside of `Projects::ArtifactsController` and
`Projects::BuildArtifactsController` we fetching the build by id using
active record directly which violates `CodeReuse/ActiveRecord` rubocop
rule. Create `get_build` inside of `project` model which does the same
thing.

`project.latest_successful_builds_for(ref)` is being used to find a
single job all the time. This results into us having to call `find_by`
inside of the controller which violates our CodeReuse/ActiveRecord
rubocop rule.

Refactor `project.latest_successful_builds_for(ref)` to
`project.latest_successful_build_for(job_name, ref)` which will execute
the `find_by` inside of the model.

Also create `project.latest_successful_build_for!(job_name, ref)` which
raises an exception instead of returning nil.