Adds `# frozen_string_literal: true` to spec/models ruby files

This completely rewrites the SnippetsFinder class from the ground up in
order to improve its performance. The old code was beyond salvaging. It
was complex, included various Rails 5 workarounds, comments that
shouldn't be necessary, and most important of all: it produced a really
poorly performing database query.

As a result, I opted for rewriting the finder from scratch, instead of
trying to patch the existing code. Instead of trying to reuse as many
existing methods as possible, I opted for defining new methods
specifically meant for the SnippetsFinder. This requires some extra code
here and there, but allows us to have much more control over the
resulting SQL queries. It is these changes that then allow us to produce
a _much_ more efficient query.

To illustrate how bad the old query was, we will use my own snippets as
an example. Currently I have 52 snippets, most of which are global ones.
To retrieve these, you would run the following Ruby code:

    user = User.find_by(username: 'yorickpeterse')

    SnippetsFinder.new(user, author: user).execute

On GitLab.com the resulting query will take between 10 and 15 seconds to
run, producing the query plan found at
https://explain.depesz.com/s/Y5IX. Apart from the long execution time,
the total number of buffers (the sum of all shared hits) is around 185
GB, though the real number is probably (hopefully) much lower as I doubt
simply summing these numbers produces the true total number of buffers
used.

The new query's plan can be found at https://explain.depesz.com/s/wHdN,
and this query takes between 10 and 100-ish milliseconds to run. The
total number of buffers used is only about 30 MB.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/52639

Remove superfluous lib: true, type: redis, service: true, models: true, services: true, no_db: true, api: true

Signed-off-by: Rémy Coutable <remy@rymai.me>

Douwe Maan authored 7 years ago and Bob Van Landuyt :neckbeard: committed 7 years ago

Refactor snippets finder & dont return internal snippets for external users

See merge request !2094

Signed-off-by: Rémy Coutable <remy@rymai.me>

This commit adds a number of _html columns and, with the exception of Note,
starts updating them whenever the content of their partner fields changes.

Note has a collision with the note_html attr_accessor; that will be fixed later

A background worker for clearing these cache columns is also introduced - use
`rake cache:clear` to set it off. You can clear the database or Redis caches
separately by running `rake cache:clear:db` or `rake cache:clear:redis`,
respectively.

This reverts commit 3e991230.

# Conflicts:
#	app/models/project.rb

There are several changes to this module:

1. The use of an explicit stack in Participable#participants
2. Proc behaviour has been changed
3. Batch permissions checking

== Explicit Stack

Participable#participants no longer uses recursion to process "self" and
all child objects, instead it uses an Array and processes objects in
breadth-first order. This allows us to for example create a single
Gitlab::ReferenceExtractor instance and pass this to any Procs. Re-using
a ReferenceExtractor removes the need for running potentially many SQL
queries every time a Proc is called on a new object.

== Proc Behaviour Changed

Previously a Proc in Participable was expected to return an Array of
User instances. This has been changed and instead it's now expected that
a Proc modifies the Gitlab::ReferenceExtractor passed to it. The return
value of the Proc is ignored.

== Permissions Checking

The method Participable#participants uses
Ability.users_that_can_read_project to check if the returned users have
access to the project of "self" _without_ running multiple SQL queries
for every user.

In 8278b763 the default behaviour of annotation
has changes, which was causing a lot of noise in diffs. We decided in #17382
that it is better to get rid of the whole annotate gem, and instead let people
look at schema.rb for the columns in a table.

Fixes: #17382

Previously this used a regular LIKE which is case-sensitive on
PostgreSQL. This ensures that for both PostgreSQL and MySQL the
searching is case-insensitive similar to searching for projects.

This was removed from the interface in
https://github.com/gitlabhq/gitlabhq/pull/6027 but its implementation
lingered around for two years.

Fixes #2384.

Snippet model was missing project association

Now there is a single source of information for which attribute a model
uses to be referenced, and its special character.

Signed-off-by: Jeroen van Baarsen <jeroenvanbaarsen@gmail.com>

Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>