- Adds a suite of rspec tests for the gitlab workhorse helper
- Checks for and removes orphan workhorse socket files if present

Signed-off-by: Robert Marshall <rmarshall@gitlab.com>

Unlike AWS and Google, Azure needs to use an Azure client inside
Workhorse to support direct uploads. Using standard HTTP transfers with
pre-signed URLs with the Azure Put Blob API
(https://docs.microsoft.com/en-us/rest/api/storageservices/put-blob)
doesn't work because Azure doesn't support chunked transfer encoding.
However, Azure does support uploading files in segments via the Put
Block and Put Block List API
(https://docs.microsoft.com/en-us/rest/api/storageservices/put-block),
but this requires an Azure client.

To support this, this commit extracts the Azure Fog credentials from the
Rails connection information and adds them to the Workhorse
configuration.

This changes requires two merge requests to work:

1. https://gitlab.com/gitlab-org/gitlab-workhorse/-/merge_requests/555
2. https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38882

Part of https://gitlab.com/gitlab-org/gitlab/-/issues/25877

Ian Baum authored 4 years ago and Balasankar C committed 4 years ago

* Matching behavior in rails codebase
* This will be default behavior in 4.0

This adds support for the gitlab.yml changes in
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/34460. This allows
GitLab to use the Workhorse S3 client with a single credential, enabling
proper MD5 checksums and multipart uploads, and paves the way for
user-specified keys for encrypted buckets.

These changes require
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35480 to work.

As discussed in
https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/4324, we
can set the default attribute for `gitlab_workhorse['auth_socket']` to
`nil` as the empty string value so we can use
`node['gitlab']['gitlab-workhorse']['auth_socket']` in the template
instead of `Gitlab['gitlab_workhorse']['auth_socket']`.

In Workhorse, it's not absolutely necessary to have both -authSocket and
-authBackend arguments, as long as least one is available to all GitLab
application hosts. In a single-node Ominbus install, -authSocket will
suffice, but in multi-node installations -authBackend may only be
needed.

To support the second case and make it possible to disable listening to
a UNIX socket, admins can do the following:

```
gitlab_workhorse['auth_socket'] = nil
gitlab_workhorse['auth_backend'] = 'https://workhorse.example.com'
```

Ian Baum authored 4 years ago and DJ Mountney committed 4 years ago

* Incoroporates our changes to the coobkook so we get our desired
  behavior
* Use start_down instead of down
* Stop checking for permissions on log directories. The runit cookbook does not explicitly create this.
* Rename runit control templates. Don't use -control in the name anymore, upstream has removed that
* In the v5 version of the runit cookbook, they are colliding with the chef service properties

1. Step into runit_service resource
2. Check for reload_log_dir block to be notified
3. Pass options as needed by runit_service

This merge request brings redis[s]:// support to Workhorse and Rails.
This makes it possible for users to set up stunnel to provide encryption
via https://redislabs.com/blog/stunnel-secure-redis-ssl/ or use AWS
ElasticCache.

This also deprecates the tcp:// scheme since it is no longer needed by
Workhorse.

Trusted Redis SSL certificates should be managed by the standard Omnibus
procedure:

https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates

Note that Redis Sentinel does not yet support SSL.
https://github.com/antirez/redis/pull/4855 may bring native support to
Redis 6.0, but time will tell.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/47702

* Enable workhorse metrics on `localhost:9229` by default.
* Add to Prometheus scrape config.

* Update the database upgrade actions to be more clear about what they
are doing
* Move the mocking of helper files out of the individual spec files,
into the main configuration