Andrew Newdigate authored 5 years ago and Nick Thomas committed 5 years ago

This change adds Distributed Tracing support for two new types of events

1. Redis Calls
1. ActiveSupport (Rails) Caching Operations

The intention is to help application developers and infrastructure
SREs to understand the pressure that caching operations can have on
the application when running at scale.

The Redis and Caching spans can be viewed in the Jaeger UI by clicking
the "Trace" link in the performance bar when running on GDK.

This change is a fairly straightforward refactor to extract the tracing
and correlation-id code from the gitlab rails codebase into the new
LabKit-Ruby project.

The corresponding import into LabKit-Ruby was in
https://gitlab.com/gitlab-org/labkit-ruby/merge_requests/1

The code itself remains very similar for now.

Extracting it allows us to reuse it in other projects, such as
Gitaly-Ruby. This will give us the advantages of correlation-ids and
distributed tracing in that project too.

This change adds three new instrumentations, driven through rails
notifications: render_template.action_view,
render_collection.action_view and render_partial.action_view.

These can help developers understand why renders are taking a long
time which may in turn help them to improve their performance.

This change will publish distributed tracing spans related to
SQL ActiveRecord queries. These queries are unbound, so will still
contain parameter substitution strings in place of actual parameter
values.

This ensures that the SQL strings do not need to be scrubbed, since the
values are not included in the SQL.

More details of rails instrumentation can be found here:
https://guides.rubyonrails.org/active_support_instrumentation.html

This change allows the GitLab rails and sidekiq components to receive
tracing spans from upstream services such as Workhorse and pass these
spans on to downstream services including Gitaly and Sidekiq.

This change will also emit traces for incoming and outgoing requests
using the propagated trace information. This will allow operators and
engineers to view traces across the Workhorse, GitLab Rails, Sidekiq and
Gitaly components.

Additional intra-service instrumentation will be added in future
changes.

This change will instantiate an OpenTracing tracer and configure it
as the global tracer when the GITLAB_TRACING environment variable is
configured. GITLAB_TRACING takes a "connection string"-like value,
encapsulating the driver (eg jaeger, etc) and options for the driver.

Since each service, whether it's written in Ruby or Golang, uses the
same connection-string, it should be very easy to configure all
services in a cluster, or even a single development machine to be
setup to use tracing.

Note that this change does not include instrumentation or propagation
changes as this is a way of breaking a previous larger change into
components. The instrumentation and propagation changes will follow
in separate changes.