Hordur Freyr Yngvason authored 5 years ago and Evan Read committed 5 years ago

The database load balancing docs had some mention of the logs that
needed to be updated, and the logs doc had an absolute link to
gitlab.com instead of a relative link to a markdown file.

Stan Hu authored 5 years ago and Mayra Cabrera committed 5 years ago

Current `auth.log` uses `fullpath` and `ip`, while `api_json.log` uses
`remote_ip` and `path` for the same fields. Let's standardize these
namings to make it easier for people working with the data.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/66167

Mayra Cabrera authored 5 years ago and Evan Read committed 5 years ago

Requests over the rate limit are logged into `auth.log`

- move uploads created by AttachmentUploader
- handle also files created for legacy_diff_notes

The githost.log name was often confused with GitHost.io. Rename this to
git_json.log to make it clear it's coming from Git and that it's JSON
data.

Luke Duncalfe authored 5 years ago and James Lopez committed 5 years ago

A project can have the same `LfsObject` linked with up to three
`LfsObjectsProject` records. Each of these records would be for a
different repository, recorded in the `repository_type` property. The
different repositories at time of writing are "project", "wiki", and
"design". See https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/13894

This change exports the list of `repository_type`s as a JSON mapping of
oid => repository_types, which are imported to recreate the correct
`LfsObjectsProject` records.

https://gitlab.com/gitlab-org/gitlab-ee/issues/11090

Stan Hu authored 5 years ago and Mayra Cabrera committed 5 years ago

This will make it easier to parse this in production and adds the
`correlation_id` if available.

Marcel Amirault authored 5 years ago and Achilleas Pipinellis committed 5 years ago

Change assorted absolute links to ce and ee docs
to relative links

From 12.1, user information (id and username) are also included on
auth.log. Documentation was updated to reflect this.

Related to https://gitlab.com/gitlab-org/gitlab-ce/issues/62756

https://community.atlassian.com/t5/Jira-questions/Is-it-quot-JIRA-quot-or-quot-Jira-quot/qaq-p/681163



Signed-off-by: Takuya Noguchi <takninnovationresearch@gmail.com>

Charlie Ablett authored 5 years ago and Charlie Ablett committed 5 years ago

- Comply doc with guidelines
- Improve tests for readability and completeness
- Separate out phases visually with newlines
- Add `format_message` test
- test readability
- code and test structure/styling
- static query analyzers
- call `as_json` on `provided_variables`
- add exception handling

Creates a new filename to register auth logs.
This change should allow SRE's queries to make better queries
through logging infrastructure.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/54528

This makes it easier to debug Gitaly performance issues in the field.

This commit also makes the tracking of query time thread-safe via
RequestStore.

`queue_duration` is a useful metric that is currently in api_json.log
but not in production_json.log. We should add it because it tells us how
long the request sat in Workhorse before Unicorn processed it. Having
this field enables the support team to better troubleshoot when delays
began to happen.

This will add audit_json.log that writes one line per audit event. For
example:

{
       "severity":"INFO",
       "time":"2018-10-17T17:38:22.523Z",
       "author_id":3,
       "entity_id":2,
       "entity_type":"Project",
       "change":"visibility",
       "from":"Private",
       "to":"Public",
       "author_name":"John Doe4",
       "target_id":2,
       "target_type":"Project",
       "target_details":"namespace2/project2"
}

Adds an importer.log so that it's easier to track what's happening with the import.

Closes #20060

The Sidekiq exporter logs were mixing with the normal Sidekiq logs. In order
to support structured logging in Sidekiq, we either need to split this data
out or convert the exporter to produce structured logs. Since Sidekiq job
processing is fundamentally different information from Web server traffic,
it seems cleaner to move the metrics traffic into a separate file, where they
can be parsed by a different filter if needed.

Relates to #20060