Skip to content

Log pg statistics at the end of a test run

George Koltsov requested to merge jmc-pg-logging into master

What does this MR do and why?

Resolves https://gitlab.com/gitlab-org/gitlab-qa/-/issues/709

At the completion of the E2E tests in the master pipelines, the test environment, including the postgres db is destroyed. In https://gitlab.com/gitlab-org/gitlab/-/issues/396649 the performance of GitLab was severely impacted due to an unusual amount of database rows being written to pg. This was difficult to debug as there were no pg statistics logged.

This MR logs postgres statistics to a pg_stats.log file and saves them as an artifact of the test job. It captures:

  • Live and dead row counts after the test run
  • Cumulative user table statistics after the test run

How to set up and validate locally

Here is a GitLab MR using using this branch of gitlab-qa.

From this job there is a pg_stats.log artifact (direct link) which has the output.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by George Koltsov

Merge request reports