Change index on ci_builds to optimize Jobs Controller
Change the index on project_id
to be on project_id, id
to enable paged listing of ci_builds for a given project.
Database Checklist
When adding migrations:
-
Updated db/schema.rb
-
Added a down
method so the migration can be reverted -
Added the output of the migration(s) to the MR body -
Added the execution time of the migration(s) to the MR body -
Added tests for the migration in spec/migrations
if necessary (e.g. when migrating data) -
Made sure the migration won't interfere with a running GitLab cluster, for example by disabling transactions for long running migrations
When adding indexes:
-
Described the need for these indexes in the MR body -
Made sure existing indexes can not be reused instead
When removing columns, tables, indexes or other structures:
-
Removed these in a post-deployment migration -
Made sure the application no longer uses (or ignores) these structures
General Checklist
-
Changelog entry added, if necessary - Review
-
Has been reviewed by Backend -
Has been reviewed by Database
-
-
Conform by the merge request performance guides -
Conform by the style guides -
Squashed related commits together
stark@tweedle:~/gitlab/gdk/gitlab-development-kit/gitlab$ bundle exec rake db:migrate
== 20170927095921 AddCiBuildsIndexForJobscontroller: migrating ================
-- index_exists?(:ci_builds, [:project_id, :id])
-> 0.0079s
-- transaction_open?()
-> 0.0000s
-- execute("SET statement_timeout TO 0")
-> 0.0002s
-- add_index(:ci_builds, [:project_id, :id], {:algorithm=>:concurrently})
-> 0.0996s
-- index_exists?(:ci_builds, :project_id)
-> 0.0089s
-- transaction_open?()
-> 0.0000s
-- select_one("SELECT current_setting('server_version_num') AS v")
-> 0.0004s
-- execute("SET statement_timeout TO 0")
-> 0.0002s
-- remove_index(:ci_builds, {:algorithm=>:concurrently, :column=>:project_id})
-> 0.0114s
== 20170927095921 AddCiBuildsIndexForJobscontroller: migrated (0.1294s) =======
The old query plan:
gitlabhq_production=# explain SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."project_id" = 7764 AND "ci_builds"."status" IN ('pending', 'running', 'success', 'failed', 'canceled', 'skipped', 'manual') ORDER BY id DESC LIMIT 30 OFFSET 0;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=0.56..2903.76 rows=30 width=1373)
-> Index Scan Backward using ci_builds_pkey on ci_builds (cost=0.56..8183160.24 rows=84560 width=1373)
Filter: (((type)::text = 'Ci::Build'::text) AND (project_id = 7764) AND ((status)::text = ANY ('{pending,running,success,failed,canceled,skipped,manual}'::text[])))
(3 rows)
The new query plan:
gitlabhq_development***=# explain SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."project_id" = 7764 AND "ci_builds"."status" IN ('pending', 'running', 'success', 'failed', 'canceled', 'skipped', 'manual') ORDER BY created_at DESC LIMIT 30 OFFSET 0;
┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ QUERY PLAN │
├──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Limit (cost=0.28..8.30 rows=1 width=501) │
│ -> Index Scan Backward using tmp2 on ci_builds (cost=0.28..8.30 rows=1 width=501) │
│ Index Cond: (project_id = 7764) │
│ Filter: (((type)::text = 'Ci::Build'::text) AND ((status)::text = ANY ('{pending,running,success,failed,canceled,skipped,manual}'::text[]))) │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
Edited by yorickpeterse-staging