Skip to content
Snippets Groups Projects
Commit ba9b4c4d authored by Sean McGivern's avatar Sean McGivern
Browse files

Avoid hitting statement timeout finding MR pipelines

For MRs with many thousands of commits, `SELECT DISTINCT(sha)` will be very
slow.

What we can't do to fix this:

1. Add an index. Postgres won't use it for DISTINCT without a lot of ceremony.
2. Do the `uniq` in Ruby. That can still be very slow with hundreds of
   thousands of commits.
3. Use a subquery. We haven't removed the `st_commits` column yet, but we will
   soon.

Until 3 is available to us, we can just do 2, but also add a limit clause. There
is no ordering, so this may return different results, but our goal with these
MRs is just to get them to load, so it's not a huge deal.
parent e9d35261
No related branches found
No related tags found
No related merge requests found
Loading
Loading
@@ -874,7 +874,7 @@ class MergeRequest < ActiveRecord::Base
#
def all_commit_shas
if persisted?
column_shas = MergeRequestDiffCommit.where(merge_request_diff: merge_request_diffs).pluck('DISTINCT(sha)')
column_shas = MergeRequestDiffCommit.where(merge_request_diff: merge_request_diffs).limit(10_000).pluck('sha')
serialised_shas = merge_request_diffs.where.not(st_commits: nil).flat_map(&:commit_shas)
 
(column_shas + serialised_shas).uniq
Loading
Loading
---
title: Stop merge requests with thousands of commits from timing out
merge_request: 15063
author:
type: performance
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment