Merge request dashboard page takes over a minute to load

https://gitlab.com/dashboard/merge_requests?assignee_id=300478

That page takes over 1 minute to load Cc @stanhu @DouweM @smcgivern @pcarranza

Which means that it just times out the whole time?

Should this be part of the discussion team? I mean, we (infra/performance) could investigate why, propose a solution and hand it of to the discussion team. Does that make sense @jschatz1 ?

There are two changes which will be in the next RC that should help with this!

• More aggressively preload on merge request and issue index pages
• Precalculate authorized projects in database

I'll assign this to me for now to check the timings on that page after we've deployed the next RC.

Added ~874211 label

Reassigned to @smcgivern

How did this happen in the first place, where there any recent changes to this page?

Milestone changed to %8.14

According to http://performance.gitlab.net/dashboard/db/rails-controllers?var-action=DashboardController%23merge_requests&var-database=Production&from=now-6h&to=now, this has gone from 'terrible, times out all the time' to 'terrible, mostly doesn't time out':

Let's investigate @yorickpeterse's question this release.

Added Deliverable label

Milestone changed to %8.15

Attached is the profile of that dashboard for my user. You can see that there are a LOT of SQL queries involved.

dashboard-mr.zip

any? followed by each on a relation, and I'm the last person to touch that line https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/views/shared/_merge_requests.html.haml

We also seem to spend a lot of time in issuables_state_counter_text: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/helpers/issuables_helper.rb#L121

This caches the result, but maybe we'd be better off just doing one query to count all of the states at once. I think they are mutually exclusive, so we should be able to do it. Certainly the MilestonesHelper does it: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/helpers/milestones_helper.rb#L41

If I'm reading the profile right, doing those two things would save roughly (12% + (11% * 2)) = 34% of the time, which seems useful. (It wouldn't be exactly that, but we can try!)

Started branch 24669-merge-request-dashboard-page-takes-over-a-minute-to-load

Mentioned in merge request !7760 (merged)

Per http://performance.gitlab.net/dashboard/db/rails-controllers?var-action=DashboardController%23merge_requests&var-database=Production performance seems to have been improved, but it's a bit hard to tell for certain due to the low amount of requests. Having said that, overall the loading times are still slow enough to be annoying.

IssuableFinder#count_by_state seems to be the slowest part of this page.

Oh it will be - that was just a way of making three slow queries be one slow query. I didn't expect it to be fast, just clear out some of the obvious causes of slowness and reduce the chance of it timing out.

I wonder if we should consolidate this and https://gitlab.com/gitlab-org/gitlab-ce/issues/24201 into one issue, or at least create a meta issue about the dashboard. Wdyt?

@smcgivern That might not be a bad idea since problems like this are probably to be found across the various dashboards.

OK, so I'll close this (as the page does not take a minute to load any more, just 30s ) in favour of https://gitlab.com/gitlab-org/gitlab-ce/issues/25504.

closed