Skip to content

Profile capybara network requests with poltergeist

James EJ requested to merge 30923-profile-capybara-requests into master

What

Uses poltergeist's page.driver.network_traffic to profile browser network requests and store them to disk. This can then be used to display total/average time per endpoint.

Why

Help us determine which requests are slowing down our test suite. Inspired by noticing that sign in was taking seconds per test, this might help us identify other requests. If, for example, we noticed that GET 127.0.0.1/:namespace/:project/settings/ci_cd was taking a long time we might either opt to stub it with a quick response for tests, or to improve its performance generally.

Preview

Output in https://gitlab.com/gitlab-org/gitlab-ce/builds/14374233, with all requests stored in the build artifacts.

Total Time - Average Time -			 Endpoint
418.s - 271ms - GET  127.0.0.1/
408.s - 70ms - GET  127.0.0.1/assets/application.css
256.s - 45ms - GET  127.0.0.1/assets/print.css
235.s - 151ms - POST 127.0.0.1/users/sign_in
233.s - 144ms - GET  127.0.0.1/users/sign_in
177.s - 619ms - GET  127.0.0.1/namespace$/gitlabhq/merge_requests/$
174.s - 508ms - GET  127.0.0.1/namespace$/gitlabhq/merge_requests/$/pipeline_status.json
159.s - 427ms - GET  127.0.0.1/namespace$/gitlabhq/merge_requests/$.json
158.s - 621ms - GET  127.0.0.1/namespace$/gitlabhq/merge_requests/$/merge_check
143.s - 405ms - GET  127.0.0.1/namespace$/gitlabhq/merge_requests/$/ci_environments_status
129.s - 405ms - GET  127.0.0.1/namespace$/project$/boards/$/lists/$/issues
122.s - 343ms - GET  127.0.0.1/namespace$/gitlabhq/merge_requests/$/ci_status
90.3s - 1.11s - GET  127.0.0.1/namespace$/gitlabhq/merge_requests/$/diffs.json
79.9s - 515ms - GET  127.0.0.1/group$/gitlabhq/issues
77.6s - 392ms - GET  127.0.0.1/namespace$/gitlabhq/issues/$
70.3s - 364ms - GET  127.0.0.1/namespace$/gitlabhq/issues/$/related_branches
67.1s - 524ms - POST 127.0.0.1/namespace$/gitlabhq/notes
66.8s - 1.31s - GET  127.0.0.1/user$/gitlabhq/refs/master/logs_tree/
66.2s - 860ms - GET  127.0.0.1/namespace$/gitlabhq/merge_requests/$/diffs
65.7s - 388ms - GET  127.0.0.1/namespace$/gitlabhq/issues/$/can_create_branch
61.4s - 318ms - GET  127.0.0.1/namespace$/gitlabhq/issues/$/referenced_merge_requests
50.5s - 318ms - GET  127.0.0.1/namespace$/project$/issues
47.0s - 220ms - GET  127.0.0.1/namespace$/gitlabhq/issues/$.json
43.1s - 214ms - GET  127.0.0.1/namespace$/gitlabhq/issues/$/rendered_title
38.7s - 322ms - GET  127.0.0.1/namespace$/gitlabhq/merge_requests
31.9s - 625ms - GET  127.0.0.1/profile/two_factor_auth
28.4s - 261ms - GET  127.0.0.1/namespace$/project$/boards/$
25.4s - 1.41s - GET  127.0.0.1/namespace$/gitlabhq/refs/master/logs_tree/
22.1s - 1.58s - GET  127.0.0.1/user$/gitlabhq/wikis/home
21.9s - 318ms - GET  127.0.0.1/assets/lib/ace.js

Are there points in the code the reviewer needs to double check?

If we decide to use this long term:

Should we use the redash dashboard for this? How would that work?
Does this profiling slow tests down?

Acceptance criteria

Relevant issues

Closes #30923 (moved)

Merge request reports