GitLab FOSS merge requestshttps://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests2019-08-13T17:30:45Zhttps://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14781IssuesReferenceParser use collection cache to get associations2019-08-13T17:30:45ZBob Van Landuyt :neckbeard:IssuesReferenceParser use collection cache to get associations## What does this MR do?
## Are there points in the code the reviewer needs to double check?
## Why was this MR needed?
## Screenshots (if relevant)
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs....## What does this MR do?
## Are there points in the code the reviewer needs to double check?
## Why was this MR needed?
## Screenshots (if relevant)
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs.gitlab.com/ee/development/changelog.html) added, if necessary
- [ ] [Documentation created/updated](https://docs.gitlab.com/ee/development/doc_styleguide.html)
- [ ] API support added
- [ ] Tests added for this feature/bug
- Review
- [ ] Has been reviewed by UX
- [ ] Has been reviewed by Frontend
- [ ] Has been reviewed by Backend
- [ ] Has been reviewed by Database
- [ ] Conform by the [merge request performance guides](https://docs.gitlab.com/ee/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
Closes #141928.13https://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14779Resolve "Deleting a project results in a large number of SQL requests that co...2019-07-04T10:18:31ZBob Van Landuyt :neckbeard:Resolve "Deleting a project results in a large number of SQL requests that could be optimized"## What does this MR do?
## Are there points in the code the reviewer needs to double check?
## Why was this MR needed?
## Screenshots (if relevant)
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs....## What does this MR do?
## Are there points in the code the reviewer needs to double check?
## Why was this MR needed?
## Screenshots (if relevant)
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs.gitlab.com/ee/development/changelog.html) added, if necessary
- [ ] [Documentation created/updated](https://docs.gitlab.com/ee/development/doc_styleguide.html)
- [ ] API support added
- [ ] Tests added for this feature/bug
- Review
- [ ] Has been reviewed by UX
- [ ] Has been reviewed by Frontend
- [ ] Has been reviewed by Backend
- [ ] Has been reviewed by Database
- [ ] Conform by the [merge request performance guides](https://docs.gitlab.com/ee/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
Closes #1163https://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14738Remove AjaxLoadingSpinner and CreateLabelDropdown from global namespace2017-10-24T23:58:00ZFilipa Lacerdafilipa@gitlab.comRemove AjaxLoadingSpinner and CreateLabelDropdown from global namespace## What does this MR do?
Remove AjaxLoadingSpinner and CreateLabelDropdown from global namespace
## Does this MR meet the acceptance criteria?
- Review
- [x] Has been reviewed by Frontend
- [x] Conform by the [merge request performan...## What does this MR do?
Remove AjaxLoadingSpinner and CreateLabelDropdown from global namespace
## Does this MR meet the acceptance criteria?
- Review
- [x] Has been reviewed by Frontend
- [x] Conform by the [merge request performance guides](https://docs.gitlab.com/ee/development/merge_request_performance_guidelines.html)
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
#3886010.2username-removed-636429username-removed-636429https://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14735Removes CommitsList from global namespace2017-10-24T23:58:06ZFilipa Lacerdafilipa@gitlab.comRemoves CommitsList from global namespace## What does this MR do?
Removes CommitsList from global namespace
Fixes some eslint problems
## Does this MR meet the acceptance criteria?
- Review
- [ ] Has been reviewed by Frontend
- [x] Conform by the [merge request performance g...## What does this MR do?
Removes CommitsList from global namespace
Fixes some eslint problems
## Does this MR meet the acceptance criteria?
- Review
- [ ] Has been reviewed by Frontend
- [x] Conform by the [merge request performance guides](https://docs.gitlab.com/ee/development/merge_request_performance_guidelines.html)
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
#3886910.2Phil Hughesme@iamphill.comPhil Hughesme@iamphill.comhttps://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14731WIP: Rewrite the GitHub importer to perform work in parallel and greatly impr...2017-11-13T05:36:31Zyorickpeterse-stagingWIP: Rewrite the GitHub importer to perform work in parallel and greatly improve performance## What does this MR do?
This MR rewrites the GitHub importer from scratch so it's _much_ faster and performs work in parallel. This MR is still a WIP, I'll update the body properly once we get closer to a final state.
## TODO
*...## What does this MR do?
This MR rewrites the GitHub importer from scratch so it's _much_ faster and performs work in parallel. This MR is still a WIP, I'll update the body properly once we get closer to a final state.
## TODO
* [ ] Use a separate class for importing issue comments
* [ ] Import issue comments using a worker (1 job per comment)
* [ ] Issue and comment workers should reschedule themselves in the future if we hit a rate limit, the reschedule time will simply be the reset time of the rate limit (so if our rate limit resets in 10 seconds that means we schedule jobs for 10 seconds in the future)
* [ ] Make the parallel importing schedule a job to check for progress, instead of blocking the thread in a `sleep` call
* [ ] Make sure any stuck importer jobs don't mess with a running GitHub import
* [ ] Import releases
* [ ] Test all the things
* [ ] Add tests for `User.by_any_email`
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs.gitlab.com/ee/development/changelog.html) added, if necessary
- [ ] [Documentation created/updated](https://docs.gitlab.com/ee/development/doc_styleguide.html)
- [ ] API support added
- [ ] Tests added for this feature/bug
- Review
- [ ] Has been reviewed by Backend
- [ ] Has been reviewed by Database
- [ ] Conform by the [merge request performance guides](https://docs.gitlab.com/ee/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)10.1yorickpeterse-stagingyorickpeterse-staginghttps://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14729Fetch the merged branches at once2017-11-03T05:09:44Zusername-removed-423915Fetch the merged branches at once## What does this MR do?
Fetch the merged branches at once, instead of checking it one by one in the view. We don't cache this yet because this would already much improve the performance.
## Are there points in the code the reviewe...## What does this MR do?
Fetch the merged branches at once, instead of checking it one by one in the view. We don't cache this yet because this would already much improve the performance.
## Are there points in the code the reviewer needs to double check?
Previously we could be checking with ~Gitaly, but with this change we would be using `git` command line. Could this kill the filesystem?
## Why was this MR needed?
A naive test against a particularly large repository:
``` ruby
begin
now = Time.now
branches.map{ |b| repository.merged_to_root_ref?(b.name) }
Time.now - now
end # 8.265830782
```
Around 10 times faster:
``` ruby
begin
now = Time.now
repository.merged_branches(branches.map(&:name))
Time.now - now
end # 0.807405397
```
This should make the branches page usable.
## Screenshots (if relevant)
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs.gitlab.com/ee/development/changelog.html) added, if necessary
- [ ] Tests added for this feature/bug
- Review
- [ ] Has been reviewed by Backend
## What are the relevant issue numbers?
This is a follow up from https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14678#note_4254801910.1username-removed-128633username-removed-128633https://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14690Move cycle analytics banner into a vue file2017-10-24T23:58:00ZFilipa Lacerdafilipa@gitlab.comMove cycle analytics banner into a vue file## What does this MR do?
- Moves last cycle analytics bannerinto a vue files
## Does this MR meet the acceptance criteria?
- Review
- [ ] Has been reviewed by Frontend
- [x] Conform by the [merge request performance guides](...## What does this MR do?
- Moves last cycle analytics bannerinto a vue files
## Does this MR meet the acceptance criteria?
- Review
- [ ] Has been reviewed by Frontend
- [x] Conform by the [merge request performance guides](https://docs.gitlab.com/ee/development/merge_request_performance_guidelines.html)
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
#3437110.2Phil Hughesme@iamphill.comPhil Hughesme@iamphill.comhttps://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14567WIP: Convert Icons in CI to SVG Sprite Icons2017-11-03T05:07:13ZTim ZallmannWIP: Convert Icons in CI to SVG Sprite Icons## What does this MR do?
It simply converts some of the icons to svg sprite icons.
## Are there points in the code the reviewer needs to double check?
Does everything look the same?
## Why was this MR needed?
To make ever...## What does this MR do?
It simply converts some of the icons to svg sprite icons.
## Are there points in the code the reviewer needs to double check?
Does everything look the same?
## Why was this MR needed?
To make everything faster and better with SVG Sprites
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs.gitlab.com/ee/development/changelog.html) added, if necessary
- [ ] [Documentation created/updated](https://docs.gitlab.com/ee/development/doc_styleguide.html)
- [ ] API support added
- [ ] Tests added for this feature/bug
- Review
- [ ] Has been reviewed by UX
- [ ] Has been reviewed by Frontend
- [ ] Has been reviewed by Backend
- [ ] Has been reviewed by Database
- [ ] Conform by the [merge request performance guides](https://docs.gitlab.com/ee/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
#3623110.1Tim ZallmannTim Zallmannhttps://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14480Update the default memory limits2017-10-02T17:49:38ZBen KochieUpdate the default memory limits## What does this MR do?
Update memory limit controls in `config.ru`.
## Are there points in the code the reviewer needs to double check?
## Why was this MR needed?
Memory limit default in omnibus is now 450MB, source install...## What does this MR do?
Update memory limit controls in `config.ru`.
## Are there points in the code the reviewer needs to double check?
## Why was this MR needed?
Memory limit default in omnibus is now 450MB, source installs will now have trouble with crash loops.
## Screenshots (if relevant)
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs.gitlab.com/ee/development/changelog.html) added, if necessary
- [ ] [Documentation created/updated](https://docs.gitlab.com/ee/development/doc_styleguide.html)
- [ ] API support added
- [ ] Tests added for this feature/bug
- Review
- [ ] Has been reviewed by UX
- [ ] Has been reviewed by Frontend
- [ ] Has been reviewed by Backend
- [ ] Has been reviewed by Database
- [ ] Conform by the [merge request performance guides](https://docs.gitlab.com/ee/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
https://gitlab.com/gitlab-org/gitlab-ce/issues/374379.5https://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14405WIP: Find unused CSS (selectors by usage)2017-09-26T20:58:15Zusername-removed-892863contact@ericeastwood.comWIP: Find unused CSS (selectors by usage)# tldr;
See our CSS selectors sorted by usage, https://gitlab.com/snippets/1676232
# How can I mess around with this?
Download the HTML pages from all of the rspec tests and save to `$gdk/gitlab/fixtures/pages`, [:paperclip: `gi...# tldr;
See our CSS selectors sorted by usage, https://gitlab.com/snippets/1676232
# How can I mess around with this?
Download the HTML pages from all of the rspec tests and save to `$gdk/gitlab/fixtures/pages`, [:paperclip: `gitlab-rspec-pages.zip`](https://drive.google.com/a/gitlab.com/file/d/0B4-vXgqxRSkhRE4tTWNjd05WUGs/view?usp=sharing)
If you want to generate the HTML yourself, simply run the rspec tests, `bundle exec rspec spec/features`. There is a rspec `after` hook in place to save the HTML. When I was running the tests, I had to run in smaller chunks to better follow when things started to fail because of random issues like the webpack crashing. You will also need to strip out the `<script>` tags because `uncss` will load the page into `jsdom` and fail to load the scripts and bail out. Find `<script(.*?)<\/script>` and replace with nothing.
Download the cached selector files for each page and save to `$gdk/gitlab/fixtures/pages/cache` (these are generated in the script but will save a lot of time), [`gitlab-rspec-pages-cache.zip`](/uploads/949647a336c43f39a0ed11f11d978751/gitlab-rspec-pages-cache.zip)
Run the script:
```
node fixtures/find-unused-css.js
```
See the report, `$gdk/gitlab/fixtures/pages/cache/#selector-map-by-usage.json`
---
**The code is very rough and is only meant as a quick PoC to get one-off stats around our CSS.** I wouldn't choose `uncss` again because it is very slow and doesn't give proper insight/stats.
# Journey
I saved off the HTML from every rspec test in `spec/features` using a `after` hook.
Because [`uncss`](https://www.npmjs.com/package/uncss) tries to load the `<script>` tags on the page, I stripped them via a simple find/replace script (Sublime find/replace was crashing and leaving a bunch of unsaved tabs open so I had to delete the sublime session file manually :expressionless:). Find `<script(.*?)<\/script>` and replace with nothing.
Then I ran into `uncss` choking on memory because of the sheer number of HTML files(it batches up a bunch of `jsdom` instances :sick:)
```
<--- Last few GCs --->
239117 ms: Mark-sweep 1323.8 (1434.4) -> 1322.8 (1434.4) MB, 992.2 / 0 ms [allocation failure] [GC in old space requested].
240073 ms: Mark-sweep 1322.8 (1434.4) -> 1322.5 (1434.4) MB, 955.9 / 0 ms [allocation failure] [GC in old space requested].
241047 ms: Mark-sweep 1322.5 (1434.4) -> 1322.5 (1434.4) MB, 973.6 / 0 ms [last resort gc].
242000 ms: Mark-sweep 1322.5 (1434.4) -> 1322.5 (1434.4) MB, 952.7 / 0 ms [last resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0000003B3D8C9E51 <JS Object>
1: filter(aka filter) [uncss\node_modules\jsdom\lib\jsdom\living\node.js:95] [pc=0000036AE42573CA] (this=0000003B3D804189 <undefined>,node=000003E2FFBDB901 <an EventTargetImpl with map 000002294BDB22C1>)
2: treeToArray [uncss\node_modules\symbol-tree\lib\SymbolTree.js:~338] [pc=0000036AE35A29D1] (this=000002F4DD4ABFD9 <a Symb...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
```
So I [forked `uncss`](https://github.com/MadLittleMods/uncss/pull/1) and made some rough PoC changes. It now creates the `jsdom` instances only when needed inside the loop. I simplified the CSS fetching process so it only looks at the stylesheet provided in the options. Added some result caching so we don't have to find the selectors on a page every single run-through (helpful when testing things). I also bodged in some nicer reporting so we can get the selector usage.
---
See https://gitlab.com/gitlab-org/gitlab-ce/issues/38132
cc @timzallmann
https://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14375lossless image optimization2017-10-05T13:51:15Zusername-removed-1118608lossless image optimizationthis is a replacement for https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13857this is a replacement for https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/1385710.1Jacob SchatzJacob Schatzhttps://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14268WIP: Improve participants performance2017-09-26T14:56:55Zusername-removed-7736WIP: Improve participants performance## What does this MR do?
Improve participants performance
## Are there points in the code the reviewer needs to double check?
## Why was this MR needed?
## Screenshots (if relevant)
## Does this MR meet the acceptance crit...## What does this MR do?
Improve participants performance
## Are there points in the code the reviewer needs to double check?
## Why was this MR needed?
## Screenshots (if relevant)
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs.gitlab.com/ce/development/changelog.html) added, if necessary
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- [ ] Tests added for this feature/bug
- Review
- [ ] Has been reviewed by UX
- [ ] Has been reviewed by Frontend
- [ ] Has been reviewed by Backend
- [ ] Has been reviewed by Database
- [ ] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
gitlab-org/gitlab-ce#22375username-removed-7736username-removed-7736https://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14022Simplify SQL query used for getting issues/MRs by labels2017-09-26T10:23:16Zusername-removed-7736Simplify SQL query used for getting issues/MRs by labels## What does this MR do?
Simplify SQL query used for getting issues/MRs by labels
## Are there points in the code the reviewer needs to double check?
- SQL performance
Query generated by `Issue.with_label(['bug, critical']).t...## What does this MR do?
Simplify SQL query used for getting issues/MRs by labels
## Are there points in the code the reviewer needs to double check?
- SQL performance
Query generated by `Issue.with_label(['bug, critical']).to_sql` is following
<details>
<summary>Test data</summary>
<pre>
SELECT COUNT(*) FROM issues;
count
-------
12337
SELECT COUNT(*) FROM label_links;
count
-------
3060
SELECT COUNT(*) FROM labels;
count
-------
1010
</pre>
</details>
<details>
<summary>Query</summary>
<pre>
SELECT "issues".*
FROM "issues"
WHERE "issues"."deleted_at" IS NULL
AND (EXISTS
( SELECT TRUE
FROM label_links
INNER JOIN labels ON labels.id = label_links.label_id
WHERE labels.title = 'bug'
AND target_type = 'Issue'
AND target_id = issues.id ))
AND (EXISTS
( SELECT TRUE
FROM label_links
INNER JOIN labels ON labels.id = label_links.label_id
WHERE labels.title = 'critical'
AND target_type = 'Issue'
AND target_id = issues.id ))
ORDER BY "issues"."id" DESC
</pre>
</details>
<details>
<summary>EXPLAIN ANALYSE</summary>
<pre>
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Sort (cost=37.44..37.45 rows=1 width=347) (actual time=0.596..0.596 rows=1 loops=1)
Sort Key: issues.id DESC
Sort Method: quicksort Memory: 25kB
-> Nested Loop Semi Join (cost=24.84..37.43 rows=1 width=347) (actual time=0.445..0.565 rows=1 loops=1)
-> Nested Loop (cost=24.29..32.57 rows=3 width=351) (actual time=0.107..0.217 rows=25 loops=1)
-> HashAggregate (cost=24.00..24.03 rows=3 width=4) (actual time=0.100..0.105 rows=25 loops=1)
Group Key: label_links_1.target_id
-> Nested Loop (cost=4.59..24.00 rows=3 width=4) (actual time=0.055..0.091 rows=25 loops=1)
-> Index Scan using index_labels_on_title on labels labels_1 (cost=0.28..8.29 rows=1 width=4) (actual time=0.009..0.010 rows=1 loops=1)
Index Cond: ((title)::text = 'critical'::text)
-> Bitmap Heap Scan on label_links label_links_1 (cost=4.31..15.66 rows=4 width=8) (actual time=0.043..0.073 rows=25 loops=1)
Recheck Cond: (label_id = labels_1.id)
Filter: ((target_type)::text = 'Issue'::text)
Heap Blocks: exact=15
-> Bitmap Index Scan on index_label_links_on_label_id (cost=0.00..4.31 rows=4 width=0) (actual time=0.031..0.031 rows=25 loops=1)
Index Cond: (label_id = labels_1.id)
-> Index Scan using issues_pkey on issues (cost=0.29..2.83 rows=1 width=347) (actual time=0.003..0.003 rows=1 loops=25)
Index Cond: (id = label_links_1.target_id)
Filter: (deleted_at IS NULL)
-> Nested Loop (cost=0.56..1.35 rows=1 width=4) (actual time=0.013..0.013 rows=0 loops=25)
-> Index Scan using index_label_links_on_target_id_and_target_type on label_links (cost=0.28..0.35 rows=3 width=8) (actual time=0.004..0.005 rows=3 loops=25)
Index Cond: ((target_id = issues.id) AND ((target_type)::text = 'Issue'::text))
-> Index Scan using labels_pkey on labels (cost=0.28..0.32 rows=1 width=4) (actual time=0.002..0.002 rows=0 loops=79)
Index Cond: (id = label_links.label_id)
Filter: ((title)::text = 'bug'::text)
Rows Removed by Filter: 1
Planning time: 1.316 ms
Execution time: 0.825 ms
</pre>
</details>
## Why was this MR needed?
SQL query used for getting issues/MRs by labels is overly complex
## Screenshots (if relevant)
None
## Does this MR meet the acceptance criteria?
- [x] [Changelog entry](https://docs.gitlab.com/ce/development/changelog.html) added, if necessary
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- [ ] Tests added for this feature/bug
- Review
- [ ] Has been reviewed by UX
- [ ] Has been reviewed by Frontend
- [ ] Has been reviewed by Backend
- [ ] Has been reviewed by Database
- [ ] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
Closes #3713710.1username-removed-7736username-removed-7736https://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/13847Move expensive test jobs to new high-cpu runners2017-09-12T11:37:30ZTomasz Maczukintomasz@gitlab.comMove expensive test jobs to new high-cpu runners## What does this MR do?
Moves some compute expensive tests to new `high-cpu` runners as discussed in gitlab-com/infrastructure#2046.
## Are there points in the code the reviewer needs to double check?
To new runners I've moved:...## What does this MR do?
Moves some compute expensive tests to new `high-cpu` runners as discussed in gitlab-com/infrastructure#2046.
## Are there points in the code the reviewer needs to double check?
To new runners I've moved:
- all `rspec` jobs,
- all `spinach` jobs,
- `gitlab:assets:compile` job,
- `setup-test-env` job.
Is there anything else that is compute expensive and we'd like it to be executed on `high-cpu`?
## Why was this MR needed?
Please read gitlab-com/infrastructure#2046 for a reference.
## Screenshots (if relevant)
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs.gitlab.com/ce/development/changelog.html) added, if necessary
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- [ ] Tests added for this feature/bug
- Review
- [ ] Has been reviewed by UX
- [ ] Has been reviewed by Frontend
- [ ] Has been reviewed by Backend
- [ ] Has been reviewed by Database
- [ ] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
Closes gitlab-com/infrastructure#2046Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/13788WIP: Improve GitHub import branches performance2017-09-28T15:56:33ZJames LopezWIP: Improve GitHub import branches performanceThis will allow us to:
* Creating MR diffs was timing out recently, so we couldn't even resume the import. Having this should help with performance.
* Fix current MRs that have no actual branch associated and are throwing 500s
* Fix...This will allow us to:
* Creating MR diffs was timing out recently, so we couldn't even resume the import. Having this should help with performance.
* Fix current MRs that have no actual branch associated and are throwing 500s
* Fix MRs that have been updated upstream (so the diff has changed) but not locally
* Performance should be improved as we won't be fetching the branch per MR
After this is merged we can then remove all (non-required) branches.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/36288
- [ ] [Changelog entry](https://docs.gitlab.com/ce/development/changelog.html) added, if necessary
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- [ ] Tests added for this feature/bug
- Review
- [ ] Has been reviewed by UX
- [ ] Has been reviewed by Frontend
- [ ] Has been reviewed by Backend
- [ ] Has been reviewed by Database
- [ ] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?James LopezJames Lopezhttps://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/12520enable transform runtime for auto-polyfills and smaller bundles2017-07-25T07:27:12Zusername-removed-548703enable transform runtime for auto-polyfills and smaller bundles## What does this MR do?
enable transform runtime for auto-polyfills and smaller bundles
## Are there points in the code the reviewer needs to double check?
## Why was this MR needed?
This also supports Math.log10
## Screenshots (if...## What does this MR do?
enable transform runtime for auto-polyfills and smaller bundles
## Are there points in the code the reviewer needs to double check?
## Why was this MR needed?
This also supports Math.log10
## Screenshots (if relevant)
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs.gitlab.com/ce/development/changelog.html) added, if necessary
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?https://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/11575Notify workhorse about etag changes2017-07-25T17:32:01ZKamil TrzcińśkiNotify workhorse about etag changes## What does this MR do?
Makes E-tag caching to notify about resource changes.
## Are there points in the code the reviewer needs to double check?
## Why was this MR needed?
## Screenshots (if relevant)
## Does this MR meet the accep...## What does this MR do?
Makes E-tag caching to notify about resource changes.
## Are there points in the code the reviewer needs to double check?
## Why was this MR needed?
## Screenshots (if relevant)
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs.gitlab.com/ce/development/changelog.html) added, if necessary
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?https://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10869WIP: Simplify ProjectsFinder query2017-04-28T13:41:44Zusername-removed-840794WIP: Simplify ProjectsFinder query## What does this MR do?
- Remove unnecessary conditions for admin user from `ProjectsFinder` query.
- Replace `UNION` with `OR` conditions and `WHERE IN` with `WHERE EXISTS` when in user authorization query.
- Remove duplicate cond...## What does this MR do?
- Remove unnecessary conditions for admin user from `ProjectsFinder` query.
- Replace `UNION` with `OR` conditions and `WHERE IN` with `WHERE EXISTS` when in user authorization query.
- Remove duplicate condition from calls to `Project#self.with_feature_available_for_user`.
This will affect other finders queries (ref #22145).
Updated query for getting events in user profile (logged as a normal user with id: `669`):
```sql
SELECT "events"."id"
-- ...
FROM "events"
LEFT OUTER JOIN "projects" ON "projects"."id" = "events"."project_id"
AND "projects"."pending_delete" = 'f'
LEFT OUTER JOIN "routes" ON "routes"."source_id" = "projects"."id"
AND "routes"."source_type" = 'Project'
LEFT OUTER JOIN "namespaces" ON "namespaces"."id" = "projects"."namespace_id"
AND "namespaces"."deleted_at" IS NULL
LEFT OUTER JOIN "users" ON "users"."id" = "events"."author_id"
WHERE ("events"."author_id" IS NOT NULL)
AND "events"."author_id" = 669
AND "projects"."pending_delete" = 'f'
AND ("projects"."visibility_level" IN (20,
10)
OR EXISTS
(SELECT 1
FROM "project_authorizations"
WHERE "projects"."id" = "project_authorizations"."project_id"
AND "project_authorizations"."user_id" = 669))
ORDER BY "events"."id" DESC LIMIT 20
OFFSET 0
```
Original -- normal user:
```sql
SELECT "events"."id"
-- ...
FROM "events"
LEFT OUTER JOIN "projects" ON "projects"."id" = "events"."project_id"
AND "projects"."pending_delete" = 'f'
LEFT OUTER JOIN "routes" ON "routes"."source_id" = "projects"."id"
AND "routes"."source_type" = 'Project'
LEFT OUTER JOIN "namespaces" ON "namespaces"."id" = "projects"."namespace_id"
AND "namespaces"."deleted_at" IS NULL
LEFT OUTER JOIN "users" ON "users"."id" = "events"."author_id"
WHERE ("events"."author_id" IS NOT NULL)
AND "events"."author_id" = 669
AND "projects"."pending_delete" = 'f'
AND (projects.id IN
(SELECT "projects"."id"
FROM "projects"
INNER JOIN "project_authorizations" ON "projects"."id" = "project_authorizations"."project_id"
WHERE "projects"."pending_delete" = 'f'
AND "project_authorizations"."user_id" = 669
UNION SELECT "projects"."id"
FROM "projects"
WHERE "projects"."visibility_level" IN (20,
10)))
ORDER BY "events"."id" DESC LIMIT 20
OFFSET 0
```
Updated -- admin
```sql
SELECT "events"."id"
-- ...
FROM "events"
LEFT OUTER JOIN "projects" ON "projects"."id" = "events"."project_id"
AND "projects"."pending_delete" = 'f'
LEFT OUTER JOIN "routes" ON "routes"."source_id" = "projects"."id"
AND "routes"."source_type" = 'Project'
LEFT OUTER JOIN "namespaces" ON "namespaces"."id" = "projects"."namespace_id"
AND "namespaces"."deleted_at" IS NULL
LEFT OUTER JOIN "users" ON "users"."id" = "events"."author_id"
WHERE ("events"."author_id" IS NOT NULL)
AND "events"."author_id" = 669
AND "projects"."pending_delete" = 'f'
ORDER BY "events"."id" DESC LIMIT 20
OFFSET 0
```
Original -- admin
```sql
SELECT "events"."id"
-- ...
FROM "events"
LEFT OUTER JOIN "projects" ON "projects"."id" = "events"."project_id"
AND "projects"."pending_delete" = 'f'
LEFT OUTER JOIN "routes" ON "routes"."source_id" = "projects"."id"
AND "routes"."source_type" = 'Project'
LEFT OUTER JOIN "namespaces" ON "namespaces"."id" = "projects"."namespace_id"
AND "namespaces"."deleted_at" IS NULL
LEFT OUTER JOIN "users" ON "users"."id" = "events"."author_id"
WHERE ("events"."author_id" IS NOT NULL)
AND "events"."author_id" = 660
AND "projects"."pending_delete" = 'f'
AND (projects.id IN
(SELECT "projects"."id"
FROM "projects"
INNER JOIN "project_authorizations" ON "projects"."id" = "project_authorizations"."project_id"
WHERE "projects"."pending_delete" = 'f'
AND "project_authorizations"."user_id" = 1
UNION SELECT "projects"."id"
FROM "projects"))
ORDER BY "events"."id" DESC LIMIT 20
OFFSET 0
```
Some affected queries by removing duplicate condition for union query in `Project#self.with_feature_available_for_user` (with/without `unscope`ing):
http://localhost:3000/search?utf8=%E2%9C%93&snippets=&scope=&search=seed
- `Issue`: https://www.diffchecker.com/uy42QCGD
- `MergeRequest`: https://www.diffchecker.com/1QVRiHIZ
## Are there points in the code the reviewer needs to double check?
There are some issues I had to workaround:
- missing binding values in some queries: http://gitlab.com/0xbsec/gitlab-ce/blob/31255-simplify-project-finder-query/app/models/project.rb#L293-L295 (to reproduce run `spec/finders/issues_finder_spec.rb` spec without `unscoping`). Idealy this should be fixed from the root cause (the query should stay unscoped).
- calling `user.refresh_authorized_projects: https://gitlab.com/0xbsec/gitlab-ce/blob/31255-simplify-project-finder-query/app/models/project.rb#L280 (reproduce in spec: `spec/finders/projects_finder_spec.rb:96`)
## Why was this MR needed?
## Screenshots (if relevant)
## Does this MR meet the acceptance criteria?
- [x] [Changelog entry](https://docs.gitlab.com/ce/development/changelog.html) added, if necessary
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
Closes #31255username-removed-840794username-removed-840794https://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10403Eager load to avoid N+1 queries when there are labels from multiple projects ...2019-09-20T02:46:33ZStan HuEager load to avoid N+1 queries when there are labels from multiple projects in a milestoneFound by Bullet in !10263 in https://gitlab.com/gitlab-org/gitlab-ce/builds/13490762
This one is subtle: even though we have a `QueryRecorder` checking for N+1 queries, what we don't test is labels in multiple projects of a milestone.Found by Bullet in !10263 in https://gitlab.com/gitlab-org/gitlab-ce/builds/13490762
This one is subtle: even though we have a `QueryRecorder` checking for N+1 queries, what we don't test is labels in multiple projects of a milestone.9.1Stan HuStan Huhttps://staging.gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10072WIP: remove underscore - use lodash and babel plugin2017-09-05T14:34:28Zusername-removed-642931git@selfup.meWIP: remove underscore - use lodash and babel plugin## What does this MR do?
It removes the non tree-shakeable underscore and uses lodash instead (which is also faster :smile:)
## Are there points in the code the reviewer needs to double check?
All of the things
## Why was thi...## What does this MR do?
It removes the non tree-shakeable underscore and uses lodash instead (which is also faster :smile:)
## Are there points in the code the reviewer needs to double check?
All of the things
## Why was this MR needed?
Because sanity and performance
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs.gitlab.com/ce/development/changelog.html) added, if necessary
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
#231109.1