Gitlab::Redis bigger connection pool on sidekiq workers
What does this MR do?
This MR configure Gitlab::Redis to use a pool with a size related with the number of threads running the current process.
Are there points in the code the reviewer needs to double check?
Are there any race condition setting the pool class instance variable?
Why was this MR needed?
Screenshots (if relevant)
Does this MR meet the acceptance criteria?
-
CHANGELOG entry added -
Documentation created/updated -
API support added - Tests
-
Added for this feature/bug -
All builds are passing
-
-
Conform by the merge request performance guides -
Conform by the style guides -
Branch has no merge conflicts with master
(if you do - rebase it please) -
Squashed related commits together
What are the relevant issue numbers?
Relates #18663 (moved)
Merge request reports
Activity
- Resolved by username-removed-423915
Reassigned to @pacoguzman
Added 1 commit:
- 0c414ab9 - Use higher size on Gitlab::Redis connection pool on Sidekiq servers
Added 3 commits:
-
8b5207fe...77de91e1 - 2 commits from branch
master
- 33c236a2 - Use higher size on Gitlab::Redis connection pool on Sidekiq servers
-
8b5207fe...77de91e1 - 2 commits from branch
Added 64 commits:
-
0c414ab9...8b5207fe - 64 commits from branch
master
-
0c414ab9...8b5207fe - 64 commits from branch
Reassigned to @jacobvosmaer-gitlab
24 24 end 25 25 26 26 def with 27 @pool ||= ConnectionPool.new { ::Redis.new(params) } 27 @pool ||= ConnectionPool.new(size: (Sidekiq.server? ? (Sidekiq.options[:concurrency] + 5) : 5)) { ::Redis.new(params) } OK it has to be compatible with running in the sidekiq process. Move this to a separate method,
pool_size
? And then annotate it.if Sidekiq.server? # the pool will be used in a multi-threaded context Sidekiq.options[:concurrency] + 5 else # probably this is a Unicorn process, so single threaded 5 end
Thanks @jacobvosmaer-gitlab , it didn't seem right to have
Sidekiq
in the middle of class, so extracting to a method makes sense.
Reassigned to @pacoguzman
Added 1 commit:
- ca53fcf9 - Use higher size on Gitlab::Redis connection pool on Sidekiq servers
Reassigned to @jacobvosmaer-gitlab
Reassigned to @pacoguzman
Reassigned to @yorickpeterse
88 88 end 89 89 end 90 90 91 describe '._with' do Opps thanks @yorickpeterse
Reassigned to @pacoguzman
Added 218 commits:
-
ca53fcf9...66a9511b - 217 commits from branch
master
- d99538fd - Use higher size on Gitlab::Redis connection pool on Sidekiq servers
-
ca53fcf9...66a9511b - 217 commits from branch
Added 1 commit:
- 1df68070 - Use higher size on Gitlab::Redis connection pool on Sidekiq servers
Added 12 commits:
-
1df68070...0bbeff3d - 11 commits from branch
master
- 640a4c88 - Use higher size on Gitlab::Redis connection pool on Sidekiq servers
-
1df68070...0bbeff3d - 11 commits from branch