Skip to content

Batch size >1000 does not pay off

username-removed-5302 requested to merge comment-batch-size into master

We did a small experiment to see how a full scan of the Redis keys on gitlab.com speeds up as we increase the batch size. The values on the right are time in seconds for a full scan (no delete operations).

count: 10);      284.500529021
count: 100);      86.21216934
count: 1_000);    60.931676195
count: 10_000);   60.96355610
count: 100_000);  62.378172541

It looks like 1,000 is a good number.

Merge request reports