Skip to content
Snippets Groups Projects

Optimize GitHub importer for speed and memory

What does this MR do?

It applies some improvements to GitHub importer. Importing of a private GH repo (~1500 issues, ~180 PRs, ~3600 commits), was taking over 30 minutes and memory usage reached 950 MB, after these changes it took 8 minutes and memory was at 700 MB max (for both cases, Sidekiq starts at ~400 MB).

Are there points in the code the reviewer needs to double check?

N/A

Why was this MR needed?

People are asking for it, see issues below.

Screenshots (if relevant)

N/A

Does this MR meet the acceptance criteria?

What are the relevant issue numbers?

https://gitlab.com/gitlab-com/support-forum/issues/996 and https://gitlab.com/gitlab-org/gitlab-ce/issues/19124#note_13588843

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply
Loading