Break down ES guide into two separate guides for small and big instances

What does this MR do?

Improves ES documentation

Are there points in the code the reviewer needs to double check?

Why was this MR needed?

Screenshots (if relevant)

Does this MR meet the acceptance criteria?

• Changelog entry added, if necessary
• Documentation created/updated
• API support added
• Tests
  • Added for this feature/bug
  • All builds are passing
• Conform by the merge request performance guides
• Conform by the style guides
• Branch has no merge conflicts with master (if it does - rebase it please)
• Squashed related commits together

What are the relevant issue numbers?

closes https://gitlab.com/gitlab-org/gitlab-ee/issues/1472

@nick.thomas Wdyt?

changed the description

assigned to @nick.thomas

mentioned in issue #1472 (closed)

assigned to @vsizov

Thanks @vsizov - the separation makes sense, just a few comments.

Is it worth talking about how the number of shards is hardcoded to 5 and including instruction on how to manually change it for large instances?

Do we also want to talk about sidekiq queues, throttling, dedicated sidekiq runners, etc, here? These are all things GitLab.com needs to do to make elasticsearch work, so we should share with the community as much as possible.

Perhaps it can be deferred to a second merge request once we've actually got GitLab.com working with elasticsearch

mentioned in issue gitlab-com/infrastructure#2157 (closed)

changed title from Rework ES documentation to Break down ES guide into two separate guides for small and big instances

@nick.thomas I propose to improve this doc iteratively. All your suggestions make sense to me. Do you want me to create a new issue with all suggestions?

Thanks @vsizov. So just the screen / tmux note in this MR, then the rest can go into a new issue

mentioned in issue #2903

@nick.thomas We have an issue regarding number of shard hardcoded https://gitlab.com/gitlab-org/gitlab-ee/issues/2087

resolved all discussions

assigned to @axil

@axil Please take a look & merge if you're OK with this.

LGTM now

mentioned in issue #2907

@victorcete Would you mind to review changes?

@vsizov I added a small comment about the number_of_replicas but apart from that LGTM too!

resolved all discussions

OK, this doc was hardly changed in parallel https://gitlab.com/gitlab-org/gitlab-ee/commit/334c91085bb340e5bf9c7e9e11f1c2c6ed8b6df3 so I need to re-apply all the changes basically...

resolved all discussions

added 475 commits

• 23d4001a...16b3dfe7 - 474 commits from branch master
• cca9563f - Break down ES guide into two separate guides for small and big instances

Compare with previous version

@vsizov sorry about that :/ I'll review asap.

merged

assigned to @vsizov

@axil Feel free to push fixes.