git clone with depth=10 argument may fail during a release

As mentioned in https://gitlab.com/gitlab-org/gitlab-ce/issues/31934#note_29468389, the depth=10 argument used in the release task may erroneously show merge conflicts.

Should we remove this?

/cc: @rymai, @rspeicher, @selfup

I can't recall exactly but I believe this was for speed reasons, right @rymai? Doing a full clone of both CE and EE for the release is fairly time-consuming and bandwidth-intensive.

If that's the case, we'd probably want to increase the number a bit before we remove it entirely.

This happens quite often but I believe it's due to not syncing master often enough - due to mirror errors :(

It's also significantly much faster to clone with depth=10 than no depth at all...

Yeah it was definitely to speed up the task. We could start by increasing 10 to 100? The release duration will still be smaller then the pipelines duration anyway...

Ran into this again. @selfup was really confused when the release process attempted to sync remotes and then the git pull --depth=10 gitlab master caused a terminal to pop up. GitLab.com was about 30 commits ahead of dev.

@selfup suggested that we determine the required depth by looking at the HEAD for each of the remotes and then determining the number of commits that differ between the two.

Bumping it up to 100 would have helped this case.

Yea I just didn't expect to be able to remotely accept a merge 😂

Something like this can give us the correct depth:

## we would need to get the output and join the commands - a custom `run_git`
def run_git(commands)
  out, err, st = Open3.capture3("git #{commands.join(" ")}")

  raise ScriptError.new(err) unless st.success?

  out
end

latest_sha_from_dev = run_git(%W(
  log HEAD..dev/#{branch} --format=format:%H
)).split("\n")[0]

depth = run_git(%W(log HEAD..#{remote}/#{branch} --format=format:%H))
  .split("\n")
  .index(latest_sha_from_dev)

depth_limit = depth + 1

run_git %W(pull --quiet --depth=#{depth_limit} #{remote} #{branch})

Then we can use the depth_limit and not run into issues, as it will pull the exact amount 😄

We would have to check if certain remotes exist locally, create them if they do not exist, and fetch on every run so that we can do this with the most recent/relevant data

So for example we can just run something like:

[7] pry(main)> system("git remote add dev git@dev.gitlab.org:gitlab/omnibus-gitlab.git")
fatal: remote dev already exists.
=> false

Which helps a bunch since it takes care of a a few if statements for us 😂

## simple example
class CannotFetchError < StandardError; end

remotes = %w(dev)
remote_urls = %w(git@dev.gitlab.org:gitlab/omnibus-gitlab.git)

remotes.each_with_index do |remote, idx|
  remote_url = remote_urls[idx]

  ## if it doesn't exist, it will add it anyways - then we fetch
  ## if it does exist, nothing happens - then we fetch
  system("git remote add #{remote} #{remote_url}")
  
  if !system("git fetch #{remote}")
    raise CannotFetchError, "Could not fetch from #{remote} - #{remote_url}"
  end
end

if we can fix the project mirroring issues, we wouldn't need any of this :)

mentioned in merge request !179 (merged)