Use diff based updating for project_authorizations

Currently User#refresh_project_authorizations essentially takes these steps:

1. Start a transaction
2. Remove existing rows for a user
3. Insert all fresh rows for a user
4. Commit

This can lead to a high amount of dead tuples whenever users with access to many projects have their list updated.

To deal with this we should change this method so that it:

1. Only removes rows for which access was removed, or for which the access level was changed
2. Only inserts rows for which access was granted, or the access level was changed

This means that if user A is granted access to project B only 1 new row is added, instead of X rows being removed and Y new rows being inserted.

mentioned in issue gitlab-com/infrastructure#791 (closed)

The logic for this would be something like:

stale = current_authorizations.select(:id, :project_id, :access_level).each_with_object({}) do |row, hash|
  hash[row.project_id] = { id: row.id, level: row.access_level }
end

fresh = new_authorizations.select(:project_id, :access_level).each_with_object({}) do |row, hash|
  hash[row.project_id] = row.access_level
end

to_add = []
to_remove = []

stale.each do |project_id, hash|
  if !fresh[project_id] || fresh[project_id] != hash[:level]
    to_remove << hash[:id]
  end
end

fresh.each do |(project_id, access_level)|
  if !stale[project_id] || stale[project_id][:level] != access_level
    to_add << [user.id, project_id, access_level]
  end
end

transaction do
  project_authorizations.where(id: to_remove).delete_all

  unless to_add.empty?
    ProjectAuthorization.connection.execute <<-EOF
    INSERT INTO project_authorizations (user_id, project_id, access_level)
    VALUES #{to_add.map { |tuple| "(#{tuple.join(', ')})" }.join(', ')}
    EOF
  end
end

This is completely untested so it may not work.

I think I'm going to move this to 8.15. The logic for this is complex enough that I don't feel very comfortable introducing this in a patch release.

changed milestone to %8.15

mentioned in merge request !7956 (merged)

Merge request can be found here: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7956

mentioned in commit 04f5d77f

mentioned in commit 8b7844fa

mentioned in commit ed239dce

mentioned in commit 15c6ec72

mentioned in commit 9f5271e2

mentioned in commit 48fa5b9c

mentioned in commit d3621578

mentioned in commit b9a57a93

mentioned in commit f73193c3

closed via merge request !7956 (merged)

mentioned in commit 1139da27

mentioned in commit 896f09b9

I wonder if https://forum.gitlab.com/t/8-15-broke-all-my-machine-accounts/5632 is related to this?

mentioned in merge request !14785