Use consistent load balancing in HAproxy
Thinking about sharding in gitlab/gitlab-ee#341 made me realize we could try out the effect of pinning repositories to app servers with some smart HAproxy rules.
We can let HAproxy balance based on the first two N components, and use a 'use_backend' rule + ACL to have separate balancing policies for projects (pin), API requests for projects (pin), and known non-project requests like /
(do not pin). Normal requests and API requests for a single project would land on two different servers, but that is a lot better than landing on 48 different servers. SSH and Sidekiq would remain spread out over all servers.
cc @pcarranza