Use eTags for new MR widget polling endpoint?

MergeRequestsController#merge_check proved to be a resource consuming endpoint on production.

Basically, it's used for real-time updating the MR widget and it's polled every 10 seconds.

The main drawback for eTag caching on this endpoint is how broad the data changes can be made on the codebase:

See https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/serializers/merge_request_entity.rb

Would like to bring up that we've dropped ci_status endpoint and also moved this data to merge_check endpoint (i.e. https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/serializers/merge_request_entity.rb#L33), so there's only one endpoint responsible for the MR widget data flow (despite Approvals and Deployments).

That being said, I would like to discuss an approach for tackling this particular performance improvement.

A few options:

• Invalidate the eTag cache on every possible place
• Check the pain-points and focus on caching that particular data

cc @yorickpeterse @ayufan @adamniedzielski

EDIT: We're now using just /show.json endpoint - /merge_check got removed

It seems that we are heading more towards event-base model in a simple way, by probably introducing some notification/data invalidation router, based on what we store in serializers.

mentioned in issue gitlab-foss#32536 (closed)

changed the description

What are the most expensive parts of this for:

• Check the pain-points and focus on caching that particular data

I'm a little concerned about that, but I think @ayufan's suggestion makes sense, and this would be a good place to start if we want to go that route.

changed milestone to %9.3 in gitlab-foss

@victorwu can you make this next 2-3 months please?

changed milestone to %Next 2-3 months in gitlab-foss

@smcgivern done.

mentioned in issue gitaly#264 (closed)

moved from gitlab-foss#32545 (moved)