Support cache policies in .gitlab-ci.yml

What does this MR do?

Respects a policy key in gitlab-ci.yml cache entries, which can skip cache upload or download steps.

Why was this MR needed?

This can reduce the load on cache servers significantly

Are there points in the code the reviewer needs to double check?

Does this MR meet the acceptance criteria?

• Documentation created/updated (in gitlab-ce)
• Tests
  • Added for this feature/bug
  • All builds are passing
• Branch has no merge conflicts with master (if you do - rebase it please)

What are the relevant issue numbers?

Closes #2539 (closed)

changed the description

@ayufan this needs tests and a CE component, but WDYT to trying to get this into %9.4 ? I'm happy to jump on the rails side too.

I know we're having problems with our cache server at the moment and I get the feeling this would help at least a bit!

It would help crazy :)

I like that simple changes :)

marked as a Work In Progress

marked the checklist item Added for this feature/bug as completed

marked the checklist item Documentation created/updated as completed

marked the checklist item All builds are passing as completed

changed the description

marked the checklist item Branch has no merge conflicts with master (if you do - rebase it please) as completed

@tmaczukin can you review? Tests added, and gitlab-rails side (incl. documentation) can be found here: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12483

assigned to @tmaczukin

unmarked as a Work In Progress

Note the implementation may need to change: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12483#note_33574803

@nick.thomas Looks nice! However I like the idea from https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12483#note_33574803 and I think it's worth to implement it :)

Any way - could you update https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/network/gitlab_test.go#L292 so the new field will be used in tests?

assigned to @nick.thomas

changed title from Support readonly cache entries in .gitlab-ci.yml to Support cache policies in .gitlab-ci.yml

changed the description

assigned to @tmaczukin

@tmaczukin I'm just moving onto the gitlab-ce MR now. How does this look?

Ach, complexity failures: https://gitlab.com/nick.thomas/gitlab-ci-multi-runner/-/jobs/20628298

assigned to @nick.thomas

That's better :)

assigned to @ayufan

@ayufan since you have the CE MR, it probably makes sense for you to review this one at the same time

assigned to @nick.thomas

resolved all discussions

Also ready for review. "policy": "pull-push", "policy": null and "policy": "" are all treated the same.

assigned to @ayufan

resolved all discussions

resolved all discussions

approved this merge request

merged