Trigger cache clear by rails db migrate

See https://gitlab.com/gitlab-org/gitlab-ce/issues/3643#note_2762188

cc @jacobvosmaer

I do not understand what this is about @marin .

@jacobvosmaer cache:clear gets triggered before db:migrate . I am suggesting to trigger cache:clear when db:migrate runs. Possibly patch release material.

Why is it a problem to run 'rake cache:clear' before 'rake db:migrate'?

@jacobvosmaer Kamil commented: I think that this is causing the problems: https://gitlab.com/gitlab-org/gitlab-ce/commit/2e4a673cbc1a43532e8aa096e4ab5ca034b804f7. We don't invalidate the cache after doing migrations.

@marin OK

Milestone changed to 8.4

After looking at the code more in depth I don't think this even valid(this == cache running before db:migrate).

Both migration and cache clear are tied to version file changing and these are the steps I see this executing:

1. In compilation stage, version file gets stored for executing during convergence stage, line
2. In compilation stage, we say that if the resource gets executed notify these 2 actions, namely db:migrate and cache:clear, line
3. migrate_database is actually a definition and a bash resource
4. In compilation stage, that means that nothing is going to be executed and the code block is being stored for executing during convergence stage
5. Now we are in convergence stage and reconfigure is running chef-zero
6. Remote file is being executed and source placed to targed file if there are changes(checksum is running first)
7. All resources that are being executed via notify are queued to the end of the convergence run
8. bash resource in the definition is now being executed and the last command in that block is run db:migrate
9. because this action notifies service restart with :immediately that means that all dependent services will run right after the migration. This means, unicorn and sidekiq restarts will be run.
10. Now is the time to run cache:clear as this was the next item in the notify queue.
11. As far as this problem is considered, the run is done.

Lol, after writing it out this way, I think I know what happens and where it goes wrong. Because we are triggering cache:clear when gitlab.yml changes at this line, the run gets queued before db:migrate. Since chef doesn't queue the same service twice, this will sometimes run cache:clear before db:migrate, and other times in the correct order!

@marin Good find! Way to debug by explanation. :)

Lets talk about possible solutions.

So we have cache:clear that gets triggered either when there is a change in gitlab.yml file, or during the upgrade(or both).

cache:clear in upgrade should take precedence I think. Problem is, we cannot just add :immediately to the trigger in version file change because it will trigger before db:migrate.

Only way to ensure the order is to add :immediately to both db:migrate and cache:clear.

I am wondering where would this break...

Thanks @stanhu :)

There is another way and that is to tie any db:migrate event with cache:clear. That might be a better solution. Basically what I've suggested at the beginning.

If I do this and set :immediately then the 2 notifications will be different for chef. Downside is that it will be run twice, first on gitlab.yml trigger and then on db:migrate. So it will be slow(ish).

Problem is, there is no better trigger that I can think of to cover both cases.

mentioned in merge request !598 (merged)

Additional attempt to not repeat cache clear was described here: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/598#note_3181939

Status changed to closed by commit b4dfb1f7