DB migrate from scratch is broken in test env

Every time I see another exception in db migration

cc @stanhu if you have time to fix another one.

That said, why are we running migrations from scratch in the test environment? The whole point of db/schema.rb is that you can create the structure of the database without running migrations -- the test env doesn't care about the data in the database, so why do we need to migrate it if we can just get the structure?

@stanhu @DouweM @rspeicher migrations should always work from scratch. From first to last. Its quite unprofessional if your db:migrate on empty database does not work. I strongly recommend GitLab team to keep migrations working.

@dzaporozhets can we add this as a separate CI task?

mentioned in issue gitlab-development-kit#109 (closed)

This is closely related to https://gitlab.com/gitlab-org/gitlab-ce/issues/14549, where I suggested testing migration paths from one version to another. We can use the same structure to test a migration from scratch.

Note that in this case the failure was due to this commit/migration from 2 years ago: https://gitlab.com/gitlab-org/gitlab-ce/commit/d10b34a685509eebbe52391ffce861cde45cc0ca#diff-4. It uses models, which we have since stopped doing exactly for this reason.

mentioned in issue gitlab-development-kit#111 (closed)

sucks! - onboarding new devs is a pitty! any chances someone can fix this?

@sytses @DouweM I created MR that adds db:migrate:reset job to CI - https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3788. It should prevent situation like this. Once master is fixed I can rebase it and merge so we have it in master.

@DouweM btw maybe assign this issue to someone. I am afraid it will stay unnoticed otherwise.

mentioned in merge request !3788 (merged)

Reassigned to @stanhu

Are there any side effects of just backdating the migration that adds the pending_delete column? Doing something like this:

mv 20160122185421_add_pending_delete_to_project.rb 20140502125219_add_pending_delete_to_project.rb

Allows me to migrate from nothing to full without any issues.

Edit: The only change in that particular migration is that column.

mentioned in merge request !3789 (merged)

!3789 (merged) should solve this specific migration issue. Now I'm running into more fun with the pg_trgm extension:

== 20160226114608 AddTrigramIndexesForSearching: migrating ====================
-- execute("SELECT true AS enabled FROM pg_available_extensions WHERE name = 'pg_trgm' AND installed_version IS NOT NULL;")
   -> 0.0202s
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

You must enable the pg_trgm extension. You can do so by running "CREATE EXTENSION pg_trgm;" as a PostgreSQL super user, this must be done for every GitLab database. For more information see http://www.postgresql.org/docs/current/static/sql-createextension.html/Users/stanhu/gitlab-development-kit/gitlab-ce/db/migrate/20160226114608_add_trigram_indexes_for_searching.rb:8:in `up'
You must enable the pg_trgm extension. You can do so by running "CREATE EXTENSION pg_trgm;" as a PostgreSQL super user, this must be done for every GitLab database. For more information see http://www.postgresql.org/docs/current/static/sql-createextension.html
/Users/stanhu/gitlab-development-kit/gitlab-ce/db/migrate/20160226114608_add_trigram_indexes_for_searching.rb:8:in `up'
Tasks: TOP => db:migrate:reset => db:migrate
(See full trace by running task with --trace)

The annoying part is that when I attempt to run RAILS_ENV=test rails dbsconsole and CREATE EXTENSION pg_trgm;, it works, but the reset seems to clear the installation for the pg_trgm extension.

If the extension doesn't exist, I propose that we check if the user has privileges and attempt to do it there.

@stanhu we had a lot of troubles with the pg_trgm in our docker deployed gitlab-installations - so it would be nice to check if it is possible to create the extensions if it does not exists.

mentioned in commit f30053ba

mentioned in commit 007a683f

!3789 (merged) should solve this specific migration issue

@stanhu thanks

The annoying part is that when I attempt to run RAILS_ENV=test rails dbsconsole and CREATE EXTENSION pg_trgm;, it works, but the reset seems to clear the installation for the pg_trgm extension. If the extension doesn't exist, I propose that we check if the user has privileges and attempt to do it there.

@stanhu attempt would be great. At least it should work 100% for development/test env

mentioned in merge request !3805 (merged)

Status changed to closed by commit dcd54f63

mentioned in commit dcd54f63

DB migrate from scratch is broken in test env

Designs

Child items ...

Activity

Admin message

Admin message

DB migrate from scratch is broken in test env

Activity