Skip to content

PostDeployMigrateUserExternalMailData fails with PG::UniqueViolation

Summary

After upgrading to Gitlab 10.0.0 from 9.5.5 the migration fails

Steps to reproduce

  • Running Gitlab 9.5.5
  • Upgrade to 10.0.0 and migrate

What is the current bug behavior?

The migrations fails.

What is the expected correct behavior?

It should migrate

Relevant logs and/or screenshots

== 20170828170502 PostDeployMigrateUserExternalMailData: migrating ============
-- execute("        INSERT INTO user_synced_attributes_metadata (user_id, provider, email_synced)\n        SELECT id, email_provider, external_email\n        FROM users\n        WHERE external_email = TRUE\n        AND NOT EXISTS (\n          SELECT true\n          FROM user_synced_attributes_metadata\n          WHERE user_id = users.id\n          AND provider = users.email_provider\n        )\n        AND id BETWEEN 1 AND 86\n")
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint "index_user_synced_attributes_metadata_on_user_id"
DETAIL:  Key (user_id)=(68) already exists.
:         INSERT INTO user_synced_attributes_metadata (user_id, provider, email_synced)
        SELECT id, email_provider, external_email
        FROM users
        WHERE external_email = TRUE
        AND NOT EXISTS (
          SELECT true
          FROM user_synced_attributes_metadata
          WHERE user_id = users.id
          AND provider = users.email_provider
        )
        AND id BETWEEN 1 AND 86
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/peek-pg-1.3.0/lib/peek/views/pg.rb:17:in `async_exec'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/peek-pg-1.3.0/lib/peek/views/pg.rb:17:in `async_exec'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql/database_statements.rb:155:in `block in execute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql/database_statements.rb:154:in `execute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:665:in `block in method_missing'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:634:in `block in say_with_time'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:634:in `say_with_time'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:654:in `method_missing'
/home/git/gitlab/db/post_migrate/20170828170502_post_deploy_migrate_user_external_mail_data.rb:27:in `block in up'
/home/git/gitlab/app/models/concerns/each_batch.rb:75:in `block in each_batch'
/home/git/gitlab/app/models/concerns/each_batch.rb:56:in `step'
/home/git/gitlab/app/models/concerns/each_batch.rb:56:in `each_batch'
/home/git/gitlab/db/post_migrate/20170828170502_post_deploy_migrate_user_external_mail_data.rb:24:in `up'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:611:in `exec_migration'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:591:in `block in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:590:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:768:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:1046:in `ddl_transaction'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:959:in `block in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:955:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:955:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:823:in `up'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:801:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/tasks/database_tasks.rb:137:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'
ActiveRecord::RecordNotUnique: PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint "index_user_synced_attributes_metadata_on_user_id"
DETAIL:  Key (user_id)=(68) already exists.
:         INSERT INTO user_synced_attributes_metadata (user_id, provider, email_synced)
        SELECT id, email_provider, external_email
        FROM users
        WHERE external_email = TRUE
        AND NOT EXISTS (
          SELECT true
          FROM user_synced_attributes_metadata
          WHERE user_id = users.id
          AND provider = users.email_provider
        )
        AND id BETWEEN 1 AND 86
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/peek-pg-1.3.0/lib/peek/views/pg.rb:17:in `async_exec'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/peek-pg-1.3.0/lib/peek/views/pg.rb:17:in `async_exec'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql/database_statements.rb:155:in `block in execute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql/database_statements.rb:154:in `execute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:665:in `block in method_missing'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:634:in `block in say_with_time'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:634:in `say_with_time'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:654:in `method_missing'
/home/git/gitlab/db/post_migrate/20170828170502_post_deploy_migrate_user_external_mail_data.rb:27:in `block in up'
/home/git/gitlab/app/models/concerns/each_batch.rb:75:in `block in each_batch'
/home/git/gitlab/app/models/concerns/each_batch.rb:56:in `step'
/home/git/gitlab/app/models/concerns/each_batch.rb:56:in `each_batch'
/home/git/gitlab/db/post_migrate/20170828170502_post_deploy_migrate_user_external_mail_data.rb:24:in `up'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:611:in `exec_migration'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:591:in `block in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:590:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:768:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:1046:in `ddl_transaction'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:959:in `block in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:955:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:955:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:823:in `up'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:801:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/tasks/database_tasks.rb:137:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'
PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint "index_user_synced_attributes_metadata_on_user_id"
DETAIL:  Key (user_id)=(68) already exists.
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/peek-pg-1.3.0/lib/peek/views/pg.rb:17:in `async_exec'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/peek-pg-1.3.0/lib/peek/views/pg.rb:17:in `async_exec'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql/database_statements.rb:155:in `block in execute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql/database_statements.rb:154:in `execute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:665:in `block in method_missing'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:634:in `block in say_with_time'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:634:in `say_with_time'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:654:in `method_missing'
/home/git/gitlab/db/post_migrate/20170828170502_post_deploy_migrate_user_external_mail_data.rb:27:in `block in up'
/home/git/gitlab/app/models/concerns/each_batch.rb:75:in `block in each_batch'
/home/git/gitlab/app/models/concerns/each_batch.rb:56:in `step'
/home/git/gitlab/app/models/concerns/each_batch.rb:56:in `each_batch'
/home/git/gitlab/db/post_migrate/20170828170502_post_deploy_migrate_user_external_mail_data.rb:24:in `up'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:611:in `exec_migration'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:591:in `block in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:590:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:768:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:1046:in `ddl_transaction'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:959:in `block in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:955:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:955:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:823:in `up'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/migration.rb:801:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/tasks/database_tasks.rb:137:in `migrate'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.8/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'

Output of checks

Results of GitLab environment info

System information
System:		Ubuntu 16.04
Current User:	git
Using RVM:	no
Ruby Version:	2.3.1p112
Gem Version:	2.5.1
Bundler Version:1.15.1
Rake Version:	12.0.0
Redis Version:	3.0.6
Git Version:	2.7.4
Sidekiq Version:5.0.4
Go Version:	go1.8.3 linux/amd64

GitLab information
Version:	10.0.0
Revision:	bbf5c73
Directory:	/home/git/gitlab
DB Adapter:	postgresql
URL:		
HTTP Clone URL:	
SSH Clone URL:	
Using LDAP:	yes
Using Omniauth:	no

GitLab Shell
Version:	5.9.0
Repository storage paths:
- default: 	/home/git/repositories
Hooks:		/home/git/gitlab-shell/hooks
Git:		/usr/bin/git