7.10.1 -> 7.10.2 migration error

Added ~12980 label

I just saw the latest tweet: https://twitter.com/gitlab/status/596305563602034688 Next time, I'll give it a day or so before upgrading.

And also, this is being discussed on GitHub: https://github.com/gitlabhq/gitlabhq/issues/9255

Is there a reason why the GitHub issue tracker is still being used?

This is a different issue than the one being discussed. It's not clear to me why your database doesn't have the index index_taggings_on_tag_id. This is what I see on v7.10.1:

$ sudo -u gitlab-psql psql -h /tmp -d gitlabhq_production

psql (9.3.5, server 9.2.9)
Type "help" for help.

gitlabhq_production=# \dt taggings;
         List of relations
 Schema |   Name   | Type  | Owner  
--------+----------+-------+--------
 public | taggings | table | gitlab
(1 row)

gitlabhq_production=# \d+ taggings;
                                                           Table "public.taggings"
    Column     |            Type             |                       Modifiers                       | Storage  | Stats target | Description 
---------------+-----------------------------+-------------------------------------------------------+----------+--------------+-------------
 id            | integer                     | not null default nextval('taggings_id_seq'::regclass) | plain    |              | 
 tag_id        | integer                     |                                                       | plain    |              | 
 taggable_id   | integer                     |                                                       | plain    |              | 
 taggable_type | character varying(255)      |                                                       | extended |              | 
 tagger_id     | integer                     |                                                       | plain    |              | 
 tagger_type   | character varying(255)      |                                                       | extended |              | 
 context       | character varying(255)      |                                                       | extended |              | 
 created_at    | timestamp without time zone |                                                       | plain    |              | 
Indexes:
    "taggings_pkey" PRIMARY KEY, btree (id)
    "index_taggings_on_tag_id" btree (tag_id)
    "index_taggings_on_taggable_id_and_taggable_type_and_context" btree (taggable_id, taggable_type, context)
Has OIDs: no

This is also consistent with what I see in schema.rb:

add_index "taggings", ["tag_id"], name: "index_taggings_on_tag_id", using: :btree

Are you using MySQL or Postgres?

I'm using Postgres. Here's what I see when I run the same commands:

$ sudo -u postgres psql gitlabhq_production
psql (9.3.6)
Type "help" for help.

gitlabhq_production=# \dt taggings;
         List of relations
 Schema |   Name   | Type  | Owner 
--------+----------+-------+-------
 public | taggings | table | git
(1 row)

gitlabhq_production=# \d+ taggings;
                                                         Table "public.taggings"
    Column     |           Type           |                       Modifiers                       | Storage  | Stats target | Description 
---------------+--------------------------+-------------------------------------------------------+----------+--------------+-------------
 id            | integer                  | not null default nextval('taggings_id_seq'::regclass) | plain    |              | 
 tag_id        | integer                  |                                                       | plain    |              | 
 taggable_id   | integer                  |                                                       | plain    |              | 
 taggable_type | character varying(510)   | default NULL::character varying                       | extended |              | 
 tagger_id     | integer                  |                                                       | plain    |              | 
 tagger_type   | character varying(510)   | default NULL::character varying                       | extended |              | 
 context       | character varying(510)   | default NULL::character varying                       | extended |              | 
 created_at    | timestamp with time zone |                                                       | plain    |              | 
Indexes:
    "taggings_pkey" PRIMARY KEY, btree (id)
Has OIDs: no

Should I just create those indices?

It looks like these indices were quietly added a year ago without a migration:

https://gitlab.com/gitlab-org/gitlab-ce/commit/75c9c68c18f287473e49b1f8e9282c1062d48897

If you had installed GitLab before then, I could see why they would be missing. I would also be concerned with other items that were not present.

cc: @dzaporozhets, @vsizov, @JobV

Ok, that's incredibly concerning. We've been running our GitLab instance for quite a long time. I believe I joined as an admin around GitLab 6 or so.

Also, a year or two ago, we migrated servers and at the same time, migrated from MySQL to Postgres (using the official migration guide at the time). Not sure if that has anything to do with it, but I can't imagine it does since this is the first issue that's cropped up.

I would run a comparison between your schema and the one GitLab has right now. If you're using the omnibus version, you can do something like:

sudo gitlab-rake db:schema:dump SCHEMA=/tmp/existing_schema.rb
sudo diff /opt/gitlab/embedded/service/gitlab-rails/db/schema.rb /tmp/existing_schema.rb

It sounds like we may want to add a migration/tweak the existing one to remove the index_taggings_on_tag_id index only if it exists since it's not guaranteed to be there if someone started with a very old version of GitLab.

@stanhu I agree. Thank you for your activity regarding this problem.

mentioned in merge request !626 (merged)

Status changed to closed by commit f6c0ef55

Also, a year or two ago, we migrated servers and at the same time, migrated from MySQL to Postgres (using the official migration guide at the time).

This may well have something do to with @gfairchild's problem. (cc @DouweM )

The converter originally dropped non-unique indices. We then added an extra step to the process that rebuilds the dropped indices, but only the non-unique ones. https://github.com/gitlabhq/mysql-postgresql-converter/blob/6f9cf5a8df6660591c91f82bc8d28e1ff61e5727/add_index_statements.rb#L3

The end result is that a DB that went from MySQL to Postgres has the same indices as one that was always Postgres, but some of the indices are named differently.

Edit: Issue for differing index names: https://github.com/gitlabhq/mysql-postgresql-converter/issues/4

FWIW gitlab.com went through the converter, so anything that goes wrong with "converted" databases can also go wrong on gitlab.com.

Here is what I see on gitlab.com:

gitlabhq_production=# gitlabhq_production=# \d+ taggings;
                                                         Table "public.taggings"
    Column     |           Type           |                       Modifiers                       | Storage  | Stats target | Description 
---------------+--------------------------+-------------------------------------------------------+----------+--------------+-------------
 id            | integer                  | not null default nextval('taggings_id_seq'::regclass) | plain    |              | 
 tag_id        | integer                  |                                                       | plain    |              | 
 taggable_id   | integer                  |                                                       | plain    |              | 
 taggable_type | character varying(510)   | default NULL::character varying                       | extended |              | 
 tagger_id     | integer                  |                                                       | plain    |              | 
 tagger_type   | character varying(510)   | default NULL::character varying                       | extended |              | 
 context       | character varying(510)   | default NULL::character varying                       | extended |              | 
 created_at    | timestamp with time zone |                                                       | plain    |              | 
Indexes:
    "taggings_pkey" PRIMARY KEY, btree (id)
    "index_taggings_on_tag_id" btree (tag_id)
    "index_taggings_on_taggable_id_and_taggable_type_and_context" btree (taggable_id, taggable_type, context)
Has OIDs: no

@gfairchild looks like your database is missing two indices. Consider rebuilding them.

I added the indices manually, and all seems to be well now. I was able to finish the upgrade. I also followed the instructions in your, so I should have all the necessary indices now. Thanks, @jacobvosmaer!

@gfairchild Great to hear! To prevent anyone else from running into this issue, particularly those who installed GitLab before v6.7.0, there is an update that was pushed yesterday to make the migration only drop the indices if they exist.

mentioned in commit tnir/gitlab-ce@99976128

Mentioned in commit maxraab/gitlab-ce@99976128

mentioned in merge request !14785

7.10.1 -> 7.10.2 migration error

Designs

Child items ...

Activity

Admin message

Admin message

7.10.1 -> 7.10.2 migration error

Activity