Skip to content
Snippets Groups Projects
New issue look: Off

Failed Merge Request with 500

    • Closed Issue created by username-removed-43452 @ryan.lovelett

    Summary

    Get HTTP 500 error when submitting a new merge request.

    Relevant logs and/or screenshots

    Started POST "/project/webview/merge_requests" for 127.0.0.1 at 2014-07-21 21:17:15 -0500
Processing by Projects::MergeRequestsController#create as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"[token]", "merge_request"=>{"title"=>"Bug 4053", "description"=>"", "assignee_id"=>"", "milestone_id"=>"", "source_project_id"=>"28", "target_project_id"=>"28", "target_branch"=>"master", "source_branch"=>"bug-4053"}, "project_id"=>"project"}
Mysql2::Error: Incorrect string value: '\xE2\x80\x90dia...' for column 'st_diffs' at row 1: [diff redacted]
Completed 500 Internal Server Error in 266ms
  app/models/merge_request_diff.rb:146:in `reload_diffs'
  app/models/merge_request_diff.rb:44:in `reload_content'
  app/services/merge_requests/create_service.rb:9:in `execute'
  app/controllers/projects/merge_requests_controller.rb:108:in `create'

    Output of checks

    $ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
[sudo] password for lovelettr:
Instance method "lock!" is already defined in ActiveRecord::Base, use generic helper instead or set StateMachine::Machine.ignore_method_conflicts = true.
Checking Environment ...

Git configured for git user? ... yes

Checking Environment ... Finished

Checking GitLab Shell ...

GitLab Shell version >= 1.9.6 ? ... OK (1.9.6)
Repo base directory exists? ... yes
Repo base directory is a symlink? ... no
Repo base owned by git:git? ... yes
Repo base access is drwxrws---? ... yes
Satellites access is drwxr-x---? ... yes
update hook up-to-date? ... yes
update hooks in repos are links: ...
****** / libinterp ... ok
****** / ****** ... ok
****** / 3D ... ok
****** / Base ... ok
****** / ****** ... ok
****** / *******-archived ... ok
****** / *** ... ok
****** / ***-****** ... ok
****** / ***uite-FSR ... ok
****** / ****** ... ok
****** / ****-Assist ... ok
****** / *** ... ok
****** / ****** ... ok
****** / ***** ... ok
****** / Recon ... ok
****** / **** Registration ... ok
****** / *****13 ... ok
****** / ****** ... ok
****** / **** Test ... ok
****** / ****** ... ok
****** / **** Decision Framework ... ok
****** / **** ASPE ... ok
****** / Authoring ... ok
****** / Interactive Graphing ... ok
****** / Content Template ... ok
****** / WebView ... ok
****** / requirejs-example ... ok
****** / ******* ... ok
****** / Titanium ... ok
****** / gitlab-webhook ... ok
****** / webhook-test ... ok
****** / ImageMapster ... ok
****** / *** Trajectory Plugin ... ok
****** / ****VIII ... repository is empty
****** / *** Ada CPP ... ok
****** / ******* ... ok
****** / One ... ok
Running /home/git/gitlab-shell/bin/check
Check GitLab API access: OK
Check directories and files:
	/home/git/repositories: OK
	/home/git/.ssh/authorized_keys: OK
Test redis-cli executable: redis-cli 2.8.13
Send ping to redis server: PONG
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Sidekiq ...

Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking LDAP ...

LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab ...

Database config exists? ... yes
Database is SQLite ... no
All migrations up? ... yes
Database contains orphaned UsersGroups? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Init script exists? ... yes
Init script up-to-date? ... yes
projects have namespace: ...
****** / libinterp ... yes
****** / ****** ... yes
****** / 3D ... yes
****** / Base ... yes
****** / ****** ... yes
****** / *******-archived ... yes
****** / *** ... yes
****** / ***-****** ... yes
****** / ***uite-FSR ... yes
****** / ****** ... yes
****** / ****-Assist ... yes
****** / *** ... yes
****** / ****** ... yes
****** / ***** ... yes
****** / Recon ... yes
****** / **** Registration ... yes
****** / *****13 ... yes
****** / ****** ... yes
****** / **** Test ... yes
****** / ****** ... yes
****** / **** Decision Framework ... yes
****** / **** ASPE ... yes
****** / Authoring ... yes
****** / Interactive Graphing ... yes
****** / Content Template ... yes
****** / WebView ... yes
****** / requirejs-example ... yes
****** / ******* ... yes
****** / Titanium ... yes
****** / gitlab-webhook ... yes
****** / webhook-test ... yes
****** / ImageMapster ... yes
****** / *** Trajectory Plugin ... yes
****** / ****VIII ... yes
****** / *** Ada CPP ... yes
****** / ******* ... yes
****** / One ... yes
Projects have satellites? ...
****** / libinterp ... yes
****** / ****** ... yes
****** / 3D ... yes
****** / Base ... yes
****** / ****** ... yes
****** / *******-archived ... yes
****** / *** ... yes
****** / ***-****** ... yes
****** / ***uite-FSR ... yes
****** / ****** ... yes
****** / ****-Assist ... yes
****** / *** ... yes
****** / ****** ... yes
****** / ***** ... yes
****** / Recon ... yes
****** / **** Registration ... yes
****** / *****13 ... yes
****** / ****** ... yes
****** / **** Test ... yes
****** / ****** ... yes
****** / **** Decision Framework ... yes
****** / **** ASPE ... yes
****** / Authoring ... yes
****** / Interactive Graphing ... yes
****** / Content Template ... yes
****** / WebView ... yes
****** / requirejs-example ... yes
****** / ******* ... yes
****** / Titanium ... yes
****** / gitlab-webhook ... yes
****** / webhook-test ... yes
****** / ImageMapster ... yes
****** / *** Trajectory Plugin ... yes
****** / ****VIII ... yes
****** / *** Ada CPP ... yes
****** / ******* ... yes
****** / One ... yes
Redis version >= 2.0.0? ... yes
Your git bin path is "/usr/local/bin/git"
Git version >= 1.7.10 ? ... yes (1.8.4)

Checking GitLab ... Finished

    GitLab 7.0.0 de9bf7da

    $ sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production
[sudo] password for lovelettr:
Instance method "lock!" is already defined in ActiveRecord::Base, use generic helper instead or set StateMachine::Machine.ignore_method_conflicts = true.

System information
System:
Current User:	git
Using RVM:	no
Ruby Version:	2.1.2p95
Gem Version:	2.2.2
Bundler Version:1.6.3
Rake Version:	10.3.2
Sidekiq Version:2.17.0

GitLab information
Version:	7.0.0
Revision:	de9bf7d
Directory:	/home/git/gitlab
DB Adapter:	mysql2
URL:		http://domain.com
HTTP Clone URL:	http://domain.com/some-project.git
SSH Clone URL:	git@domain.com:some-project.git
Using LDAP:	no
Using Omniauth:	no

GitLab Shell
Version:	1.9.6
Repositories:	/home/git/repositories/
Hooks:		/home/git/gitlab-shell/hooks/
Git:		/usr/local/bin/git

    Designs

    Child items 0

    • No child items are currently open.

    Activity

    • username-removed-30461
      username-removed-30461 @ayr-ton ·

      @ryan.lovelett Can you provide your githost.log?

    • username-removed-43452
      username-removed-43452 @ryan.lovelett ·
      Author

      Sure.

      # Logfile created on 2013-10-10 13:35:00 -0500 by logger.rb/36483
April 11, 2014 10:00 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/titanium/.git merge --no-ff -m Merge branch 'bug-3741' into 'master' source/bug-3741
April 11, 2014 15:48 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -m Merge branch 'bug-3737' into 'master' source/bug-3737
May 01, 2014 09:39 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3848' into 'master' source/bug-3848
May 08, 2014 13:36 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3872' into 'master' source/bug-3872
May 08, 2014 22:13 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3876' into 'master' source/bug-3876
May 09, 2014 07:01 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3830' into 'master' source/bug-3830
May 09, 2014 11:06 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3873' into 'master' source/bug-3873
May 09, 2014 12:09 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3876' into 'master' source/bug-3876
May 12, 2014 14:31 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3876' into 'master' source/bug-3876
May 12, 2014 14:41 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3876' into 'master' source/bug-3876
May 13, 2014 09:40 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3885' into 'master' source/bug-3885
May 13, 2014 09:41 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3885' into 'master' source/bug-3885
May 13, 2014 12:55 -> ERROR -> Grit::Git::GitTimeout
May 14, 2014 16:36 -> ERROR -> Grit::Git::GitTimeout
May 19, 2014 15:43 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3920' into 'master' source/bug-3920
May 22, 2014 14:49 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/titanium/.git merge --no-ff -mMerge branch 'bug-3911' into 'master' source/bug-3911
May 22, 2014 15:05 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/titanium/.git merge --no-ff -mMerge branch 'bug-3911' into 'master' source/bug-3911
May 22, 2014 16:25 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3926' into 'master' source/bug-3926
May 28, 2014 11:59 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3950' into 'master' source/bug-3950
May 28, 2014 14:51 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3896' into 'master' source/bug-3896
May 28, 2014 15:00 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3896' into 'master' source/bug-3896
May 28, 2014 16:11 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3950' into 'master' source/bug-3950
May 28, 2014 16:12 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3927' into 'master' source/bug-3927
May 29, 2014 13:14 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3927' into 'master' source/bug-3927
May 29, 2014 14:51 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3967' into 'master' source/bug-3967
June 03, 2014 10:55 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/*******-webview/.git merge --no-ff -mMerge branch 'bug-3979-less-css' into 'master' source/bug-3979-less-css
June 24, 2014 15:05 -> ERROR -> Grit::Git::GitTimeout
June 24, 2014 15:06 -> ERROR -> Grit::Git::GitTimeout
June 24, 2014 15:06 -> ERROR -> Grit::Git::GitTimeout
June 24, 2014 15:07 -> ERROR -> Grit::Git::GitTimeout
June 24, 2014 15:07 -> ERROR -> Grit::Git::GitTimeout
June 24, 2014 15:08 -> ERROR -> Grit::Git::GitTimeout
June 24, 2014 15:08 -> ERROR -> Grit::Git::GitTimeout
June 24, 2014 15:09 -> ERROR -> Grit::Git::GitTimeout
June 24, 2014 15:10 -> ERROR -> Grit::Git::GitTimeout
June 24, 2014 15:11 -> ERROR -> Grit::Git::GitTimeout
June 24, 2014 15:11 -> ERROR -> Grit::Git::GitTimeout
June 24, 2014 15:12 -> ERROR -> Grit::Git::GitTimeout
July 10, 2014 09:57 -> ERROR -> Command failed [1]: /usr/local/bin/git --git-dir=/home/git/gitlab-satellites/******/webview/.git merge --no-ff -mMerge branch 'bug-3897' into 'psuedo-master' source/bug-3897
July 17, 2014 10:21 -> ERROR -> Grit::Git::GitTimeout
July 17, 2014 10:22 -> ERROR -> Grit::Git::GitTimeout
July 17, 2014 10:23 -> ERROR -> Grit::Git::GitTimeout
July 17, 2014 10:24 -> ERROR -> Grit::Git::GitTimeout
July 17, 2014 13:24 -> ERROR -> Grit::Git::GitTimeout
July 17, 2014 13:25 -> ERROR -> Grit::Git::GitTimeout
July 17, 2014 13:41 -> ERROR -> Grit::Git::GitTimeout
July 17, 2014 13:41 -> ERROR -> Grit::Git::GitTimeout
July 17, 2014 13:42 -> ERROR -> Grit::Git::GitTimeout
July 17, 2014 13:43 -> ERROR -> Grit::Git::GitTimeout
July 18, 2014 12:17 -> ERROR -> Grit::Git::GitTimeout
July 18, 2014 12:17 -> ERROR -> Grit::Git::GitTimeout
July 24, 2014 13:56 -> ERROR -> Grit::Git::GitTimeout
July 24, 2014 13:57 -> ERROR -> Grit::Git::GitTimeout
July 24, 2014 13:58 -> ERROR -> Grit::Git::GitTimeout
July 24, 2014 13:58 -> ERROR -> Grit::Git::GitTimeout
July 24, 2014 14:10 -> ERROR -> Grit::Git::GitTimeout
July 24, 2014 14:11 -> ERROR -> Grit::Git::GitTimeout
    • username-removed-30461
      username-removed-30461 @ayr-ton ·

      @ryan.lovelett It feels like you are creating merge requests that run for more than 30 seconds. You will need to update your gitlab to the last version. It has the following fix:

      https://github.com/gitlabhq/gitlabhq/commit/07537faca72e2475a25e1e73c12bc924499afc27

      With this feature you will be able to add a new line in satellites section on gitlab.yml with a timeout high than 30 seconds:

      satellites:
   # Relative paths are relative to Rails.root (default: tmp/repo_satellites/)
   path: /home/git/gitlab-satellites/
   timeout: 90 # 90 seconds, for example

      In my case, I use 600 as timeout because I have really big merge requests. Is good to remember that this section has a different value than git timeout and it won't fetch git timeout settings to satellites. The default value will be 30 seconds.

      You may want to change you unicorn.rb timeout to something similar.

    • username-removed-43452
      username-removed-43452 @ryan.lovelett ·
      Author

      @ayr-ton no doubt we are having problem with merge requests that run for more than 30 seconds. Thanks for pointing out the link and the configuration tweak. I will absolutely do both.

      However, when I replicate the error and am using tail -f gitlab/log/githost.log no new ERROR -> Grit::Git::GitTimeout gets logged. Unfortunately I think those are a red-herring to the actual issue at hand.

    • username-removed-43452
      username-removed-43452 @ryan.lovelett ·
      Author

      @ayr-ton is that in Gitlab 7.1 or do I have to use Gitlab master to see those changes?

    • username-removed-30461
      username-removed-30461 @ayr-ton ·

      @ryan.lovelett Yes. It is on 7.1.

    • username-removed-43452
      username-removed-43452 @ryan.lovelett ·
      Author

      Just to follow up I have gone ahead and updated to 7.1 updated my timeouts to around 960 seconds. This issue is still occurring as before.

      Also, there is nothing new in the githost.log file.

    • username-removed-43452 Status changed to closed

      Status changed to closed

    • username-removed-43452
      username-removed-43452 @ryan.lovelett ·
      Author

      I have figured out the cause and a solution for this. Basically, while the database.yml was configured to handle utf8 the actual MySQL server was configured to use latin1.

      The fix was rather simple and is based from this StackOverflow question: How to convert an entire MySQL database characterset and collation to UTF-8?.

      For our configuration (GitLab 7.1) I ran the following MySQL queries:

      ALTER DATABASE `gitlab_production` CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `broadcast_messages` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `deploy_keys_projects` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `emails` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `events` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `forked_project_links` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `issues` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `keys` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `merge_request_diffs` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `merge_requests` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `milestones` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `namespaces` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `notes` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `projects` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `protected_branches` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `schema_migrations` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `services` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `snippets` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `taggings` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `tags` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `users` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `users_groups` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `users_projects` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE `web_hooks` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;

      I'm not sure any of this needs to be codified into any kind of bug fix to apply to the GitLab project itself. Perhaps there should be some sort of check built into the gitlab:check Rake task to verify the right encoding? Regardless I am going to close this issue. If it should be pursued as a change request. Someone let me know and I can do that too.

    • username-removed-90962 mentioned in issue #1432 (closed)

      mentioned in issue #1432 (closed)

    • username-removed-1606700 mentioned in issue #37761 (closed)

      mentioned in issue #37761 (closed)

    Please register or sign in to reply