Gitlab-CE reconfigure time out and gitlab-rake had no response

Hi, Any guys can help me? I hava been bored by this problem for two weeks.

I had installed Gitlab-CE 9.0.4 in Ubuntu 14.04_x86_64 by using apt-get install gitlab-ce command. And it had been ok for me for about 2 months. But suddently a day, when I pushed a commit to my gitlab-ce, I found that gitlat-ce returned 502 error. So I use lslf -i:8082 command to see whether unicorn service is OK. Because I used 8082 port for unicorn in /etc/gitlab/gitlab.rb. For this command's result, I found that unicorn was not bind its port. And it didn't work for me to change this port in gitlab.rb and reconfigure.

So I use sudo gitlab-rake gitlab:check SANITIZE=true command to see if there were any wrong things. But it had no response for a long time, even that I had been waiting for it for a whole night.

So I use apt-get update and apt-get upgrade to update Gitlab-CE. but failed. I found the update failed in follow steps:

[execute] CREATE EXTENSION
- execute /opt/gitlab/bin/gitlab-psql -d gitlabhq_production -c "CREATE EXTENSION IF NOT EXISTS pg_trgm;"

Recipe: gitlab::database_migrations

• bash[migrate gitlab-rails database] action run

  ================================================================================ Error executing action run on resource 'bash[migrate gitlab-rails database]'

  Mixlib::ShellOut::CommandTimeout

  Command timed out after 3600s: Command exceeded allowed execution time, process terminated ---- Begin output of "bash" "/tmp/chef-script20170814-104859-fk1812" ---- STDOUT: STDERR: ---- End output of "bash" "/tmp/chef-script20170814-104859-fk1812" ---- Ran "bash" "/tmp/chef-script20170814-104859-fk1812" returned

  Resource Declaration:

  In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb

  51: bash "migrate gitlab-rails database" do

And then I removed gitlab-ce followed gitlab official steps from gitlab.com. And downloaded gitlab-ce by using apt-get download gitlab-ce, and used dpkg -i gitlab-ce_9.4.3xxxxx.deb to install. It was OK but when I use gitlab-ctl reconfigure it failed for the same thing: database migrate timeout.

So I looked at gitlab-rake code, I found it use "exec /opt/gitlab/embedded/bin/chpst -e /opt/gitlab/etc/gitlab-rails/env -u git -U git /opt/gitlab/embedded/bin/bundle exec rake "$@". So I use "exec /opt/gitlab/embedded/bin/chpst -e /opt/gitlab/etc/gitlab-rails/env -u git -U git /opt/gitlab/embedded/bin/bundle exec rake "gitlab:env:info" to test. But I found it has no response for a long long long long long time.

I guess that the timeout may had some relations with db migrate timeout.

This is my os infomation:

Linux PEK1000114103 4.4.0-59-generic #80 (closed)~14.04.1-Ubuntu SMP Fri Jan 6 18:02:02 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

root@PEK1000114103:/etc/gitlab# cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 45 model name : Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz stepping : 7 microcode : 0x710 cpu MHz : 2900.042 cache size : 20480 KB physical id : 0 siblings : 16 core id : 0 cpu cores : 16 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm xsaveopt bugs : bogomips : 5800.08 clflush size : 64 cache_alignment : 64 address sizes : 46 bits physical, 48 bits virtual power management:

processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 45 model name : Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz stepping : 7 microcode : 0x710 cpu MHz : 2900.042 cache size : 20480 KB

root@PEK1000114103:/etc/gitlab# cat /proc/meminfo MemTotal: 32921716 kB MemFree: 28081740 kB MemAvailable: 31385988 kB Buffers: 640272 kB Cached: 2939556 kB SwapCached: 0 kB Active: 2265948 kB Inactive: 1978444 kB Active(anon): 742188 kB Inactive(anon): 187336 kB Active(file): 1523760 kB Inactive(file): 1791108 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 3905532 kB SwapFree: 3905532 kB Dirty: 120 kB Writeback: 0 kB AnonPages: 664908 kB Mapped: 366752 kB Shmem: 264968 kB Slab: 450744 kB SReclaimable: 390132 kB SUnreclaim: 60612 kB KernelStack: 7568 kB PageTables: 20240 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 20366388 kB Committed_AS: 10105540 kB VmallocTotal: 34359738367 kB VmallocUsed: 0 kB VmallocChunk: 0 kB HardwareCorrupted: 0 kB AnonHugePages: 497664 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 67580 kB DirectMap2M: 33609728 kB