Gitaly with ruby is not compiling in master

I am looking into it right now.

The failure:

 [GitFetcher: gitaly] I | 2017-08-15T03:00:40+00:00 | Cleaning existing clone
        [Builder: gitaly] I | 2017-08-15T03:00:40+00:00 | Starting build
        [Builder: gitaly] I | 2017-08-15T03:00:40+00:00 | Environment:
        [Builder: gitaly] I | 2017-08-15T03:00:40+00:00 |   CFLAGS="-I/opt/gitlab/embedded/include -O2"
        [Builder: gitaly] I | 2017-08-15T03:00:40+00:00 |   CPPFLAGS="-I/opt/gitlab/embedded/include -O2"
        [Builder: gitaly] I | 2017-08-15T03:00:40+00:00 |   CXXFLAGS="-I/opt/gitlab/embedded/include -O2"
        [Builder: gitaly] I | 2017-08-15T03:00:40+00:00 |   LDFLAGS="-Wl,-rpath,/opt/gitlab/embedded/lib -L/opt/gitlab/embedded/lib"
        [Builder: gitaly] I | 2017-08-15T03:00:40+00:00 |   LD_RUN_PATH="/opt/gitlab/embedded/lib"
        [Builder: gitaly] I | 2017-08-15T03:00:40+00:00 |   MAKE="gmake"
        [Builder: gitaly] I | 2017-08-15T03:00:40+00:00 |   OMNIBUS_INSTALL_DIR="/opt/gitlab"
        [Builder: gitaly] I | 2017-08-15T03:00:40+00:00 |   PATH="/opt/gitlab/bin:/opt/gitlab/embedded/bin:/builds/gitlab/omnibus-gitlab/gems/ruby/2.3.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
        [Builder: gitaly] I | 2017-08-15T03:00:40+00:00 |   PKG_CONFIG_PATH="/opt/gitlab/embedded/lib/pkgconfig"
        [Builder: gitaly] I | 2017-08-15T03:00:40+00:00 | $ gmake install PREFIX=/opt/gitlab/embedded
        [Builder: gitaly] W | 2017-08-15T03:00:51+00:00 | [1/2] Failed to execute command. Retrying in 10 seconds...
        [Builder: gitaly] I | 2017-08-15T03:01:01+00:00 | Environment:
        [Builder: gitaly] I | 2017-08-15T03:01:01+00:00 |   CFLAGS="-I/opt/gitlab/embedded/include -O2"
        [Builder: gitaly] I | 2017-08-15T03:01:01+00:00 |   CPPFLAGS="-I/opt/gitlab/embedded/include -O2"
        [Builder: gitaly] I | 2017-08-15T03:01:01+00:00 |   CXXFLAGS="-I/opt/gitlab/embedded/include -O2"
        [Builder: gitaly] I | 2017-08-15T03:01:01+00:00 |   LDFLAGS="-Wl,-rpath,/opt/gitlab/embedded/lib -L/opt/gitlab/embedded/lib"
        [Builder: gitaly] I | 2017-08-15T03:01:01+00:00 |   LD_RUN_PATH="/opt/gitlab/embedded/lib"
        [Builder: gitaly] I | 2017-08-15T03:01:01+00:00 |   MAKE="gmake"
        [Builder: gitaly] I | 2017-08-15T03:01:01+00:00 |   OMNIBUS_INSTALL_DIR="/opt/gitlab"
        [Builder: gitaly] I | 2017-08-15T03:01:01+00:00 |   PATH="/opt/gitlab/bin:/opt/gitlab/embedded/bin:/builds/gitlab/omnibus-gitlab/gems/ruby/2.3.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
        [Builder: gitaly] I | 2017-08-15T03:01:01+00:00 |   PKG_CONFIG_PATH="/opt/gitlab/embedded/lib/pkgconfig"
        [Builder: gitaly] I | 2017-08-15T03:01:01+00:00 | $ gmake install PREFIX=/opt/gitlab/embedded
        [Builder: gitaly] W | 2017-08-15T03:01:04+00:00 | [2/2] Failed to execute command. Retrying in 20 seconds...
        [Builder: gitaly] I | 2017-08-15T03:01:24+00:00 | Environment:
        [Builder: gitaly] I | 2017-08-15T03:01:24+00:00 |   CFLAGS="-I/opt/gitlab/embedded/include -O2"
        [Builder: gitaly] I | 2017-08-15T03:01:24+00:00 |   CPPFLAGS="-I/opt/gitlab/embedded/include -O2"
        [Builder: gitaly] I | 2017-08-15T03:01:24+00:00 |   CXXFLAGS="-I/opt/gitlab/embedded/include -O2"
        [Builder: gitaly] I | 2017-08-15T03:01:24+00:00 |   LDFLAGS="-Wl,-rpath,/opt/gitlab/embedded/lib -L/opt/gitlab/embedded/lib"
        [Builder: gitaly] I | 2017-08-15T03:01:24+00:00 |   LD_RUN_PATH="/opt/gitlab/embedded/lib"
        [Builder: gitaly] I | 2017-08-15T03:01:24+00:00 |   MAKE="gmake"
        [Builder: gitaly] I | 2017-08-15T03:01:24+00:00 |   OMNIBUS_INSTALL_DIR="/opt/gitlab"
        [Builder: gitaly] I | 2017-08-15T03:01:24+00:00 |   PATH="/opt/gitlab/bin:/opt/gitlab/embedded/bin:/builds/gitlab/omnibus-gitlab/gems/ruby/2.3.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
        [Builder: gitaly] I | 2017-08-15T03:01:24+00:00 |   PKG_CONFIG_PATH="/opt/gitlab/embedded/lib/pkgconfig"
        [Builder: gitaly] I | 2017-08-15T03:01:24+00:00 | $ gmake install PREFIX=/opt/gitlab/embedded
        [Builder: gitaly] I | 2017-08-15T03:01:27+00:00 | Execute: `gmake install PREFIX=/opt/gitlab/embedded': 46.1303s
        [Builder: gitaly] I | 2017-08-15T03:01:27+00:00 | Build gitaly: 46.1331s
The following shell command exited with status 2:

    $ CFLAGS=-I/opt/gitlab/embedded/include -O2 CPPFLAGS=-I/opt/gitlab/embedded/include -O2 CXXFLAGS=-I/opt/gitlab/embedded/include -O2 LDFLAGS=-Wl,-rpath,/opt/gitlab/embedded/lib -L/opt/gitlab/embedded/lib LD_RUN_PATH=/opt/gitlab/embedded/lib MAKE=gmake OMNIBUS_INSTALL_DIR=/opt/gitlab PATH=/opt/gitlab/bin:/opt/gitlab/embedded/bin:/builds/gitlab/omnibus-gitlab/gems/ruby/2.3.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PKG_CONFIG_PATH=/opt/gitlab/embedded/lib/pkgconfig gmake install PREFIX=/opt/gitlab/embedded

Output:

    cd ruby && bundle install
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Warning: the running version of Bundler (1.13.7) is older than the version that created the lockfile (1.15.3). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.
Fetching gem metadata from https://rubygems.org/.........
Fetching version metadata from https://rubygems.org/..
Using concurrent-ruby 1.0.5
Using i18n 0.8.1
Using minitest 5.9.1
Using thread_safe 0.3.6
Using public_suffix 2.0.5
Using charlock_holmes 0.7.3
Using escape_utils 1.1.1
Using multipart-post 2.0.0
Using google-protobuf 3.3.0
Using jwt 1.5.6
Using little-plugger 1.1.4
Using multi_json 1.12.1
Using memoist 0.16.0
Using os 0.9.6
Using mime-types-data 3.2016.0521
Installing rugged 0.26.0 with native extensions
Using bundler 1.13.7
Using tzinfo 1.2.2
Using addressable 2.5.1
Using faraday 0.12.2
Using logging 2.2.2
Using mime-types 3.1
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

current directory:
/var/cache/omnibus/src/gitaly/ruby/vendor/bundle/gems/rugged-0.26.0/ext/rugged
/opt/gitlab/embedded/bin/ruby -r ./siteconf20170815-24852-cwhbdu.rb extconf.rb
--no-use-system-libraries
checking for gmake... yes
checking for cmake... yes
checking for pkg-config... yes
-- cmake .. -DBUILD_CLAR=OFF -DTHREADSAFE=ON -DBUILD_SHARED_LIBS=OFF
-DCMAKE_C_FLAGS=-fPIC -DCMAKE_BUILD_TYPE=RelWithDebInfo
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
	--with-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/opt/gitlab/embedded/bin/$(RUBY_BASE_NAME)
	--with-sha1dc
	--without-sha1dc
	--use-system-libraries
extconf.rb:21:in `sys': ERROR: 'cmake .. -DBUILD_CLAR=OFF -DTHREADSAFE=ON
-DBUILD_SHARED_LIBS=OFF -DCMAKE_C_FLAGS=-fPIC -DCMAKE_BUILD_TYPE=RelWithDebInfo 
' failed (RuntimeError)
	from extconf.rb:83:in `block (2 levels) in <main>'
	from extconf.rb:80:in `chdir'
	from extconf.rb:80:in `block in <main>'
	from extconf.rb:77:in `chdir'
	from extconf.rb:77:in `<main>'

To see why this extension failed to compile, please check the mkmf.log which can
be found here:

/var/cache/omnibus/src/gitaly/ruby/vendor/bundle/extensions/x86_64-linux/2.3.0/rugged-0.26.0/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in
/var/cache/omnibus/src/gitaly/ruby/vendor/bundle/gems/rugged-0.26.0 for
inspection.
Results logged to
/var/cache/omnibus/src/gitaly/ruby/vendor/bundle/extensions/x86_64-linux/2.3.0/rugged-0.26.0/gem_make.out

An error occurred while installing rugged (0.26.0), and Bundler cannot continue.
Make sure that `gem install rugged -v '0.26.0'` succeeds before bundling.

Error:

    gmake: *** [.ruby-bundle] Error 5

@jacobvosmaer-gitlab Suspects that our ruby_build_dir might be incorrect because gmake install got called directly. The check for existence of ruby_build_dir should prevent that.

I've checked the version on master of gitaly on dev.gitlab.org, it is the latest available version

Removed the clause checking for that directory as it should exist in master. Build is in https://dev.gitlab.org/gitlab/omnibus-gitlab/pipelines/60850

I just realised that the log says:

current directory:
/var/cache/omnibus/src/gitaly/ruby/vendor/bundle/gems/rugged-0.26.0/ext/rugged

So that would mean that we are in the right directory.

That build is also failing. https://dev.gitlab.org/gitlab/omnibus-gitlab/-/jobs/1336975

~~Looks like it's running bundle install for the wrong Gemfile.~~

I misread, it's failing in gitlab-rails. Something with charlock_holms and libicu.

@jacobvosmaer-gitlab It looks like the charlock_holms revert didn't get synced to dev. I will sync manually and retry.

I booted up a build machine to quickly check this and it seems that the directory check clause is causing an issue. After removing it, gitaly compiled. So in theory, this should now pass: https://dev.gitlab.org/gitlab/omnibus-gitlab/pipelines/60850

mentioned in merge request !1853 (merged)

I wonder if this is one of those fun problems where DSL commands get added to some queue to be executed later, and our if File.exist? runs 'too early'.

closed via merge request !1853 (merged)

Gitaly with ruby is not compiling in master

Designs

Child items ...

Activity

Admin message

Admin message

Gitaly with ruby is not compiling in master

Activity