Commit 4218116e authored by Jacob Vosmaer's avatar Jacob Vosmaer
Browse files

Merge branch 'master' of gitlab.com:gitlab-org/omnibus-gitlab into 6-6-pre

- Fix issue where Unicorn was not getting restarted by `gitlab-ctl reconfigure`
- Support for external_url configuration setting
- Faster Ruby source downloads
parents 43feccad e5409c91
......@@ -5,7 +5,10 @@ GitLab!
 
## Installation
 
### Ubuntu
After the steps below your GitLab instance should reachable over HTTP,
and have an admin user with username `root` and password `5iveL!fe`.
### Ubuntu 12.04
 
```
sudo apt-get install openssh-server
......@@ -14,194 +17,85 @@ sudo dpkg -i gitlab-x.y.z.deb # this is the .deb you downloaded
sudo gitlab-ctl reconfigure
```
 
## How to manage an Omnibus-installed GitLab
### Administrative commands
You can make configuration changes by editing `/etc/gitlab/gitlab.rb` and
`/etc/gitlab/gitlab-secrets.json`, followed by running
### CentOS 6.5
 
```
sudo yum install openssh-server
sudo yum install postfix # sendmail or exim is also OK
sudo rpm -i gitlab-x.y.z.rpm
sudo gitlab-ctl reconfigure
# Open up the firewall for HTTP and SSH
sudo lokkit -s http -s ssh
```
 
To start/stop a component of GitLab run e.g.
`sudo gitlab-ctl stop sidekiq`. To permanently disable e.g. Sidekiq, add
`sidekiq['enable'] = false` to `/etc/gitlab/gitlab.rb`, and run
`sudo gitlab-ctl reconfigure` for the change to take effect.
To invoke a GitLab rake task, use `gitlab-rake`. For example:
```
sudo gitlab-rake gitlab:backup:create
```
There is no need to change the user or the `RAILS_ENV` environment variable;
this is taken care of by the `gitlab-rake` wrapper script.
### Directory structure
Omnibus-gitlab uses four different directories.
- `/opt/gitlab` holds application code for GitLab and its dependencies.
- `/var/opt/gitlab` holds application data and configuration files that
`gitlab-ctl reconfigure` writes too.
- `/etc/gitlab` holds configuration files for omnibus-gitlab. These are
the only files that you should ever have to edit manually.
- `/var/log/gitlab` contains all log data generated by components of
omnibus-gitlab.
## Preparing a build environment
To create builds you will need a build user (`omnibus-build:omnibus-build` in our example).
Preparing the build machine requires sudo access.
### Ubuntu 12.04
## How to manage an Omnibus-installed GitLab
 
```shell
# Install dependencies
sudo apt-get install ruby1.9.1 ruby1.9.1-dev git build-essential
sudo gem install --no-ri --no-rdoc bundler
# Create build directories for use by the build user
sudo mkdir -p /opt/gitlab /var/cache/omnibus
# We assume the build user and group is 'omnibus-build'
sudo chown omnibus-build:omnibus-build /opt/gitlab /var/cache/omnibus
```
### Start/stop GitLab
 
Then, as the build user (omnibus-build):
You can start, stop or restart GitLab and all of its components with the
following commands.
 
```shell
# Clone the omnibus repo
git clone https://gitlab.com/gitlab-org/omnibus-gitlab.git
# Start all GitLab components
sudo gitlab-ctl start
 
# Install gem dependencies for omnibus-ruby
cd omnibus-gitlab
bundle install --path .bundle --binstubs
# Stop all GitLab components
sudo gitlab-ctl stop
 
# Do a build (and take a break from the computer)
bin/omnibus build project gitlab
# Restart all GitLab components
sudo gitlab-ctl restart
```
 
### Centos 6.5
It is also possible to start, stop or restart individual components.
 
```shell
sudo yum groupinstall 'Development Tools'
# Install RedHat Software Collections to get Ruby 1.9.3
sudo yum install centos-release-SCL
sudo yum install ruby193 ruby193-ruby-devel
# Create build directories for use by the build user
sudo mkdir -p /opt/gitlab /var/cache/omnibus
# We assume the build user and group is 'omnibus-build'
sudo chown omnibus-build:omnibus-build /opt/gitlab /var/cache/omnibus
sudo gitlab-ctl restart unicorn
```
 
As the build user (omnibus-build):
### Creating the gitlab.rb configuration file
 
```shell
# Enable Ruby 1.9.3 from Software Collections
echo 'exec scl enable ruby193 bash' >> .bash_profile
# Start a new login shell so we do not have to log out and in
# this one time
bash --login
# Clone the omnibus repo
git clone https://gitlab.com/gitlab-org/omnibus-gitlab.git
# Install gem dependencies for omnibus-ruby
cd omnibus-gitlab
bundle install --path .bundle --binstubs
# Do a build (and take a break from the computer)
bin/omnibus build project gitlab
sudo mkdir -p /etc/gitlab
sudo touch /etc/gitlab/gitlab.rb
sudo chmod 600 /etc/gitlab/gitlab.rb
```
 
## Usage
### Build
### Configuring the external URL for GitLab
 
You create a platform-specific package using the `build project` command:
In order for GitLab to display correct repository clone links to your users
it needs to know the URL under which it is reached by your users, e.g.
`http://gitlab.example.com`. Add the following line to `/etc/gitlab/gitlab.rb`:
 
```shell
$ bin/omnibus build project gitlab
```
The platform/architecture type of the package created will match the platform
where the `build project` command is invoked. So running this command on say a
MacBook Pro will generate a Mac OS X specific package. After the build
completes packages will be available in `pkg/`.
### Clean
You can clean up all temporary files generated during the build process with
the `clean` command:
```shell
$ bin/omnibus clean
```ruby
external_url "http://gitlab.example.com"
```
 
Adding the `--purge` purge option removes __ALL__ files generated during the
build including the project install directory (`/opt/gitlab`) and
the package cache directory (`/var/cache/omnibus/pkg`):
```shell
$ bin/omnibus clean --purge
```
Run `sudo gitlab-ctl reconfigure` for the change to take effect.
 
### Help
### Invoking Rake tasks
 
Full help for the Omnibus command line interface can be accessed with the
`help` command:
To invoke a GitLab Rake task, use `gitlab-rake`. For example:
 
```shell
$ bin/omnibus help
sudo gitlab-rake gitlab:backup:create
```
 
## Vagrant-based Virtualized Build Lab
Every Omnibus project ships will a project-specific
[Berksfile](http://berkshelf.com/) and [Vagrantfile](http://www.vagrantup.com/)
that will allow you to build your projects on the following platforms:
* CentOS 5 64-bit
* CentOS 6 64-bit
* Ubuntu 10.04 64-bit
* Ubuntu 11.04 64-bit
* Ubuntu 12.04 64-bit
Please note this build-lab is only meant to get you up and running quickly;
there's nothing inherent in Omnibus that restricts you to just building CentOS
or Ubuntu packages. See the Vagrantfile to add new platforms to your build lab.
The only requirements for standing up this virtualized build lab are:
* VirtualBox - native packages exist for most platforms and can be downloaded
from the [VirtualBox downloads page](https://www.virtualbox.org/wiki/Downloads).
* Vagrant 1.2.1+ - native packages exist for most platforms and can be downloaded
from the [Vagrant downloads page](http://downloads.vagrantup.com/).
Contrary to with a traditional GitLab installation, there is no need to change
the user or the `RAILS_ENV` environment variable; this is taken care of by the
`gitlab-rake` wrapper script.
 
The [vagrant-berkshelf](https://github.com/RiotGames/vagrant-berkshelf) and
[vagrant-omnibus](https://github.com/schisamo/vagrant-omnibus) Vagrant plugins
are also required and can be installed easily with the following commands:
```shell
$ vagrant plugin install vagrant-berkshelf
$ vagrant plugin install vagrant-omnibus
```
Once the pre-requisites are installed you can build your package across all
platforms with the following command:
### Directory structure
 
```shell
$ vagrant up
```
Omnibus-gitlab uses four different directories.
 
If you would like to build a package for a single platform the command looks like this:
- `/opt/gitlab` holds application code for GitLab and its dependencies.
- `/var/opt/gitlab` holds application data and configuration files that
`gitlab-ctl reconfigure` writes to.
- `/etc/gitlab` holds configuration files for omnibus-gitlab. These are
the only files that you should ever have to edit manually.
- `/var/log/gitlab` contains all log data generated by components of
omnibus-gitlab.
 
```shell
$ vagrant up PLATFORM
```
## Building your own package
 
The complete list of valid platform names can be viewed with the
`vagrant status` command.
See [the separate build documentation](doc/build.md).
#
# Copyright:: Copyright (c) 2012 Opscode, Inc.
# License:: Apache License, Version 2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
name "ruby"
version "1.9.3-p484"
dependency "zlib"
dependency "ncurses"
dependency "libedit"
dependency "openssl"
dependency "libyaml"
dependency "libiconv"
dependency "gdbm" if (platform == "mac_os_x" or platform == "freebsd" or platform == "aix")
dependency "libgcc" if (platform == "solaris2" and Omnibus.config.solaris_compiler == "gcc")
source :url => "http://cache.ruby-lang.org/pub/ruby/1.9/ruby-#{version}.tar.gz",
:md5 => '8ac0dee72fe12d75c8b2d0ef5d0c2968'
relative_path "ruby-#{version}"
env =
case platform
when "mac_os_x"
{
"CFLAGS" => "-arch x86_64 -m64 -L#{install_dir}/embedded/lib -I#{install_dir}/embedded/include -I#{install_dir}/embedded/include/ncurses -O3 -g -pipe",
"LDFLAGS" => "-arch x86_64 -R#{install_dir}/embedded/lib -L#{install_dir}/embedded/lib -I#{install_dir}/embedded/include -I#{install_dir}/embedded/include/ncurses"
}
when "solaris2"
if Omnibus.config.solaris_compiler == "studio"
{
"CFLAGS" => "-L#{install_dir}/embedded/lib -I#{install_dir}/embedded/include",
"LDFLAGS" => "-R#{install_dir}/embedded/lib -L#{install_dir}/embedded/lib -I#{install_dir}/embedded/include"
}
elsif Omnibus.config.solaris_compiler == "gcc"
{
"CFLAGS" => "-L#{install_dir}/embedded/lib -I#{install_dir}/embedded/include -O3 -g -pipe",
"LDFLAGS" => "-R#{install_dir}/embedded/lib -L#{install_dir}/embedded/lib -I#{install_dir}/embedded/include -static-libgcc",
"LD_OPTIONS" => "-R#{install_dir}/embedded/lib"
}
else
raise "Sorry, #{Omnibus.config.solaris_compiler} is not a valid compiler selection."
end
when "aix"
{
# see http://www.ibm.com/developerworks/aix/library/au-gnu.html
#
# specifically:
#
# "To use AIX run-time linking, you should create the shared object
# using gcc -shared -Wl,-G and create executables using the library
# by adding the -Wl,-brtl option to the link line. Technically, you
# can leave off the -shared option, but it does no harm and reduces
# confusion."
#
# AIX also uses -Wl,-blibpath instead of -R or LD_RUN_PATH, but the
# option is not additive, so requires /usr/lib and /lib as well (there
# is a -bsvr4 option to allow ld to take an -R flag in addition
# to turning on -brtl, but it had other side effects I couldn't fix).
#
# If libraries linked with gcc -shared have symbol resolution failures
# then it may be useful to add -bexpfull to export all symbols.
#
# -O2 optimized away some configure test which caused ext libs to fail
#
# We also need prezl's M4 instead of picking up /usr/bin/m4 which
# barfs on ruby.
#
"CC" => "xlc -q64",
"CXX" => "xlC -q64",
"LD" => "ld -b64",
"CFLAGS" => "-q64 -O -qhot -I#{install_dir}/embedded/include",
"CXXFLAGS" => "-q64 -O -qhot -I#{install_dir}/embedded/include",
"LDFLAGS" => "-q64 -L#{install_dir}/embedded/lib -Wl,-brtl -Wl,-blibpath:#{install_dir}/embedded/lib:/usr/lib:/lib",
"OBJECT_MODE" => "64",
"ARFLAGS" => "-X64 cru",
"M4" => "/opt/freeware/bin/m4",
"warnflags" => "-qinfo=por"
}
else
{
"CFLAGS" => "-I#{install_dir}/embedded/include -O3 -g -pipe",
"LDFLAGS" => "-Wl,-rpath,#{install_dir}/embedded/lib -L#{install_dir}/embedded/lib"
}
end
build do
configure_command = ["./configure",
"--prefix=#{install_dir}/embedded",
"--with-out-ext=fiddle",
"--enable-shared",
"--enable-libedit",
"--with-ext=psych",
"--disable-install-doc"]
case platform
when "aix"
patch :source => "ruby-aix-configure.patch", :plevel => 1
patch :source => "ruby_aix_1_9_3_448_ssl_EAGAIN.patch", :plevel => 1
# --with-opt-dir causes ruby to send bogus commands to the AIX linker
when "freebsd"
configure_command << "--without-execinfo"
configure_command << "--with-opt-dir=#{install_dir}/embedded"
when "smartos"
# Opscode patch - someara@opscode.com
# GCC 4.7.0 chokes on mismatched function types between OpenSSL 1.0.1c and Ruby 1.9.3-p286
patch :source => "ruby-openssl-1.0.1c.patch", :plevel => 1
# Patches taken from RVM.
# http://bugs.ruby-lang.org/issues/5384
# https://www.illumos.org/issues/1587
# https://github.com/wayneeseguin/rvm/issues/719
patch :source => "rvm-cflags.patch", :plevel => 1
# From RVM forum
# https://github.com/wayneeseguin/rvm/commit/86766534fcc26f4582f23842a4d3789707ce6b96
configure_command << "ac_cv_func_dl_iterate_phdr=no"
configure_command << "--with-opt-dir=#{install_dir}/embedded"
else
configure_command << "--with-opt-dir=#{install_dir}/embedded"
end
# @todo expose bundle_bust() in the DSL
env.merge!({
"RUBYOPT" => nil,
"BUNDLE_BIN_PATH" => nil,
"BUNDLE_GEMFILE" => nil,
"GEM_PATH" => nil,
"GEM_HOME" => nil
})
# @todo: move into omnibus-ruby
has_gmake = system("gmake --version")
if has_gmake
env.merge!({'MAKE' => 'gmake'})
make_binary = 'gmake'
else
make_binary = 'make'
end
command configure_command.join(" "), :env => env
command "#{make_binary} -j #{max_build_jobs}", :env => env
command "#{make_binary} -j #{max_build_jobs} install", :env => env
end
# Preparing a build environment
To create builds you will need a build user (`omnibus-build:omnibus-build` in our example).
Preparing the build machine requires sudo access.
## Ubuntu 12.04
```shell
# Install dependencies
sudo apt-get install ruby1.9.1 ruby1.9.1-dev git build-essential
sudo gem install --no-ri --no-rdoc bundler
# Create build directories for use by the build user
sudo mkdir -p /opt/gitlab /var/cache/omnibus
# We assume the build user and group is 'omnibus-build'
sudo chown omnibus-build:omnibus-build /opt/gitlab /var/cache/omnibus
```
Then, as the build user (omnibus-build):
```shell
# Clone the omnibus repo
git clone https://gitlab.com/gitlab-org/omnibus-gitlab.git
# Install gem dependencies for omnibus-ruby
cd omnibus-gitlab
bundle install --path .bundle --binstubs
# Do a build (and take a break from the computer)
bin/omnibus build project gitlab
```
## Centos 6.5
```shell
sudo yum groupinstall 'Development Tools'
# Install RedHat Software Collections to get Ruby 1.9.3
sudo yum install centos-release-SCL
sudo yum install ruby193 ruby193-ruby-devel
# Create build directories for use by the build user
sudo mkdir -p /opt/gitlab /var/cache/omnibus
# We assume the build user and group is 'omnibus-build'
sudo chown omnibus-build:omnibus-build /opt/gitlab /var/cache/omnibus
```
As the build user (omnibus-build):
```shell
# Enable Ruby 1.9.3 from Software Collections
echo 'exec scl enable ruby193 bash' >> .bash_profile
# Start a new login shell so we do not have to log out and in
# this one time
bash --login
# Clone the omnibus repo
git clone https://gitlab.com/gitlab-org/omnibus-gitlab.git
# Install gem dependencies for omnibus-ruby
cd omnibus-gitlab
bundle install --path .bundle --binstubs
# Do a build (and take a break from the computer)
bin/omnibus build project gitlab
```
## Usage
### Build
You create a platform-specific package using the `build project` command:
```shell
$ bin/omnibus build project gitlab
```
The platform/architecture type of the package created will match the platform
where the `build project` command is invoked. So running this command on say a
MacBook Pro will generate a Mac OS X specific package. After the build
completes packages will be available in `pkg/`.
### Clean
You can clean up all temporary files generated during the build process with
the `clean` command:
```shell
$ bin/omnibus clean
```
Adding the `--purge` purge option removes __ALL__ files generated during the
build including the project install directory (`/opt/gitlab`) and
the package cache directory (`/var/cache/omnibus/pkg`):
```shell
$ bin/omnibus clean --purge
```
### Help
Full help for the Omnibus command line interface can be accessed with the
`help` command:
```shell
$ bin/omnibus help
```
## Vagrant-based Virtualized Build Lab
Every Omnibus project ships will a project-specific
[Berksfile](http://berkshelf.com/) and [Vagrantfile](http://www.vagrantup.com/)
that will allow you to build your projects on the following platforms:
* CentOS 5 64-bit
* CentOS 6 64-bit
* Ubuntu 10.04 64-bit
* Ubuntu 11.04 64-bit
* Ubuntu 12.04 64-bit
Please note this build-lab is only meant to get you up and running quickly;
there's nothing inherent in Omnibus that restricts you to just building CentOS
or Ubuntu packages. See the Vagrantfile to add new platforms to your build lab.
The only requirements for standing up this virtualized build lab are:
* VirtualBox - native packages exist for most platforms and can be downloaded
from the [VirtualBox downloads page](https://www.virtualbox.org/wiki/Downloads).
* Vagrant 1.2.1+ - native packages exist for most platforms and can be downloaded
from the [Vagrant downloads page](http://downloads.vagrantup.com/).
The [vagrant-berkshelf](https://github.com/RiotGames/vagrant-berkshelf) and
[vagrant-omnibus](https://github.com/schisamo/vagrant-omnibus) Vagrant plugins
are also required and can be installed easily with the following commands:
```shell
$ vagrant plugin install vagrant-berkshelf
$ vagrant plugin install vagrant-omnibus
```
Once the pre-requisites are installed you can build your package across all
platforms with the following command:
```shell
$ vagrant up
```
If you would like to build a package for a single platform the command looks like this:
```shell
$ vagrant up PLATFORM
```
The complete list of valid platform names can be viewed with the
`vagrant status` command.
# Omnibus-gitlab release process
Our main goal is to make it clear which version of GitLab is in an omnibus package.
## On your development machine
- Pick a tag of GitLab to package (e.g. `v6.6.0`).
- Create a release branch in omnibus-gitlab (e.g. `6-6-stable`).
- Change [the gitlab-rails version in omnibus-gitlab].
In our example that would be `version "v6.6.0"`.
- Commit the new version to the release branch
```shell
git commit -m 'Pin GitLab to v6.6.0' config/software/gitlab-rails.rb
```
- Create an annotated tag on omnibus-gitlab corresponding to the GitLab tag.
GitLab tag `v6.6.0` becomes omnibus-gitlab tag `6.6.0.omnibus`.
```shell
git tag -a 6.6.0.omnibus -m 'Pin GitLab to v6.6.0'
```
- Push the branch and the tag to the main repository.
```shell
git push origin 6-6-stable 6.6.0.omnibus
```
## On the build machines
- Check out the release branch of omnibus-gitlab.
```shell
git fetch
git checkout 6-6-stable
```
- Check the version with `git describe`.
```shell
git describe # Should start with 6.6.0.omnibus
```
- Build a package with timestamps disabled.
```shell
OMNIBUS_APPEND_TIMESTAMP=0 bin/omnibus build project gitlab
```
[the gitlab-rails version in omnibus-gitlab]: ../config/software/gitlab-rails.rb#L20
......@@ -15,7 +15,7 @@
# limitations under the License.
#
 
define :template_symlink, :link_from => nil, :source => nil, :owner => nil, :group => nil, :mode => nil, :variables => nil, :notifies => nil do
define :template_symlink, :link_from => nil, :source => nil, :owner => nil, :group => nil, :mode => nil, :variables => nil, :notifies => nil, :restarts => [] do
template params[:name] do
source params[:source]
owner params[:owner]
......@@ -23,6 +23,9 @@ define :template_symlink, :link_from => nil, :source => nil, :owner => nil, :gro
mode params[:mode]
variables params[:variables]
notifies *params[:notifies] if params[:notifies]
params[:restarts].each do |resource|
notifies :restart, resource
end
end
 
link params[:link_from] do
......
......@@ -21,11 +21,13 @@ require 'chef/mash'
require 'chef/json_compat'
require 'chef/mixin/deep_merge'
require 'securerandom'
require 'uri'
 
module Gitlab
extend(Mixlib::Config)
&n