Skip to content
Snippets Groups Projects
Commit 1b464bfe authored by Kamil Trzcińśki's avatar Kamil Trzcińśki
Browse files

Update documentation and config files

parent f2d89023
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 70 additions and 642 deletions
config/gitlab.yml.example
+ 19
− 3
View file @ 1b464bfe
@@ -110,7 +110,23 @@ production: &base
# ssl_url: "https://..." # default: https://secure.gravatar.com/avatar/%{hash}?s=%{size}&d=identicon
 
#
# 2. Auth settings
# 2. GitLab CI settings
# ==========================
gitlab_ci:
# Default project notifications settings:
#
# Send emails only on broken builds (default: true)
# all_broken_builds: true
#
# Add pusher to recipients list (default: false)
# add_pusher: true
# The location where build traces are stored (default: builds/). Relative paths are relative to Rails.root
# builds_path: builds/
#
# 3. Auth settings
# ==========================
 
## LDAP settings
@@ -256,7 +272,7 @@ production: &base
 
 
#
# 3. Advanced settings
# 4. Advanced settings
# ==========================
 
# GitLab Satellites
@@ -315,7 +331,7 @@ production: &base
timeout: 10
 
#
# 4. Extra customization
# 5. Extra customization
# ==========================
 
extra:
doc/README.md
+ 22
− 0
View file @ 1b464bfe
@@ -15,6 +15,23 @@
- [Web hooks](web_hooks/web_hooks.md) Let GitLab notify you when new code has been pushed to your project.
- [Workflow](workflow/README.md) Using GitLab functionality and importing projects from GitHub and SVN.
 
## CI Documentation
+ [Quick Start](ci/quick_start/README.md)
+ [Configuring project (.gitlab-ci.yml)](ci/yaml/README.md)
+ [Configuring runner](ci/runners/README.md)
+ [Configuring deployment](ci/deployment/README.md)
+ [Using Docker Images](ci/docker/using_docker_images.md)
+ [Using Docker Build](ci/docker/using_docker_build.md)
+ [Using Variables](ci/variables/README.md)
### CI Examples
+ [Test and deploy Ruby applications to Heroku](ci/examples/test-and-deploy-ruby-application-to-heroku.md)
+ [Test and deploy Python applications to Heroku](ci/examples/test-and-deploy-python-application-to-heroku.md)
+ [Test Clojure applications](ci/examples/test-clojure-application.md)
+ Help your favorite programming language and GitLab by sending a merge request with a guide for that language.
## Administrator documentation
 
- [Custom git hooks](hooks/custom_hooks.md) Custom git hooks (on the filesystem) for when web hooks aren't enough.
@@ -31,6 +48,11 @@
- [Welcome message](customization/welcome_message.md) Add a custom welcome message to the sign-in page.
- [Reply by email](reply_by_email/README.md) Allow users to comment on issues and merge requests by replying to notification emails.
 
### Administrator documentation
+ [User permissions](permissions/README.md)
+ [API](api/README.md)
## Contributor documentation
 
- [Development](development/README.md) Explains the architecture and the guidelines for shell commands.
doc/ci/README.md
+ 0
− 4
View file @ 1b464bfe
@@ -19,9 +19,5 @@
 
### Administrator documentation
 
+ [Install](install/README.md)
+ [Update](update/README.md)
+ [User permissions](permissions/README.md)
+ [Backup/Restore](raketasks/backup_restore.md)
+ [Migrating to packaged CI](migration_to_omnibus/README.md)
+ [API](api/README.md)
doc/ci/api/README.md
+ 2
− 2
View file @ 1b464bfe
@@ -29,12 +29,12 @@ sending the `private-token` of a valid user and the `url` of an
authorized Gitlab instance via a query string along with the API
request:
 
GET http://ci.example.com/api/v1/projects?private_token=QVy1PB7sTxfy4pqfZM1U&url=http://demo.gitlab.com/
GET http://gitlab.example.com/ci/api/v1/projects?private_token=QVy1PB7sTxfy4pqfZM1U&url=http://demo.gitlab.com/
 
If preferred, you may instead send the `private-token` as a header in
your request:
 
curl --header "PRIVATE-TOKEN: QVy1PB7sTxfy4pqfZM1U" "http://ci.example.com/api/v1/projects?url=http://demo.gitlab.com/"
curl --header "PRIVATE-TOKEN: QVy1PB7sTxfy4pqfZM1U" "http://gitlab.example.com/ci/api/v1/projects?url=http://demo.gitlab.com/"
 
 
### Authentication #2: GitLab CI project token
doc/ci/api/builds.md
+ 2
− 2
View file @ 1b464bfe
@@ -8,7 +8,7 @@ __Authentication is done by runner token__
 
### Runs oldest pending build by runner
 
POST /builds/register
POST /ci/builds/register
 
Parameters:
 
@@ -32,7 +32,7 @@ Returns:
 
### Update details of an existing build
 
PUT /builds/:id
PUT /ci/builds/:id
 
Parameters:
 
doc/ci/api/commits.md
+ 2
− 2
View file @ 1b464bfe
@@ -8,7 +8,7 @@ __Authentication is done by GitLab CI project token__
 
Get list of commits per project
 
GET /commits
GET /ci/commits
 
Parameters:
 
@@ -58,7 +58,7 @@ Inform GitLab CI about new commit you want it to build.
__If commit already exists in GitLab CI it will not be created__
 
 
POST /commits
POST /ci/commits
 
Parameters:
 
doc/ci/api/forks.md
+ 1
− 1
View file @ 1b464bfe
@@ -12,7 +12,7 @@ __Authentication is done by GitLab user token & GitLab project token__
 
 
```
POST /forks
POST /ci/forks
```
 
Parameters:
doc/ci/api/projects.md
+ 8
− 8
View file @ 1b464bfe
@@ -12,7 +12,7 @@ __Authentication is done by GitLab user token & GitLab url__
Lists all projects that the authenticated user has access to.
 
```
GET /projects
GET /ci/projects
```
 
Returns:
@@ -55,7 +55,7 @@ Returns:
Lists all projects that the authenticated user owns.
 
```
GET /projects/owned
GET /ci/projects/owned
```
 
Returns:
@@ -84,7 +84,7 @@ Returns:
Returns information about a single project for which the user is
authorized.
 
GET /projects/:id
GET /ci/projects/:id
 
Parameters:
 
@@ -94,7 +94,7 @@ Parameters:
 
Creates a Gitlab CI project using Gitlab project details.
 
POST /projects
POST /ci/projects
 
Parameters:
 
@@ -109,7 +109,7 @@ Parameters:
Updates a Gitlab CI project using Gitlab project details that the
authenticated user has access to.
 
PUT /projects/:id
PUT /ci/projects/:id
 
Parameters:
 
@@ -123,7 +123,7 @@ Parameters:
 
Removes a Gitlab CI project that the authenticated user has access to.
 
DELETE /projects/:id
DELETE /ci/projects/:id
 
Parameters:
 
@@ -134,7 +134,7 @@ Parameters:
Links a runner to a project so that it can make builds (only via
authorized user).
 
POST /projects/:id/runners/:runner_id
POST /ci/projects/:id/runners/:runner_id
 
Parameters:
 
@@ -146,7 +146,7 @@ Parameters:
Removes a runner from a project so that it can not make builds (only
via authorized user).
 
DELETE /projects/:id/runners/:runner_id
DELETE /ci/projects/:id/runners/:runner_id
 
Parameters:
 
doc/ci/api/runners.md
+ 3
− 3
View file @ 1b464bfe
@@ -9,7 +9,7 @@ __Authentication is done by GitLab user token & GitLab url__
Used to get information about all runners registered on the Gitlab CI
instance.
 
GET /runners
GET /ci/runners
 
Returns:
 
@@ -33,7 +33,7 @@ __Authentication is done with a Shared runner registration token or a project Sp
 
Used to make Gitlab CI aware of available runners.
 
POST /runners/register
POST /ci/runners/register
 
Parameters:
 
@@ -58,7 +58,7 @@ __Authentication is done by runner token__
 
Used to removing runners.
 
DELETE /runners/delete
DELETE /ci/runners/delete
 
Parameters:
 
doc/ci/docker/using_docker_build.md
+ 2
− 2
View file @ 1b464bfe
@@ -35,7 +35,7 @@ GitLab Runner then executes build scripts as `gitlab-runner` user.
 
```bash
$ sudo gitlab-runner register -n \
--url http://ci.gitlab.com \
--url http://gitlab.com/ci \
--token RUNNER_TOKEN \
--executor shell
--description "My Runner"
@@ -84,7 +84,7 @@ In order to do that follow the steps:
 
```bash
$ sudo gitlab-runner register -n \
--url http://ci.gitlab.com \
--url http://gitlab.com/ci \
--token RUNNER_TOKEN \
--executor docker \
--description "My Docker Runner" \
doc/ci/docker/using_docker_images.md
+ 1
− 1
View file @ 1b464bfe
@@ -14,7 +14,7 @@ To use GitLab Runner with Docker you need to register new runner to use `docker`
 
```bash
gitlab-ci-multi-runner register \
--url "https://ci.gitlab.com/" \
--url "https://gitlab.com/" \
--registration-token "PROJECT_REGISTRATION_TOKEN" \
--description "docker-ruby-2.1" \
--executor "docker" \
doc/ci/examples/test-and-deploy-python-application-to-heroku.md
+ 2
− 2
View file @ 1b464bfe
@@ -55,11 +55,11 @@ You can do this through the [Dashboard](https://dashboard.heroku.com/).
### Create runner
First install [Docker Engine](https://docs.docker.com/installation/).
To build this project you also need to have [GitLab Runner](https://about.gitlab.com/gitlab-ci/#gitlab-runner).
You can use public runners available on `ci.gitlab.com`, but you can register your own:
You can use public runners available on `gitlab.com/ci`, but you can register your own:
```
gitlab-ci-multi-runner register \
--non-interactive \
--url "https://ci.gitlab.com/" \
--url "https://gitlab.com/ci/" \
--registration-token "PROJECT_REGISTRATION_TOKEN" \
--description "python-3.2" \
--executor "docker" \
doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md
+ 2
− 2
View file @ 1b464bfe
@@ -50,11 +50,11 @@ You can do this through the [Dashboard](https://dashboard.heroku.com/).
### Create runner
First install [Docker Engine](https://docs.docker.com/installation/).
To build this project you also need to have [GitLab Runner](https://about.gitlab.com/gitlab-ci/#gitlab-runner).
You can use public runners available on `ci.gitlab.com`, but you can register your own:
You can use public runners available on `gitlab.com/ci`, but you can register your own:
```
gitlab-ci-multi-runner register \
--non-interactive \
--url "https://ci.gitlab.com/" \
--url "https://gitlab.com/ci/" \
--registration-token "PROJECT_REGISTRATION_TOKEN" \
--description "ruby-2.1" \
--executor "docker" \
doc/ci/examples/test-clojure-application.md
+ 1
− 1
View file @ 1b464bfe
@@ -32,4 +32,4 @@ In before script we install JRE and [Leiningen](http://leiningen.org/).
Sample project uses [migratus](https://github.com/yogthos/migratus) library to manage database migrations.
So we added database migration as last step of `before_script` section
 
You can use public runners available on `ci.gitlab.com` for testing your application with such configuration.
You can use public runners available on `gitlab.com` for testing your application with such configuration.
doc/ci/install/README.md deleted 100644 → 0
+ 0
− 276
View file @ f2d89023
# Select Version to Install
Make sure you view this installation guide from the branch (version) of GitLab CI you would like to install. In most cases
this should be the highest numbered stable branch (example shown below).
![capture](http://i.imgur.com/fmdlXxa.png)
If this is unclear check the [GitLab Blog](http://blog.gitlab.org/) for installation guide links by version.
## GitLab CI 7.12 requires GitLab 7.12 or newer
other [requirements](requirements.md)
# Setup:
## 1. Packages / Dependencies
`sudo` is not installed on Debian by default. Make sure your system is
up-to-date and install it.
sudo apt-get update
sudo apt-get upgrade
**Note:**
During this installation some files will need to be edited manually. If
you are familiar with vim set it as default editor with the commands
below. If you are not familiar with vim please skip this and keep using
the default editor.
# Install vim
sudo apt-get install vim
sudo update-alternatives --set editor /usr/bin/vim.basic
Install the required packages:
sudo apt-get install wget curl gcc checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libreadline6-dev libc6-dev libssl-dev libmysql++-dev make build-essential zlib1g-dev openssh-server git-core libyaml-dev postfix libpq-dev libicu-dev openssl nodejs
sudo apt-get install redis-server
# 2. Ruby
Download Ruby and compile it:
mkdir /tmp/ruby && cd /tmp/ruby
curl --progress http://cache.ruby-lang.org/pub/ruby/ruby-2.1.6.tar.bz2 | tar xj
cd ruby-2.1.6/
./configure --disable-install-rdoc
make
sudo make install
Install the Bundler Gem:
sudo gem install bundler --no-ri --no-rdoc
## 3. GitLab CI user:
sudo adduser --disabled-login --gecos 'GitLab CI' gitlab_ci
## 4. Prepare the database
We recommend PostgreSQL but you can also use MySQL
### MySQL
# Install the database packages
sudo apt-get install mysql-server mysql-client libmysqlclient-dev
# Login to MySQL
$ mysql -u root -p
# Create the GitLab CI database
mysql> CREATE DATABASE IF NOT EXISTS `gitlab_ci_production` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`;
# Create the MySQL User change $password to a real password
mysql> CREATE USER 'gitlab_ci'@'localhost' IDENTIFIED BY '$password';
# Grant proper permissions to the MySQL User
mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, LOCK TABLES ON `gitlab_ci_production`.* TO 'gitlab_ci'@'localhost';
# Logout MYSQL
mysql> exit;
### PostgreSQL
# Install the database packages
sudo apt-get install -y postgresql-9.1 libpq-dev
# Login to PostgreSQL
sudo -u postgres psql -d template1
# Create a user for GitLab CI. We do not specify a password because we are using peer authentication.
template1=# CREATE USER gitlab_ci;
# Create the GitLab CI production database & grant all privileges on database
template1=# CREATE DATABASE gitlab_ci_production OWNER gitlab_ci;
# Quit the database session
template1=# \q
# Try connecting to the new database with the new user
sudo -u gitlab_ci -H psql -d gitlab_ci_production
## 5. Get code
cd /home/gitlab_ci/
sudo -u gitlab_ci -H git clone https://gitlab.com/gitlab-org/gitlab-ci.git
cd gitlab-ci
sudo -u gitlab_ci -H git checkout 7-12-stable
## 6. Setup application
# Edit application settings
# Production
sudo -u gitlab_ci -H cp config/application.yml.example config/application.yml
sudo -u gitlab_ci -H editor config/application.yml
# Development
#sudo -u gitlab_ci -H cp config/application.yml.example.development config/application.yml
# Copy the example secrets file
sudo -u gitlab_ci -H cp config/secrets.yml.example config/secrets.yml
sudo -u gitlab_ci -H chmod 0600 config/secrets.yml
# Edit web server settings
sudo -u gitlab_ci -H cp config/unicorn.rb.example config/unicorn.rb
sudo -u gitlab_ci -H editor config/unicorn.rb
# Create socket and pid directories
sudo -u gitlab_ci -H mkdir -p tmp/sockets/
sudo chmod -R u+rwX tmp/sockets/
sudo -u gitlab_ci -H mkdir -p tmp/pids/
sudo chmod -R u+rwX tmp/pids/
# Change the permissions of the directory where build traces are stored
sudo chmod -R u+rwX builds/
### Install gems
# For MySQL (note, the option says "without ... postgres")
sudo -u gitlab_ci -H bundle install --without development test postgres --deployment
# Or for PostgreSQL (note, the option says "without ... mysql")
sudo -u gitlab_ci -H bundle install --without development test mysql --deployment
### Setup db
# mysql
sudo -u gitlab_ci -H cp config/database.yml.mysql config/database.yml
# postgres
sudo -u gitlab_ci -H cp config/database.yml.postgresql config/database.yml
# Edit user/password (not necessary with default Postgres setup)
sudo -u gitlab_ci -H editor config/database.yml
# Setup tables
sudo -u gitlab_ci -H bundle exec rake setup RAILS_ENV=production
# Setup schedules
sudo -u gitlab_ci -H bundle exec whenever -w RAILS_ENV=production
### Secure secrets.yml
The `secrets.yml` file stores encryption keys for sessions and secure variables.
Backup `secrets.yml` someplace safe, but don't store it in the same place as your database backups.
Otherwise your secrets are exposed if one of your backups is compromised.
## 8. Install Init Script
Copy the init script (will be /etc/init.d/gitlab_ci):
sudo cp /home/gitlab_ci/gitlab-ci/lib/support/init.d/gitlab_ci /etc/init.d/gitlab_ci
Make GitLab CI start on boot:
sudo update-rc.d gitlab_ci defaults 21
Start your GitLab CI instance:
sudo service gitlab_ci start
# or
sudo /etc/init.d/gitlab_ci start
# 8. Nginx
## Installation
sudo apt-get install nginx
## Site Configuration
Download an example site config:
sudo cp /home/gitlab_ci/gitlab-ci/lib/support/nginx/gitlab_ci /etc/nginx/sites-available/gitlab_ci
sudo ln -s /etc/nginx/sites-available/gitlab_ci /etc/nginx/sites-enabled/gitlab_ci
Make sure to edit the config file to match your setup:
# Change **YOUR_SERVER_IP** and **YOUR_SERVER_FQDN**
# to the IP address and fully-qualified domain name
# of your host serving GitLab CI
sudo editor /etc/nginx/sites-enabled/gitlab_ci
## Check your configuration
sudo nginx -t
## Start nginx
sudo /etc/init.d/nginx start
# 9. GitLab OAuth2 application
Go to the admin area of GitLab, to the `Application` section. Create an application for the GitLab CI
For callback URL use: `http://ci.example.com/user_sessions/callback` if you use http, or `https://ci.example.com/user_sessions/callback` if you use https.
When `app_id` and `app_secret` are generated add them to the GitLab CI config:
```
production:
gitlab_server:
url: 'http://gitlab.example.com'
app_id: XXXXXX
app_secret: XXXXXX
```
# 10. Runners
Now you need Runners to process your builds.
Checkout the [Gitlab Runner section](https://about.gitlab.com/gitlab-ci/#gitlab-runner) to install it
# Done!
Visit YOUR_SERVER for your first GitLab CI login.
You will be asked to authorize with your GitLab credentials.
**Enjoy!**
## Advanced settings
### SMTP email settings
If you want to use SMTP do next:
# Copy config file
sudo -u gitlab_ci -H cp config/initializers/smtp_settings.rb.sample config/initializers/smtp_settings.rb
# Edit it with your settings
sudo -u gitlab_ci -H editor config/initializers/smtp_settings.rb
Restart application
### Custom Redis Connection
If you'd like Resque to connect to a Redis server on a non-standard port or on
a different host, you can configure its connection string via the
`config/resque.yml` file.
# example
production: redis://redis.example.tld:6379
If you want to connect the Redis server via socket, then use the "unix:" URL scheme
and the path to the Redis socket file in the `config/resque.yml` file.
# example
production: unix:/path/to/redis/socket
doc/ci/install/requirements.md deleted 100644 → 0
+ 0
− 61
View file @ f2d89023
# Requirements
## Operating Systems
### Supported Unix distributions
- Ubuntu
- Debian
- CentOS
- Red Hat Enterprise Linux (please use the CentOS packages and instructions)
- Scientific Linux (please use the CentOS packages and instructions)
- Oracle Linux (please use the CentOS packages and instructions)
For the installations options please see [the installation page on the GitLab website](https://about.gitlab.com/installation/).
### Unsupported Unix distributions
- OS X
- Arch Linux
- Fedora
- Gentoo
- FreeBSD
### Non-Unix operating systems such as Windows
GitLab CI is developed for Unix operating systems.
GitLab CI does **not** run on Windows and we have no plans of supporting it in the near future.
Please consider using a virtual machine to run GitLab CI.
## Ruby versions
GitLab requires Ruby (MRI) 2.0 or 2.1
You will have to use the standard MRI implementation of Ruby.
We love [JRuby](http://jruby.org/) and [Rubinius](http://rubini.us/) but GitLab CI needs several Gems that have native extensions.
### Memory
You need at least 1GB of addressable memory (RAM + swap) to install and use GitLab CI!
## Unicorn Workers
It's possible to increase the amount of unicorn workers and this will usually help for to reduce the response time of the applications and increase the ability to handle parallel requests.
For most instances we recommend using: CPU cores + 1 = unicorn workers.
So for a machine with 2 cores, 3 unicorn workers is ideal.
For all machines that have 1GB and up we recommend a minimum of three unicorn workers.
If you have a 512MB machine with a magnetic (non-SSD) swap drive we recommend to configure only one Unicorn worker to prevent excessive swapping.
With one Unicorn worker only git over ssh access will work because the git over HTTP access requires two running workers (one worker to receive the user request and one worker for the authorization check).
If you have a 512MB machine with a SSD drive you can use two Unicorn workers, this will allow HTTP access although it will be slow due to swapping.
To change the Unicorn workers when you have the Omnibus package please see [the Unicorn settings in the Omnibus GitLab documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/unicorn.md#unicorn-settings).
## Supported web browsers
- Chrome (Latest stable version)
- Firefox (Latest released version and [latest ESR version](https://www.mozilla.org/en-US/firefox/organizations/))
- Safari 7+ (known problem: required fields in html5 do not work)
- Opera (Latest released version)
- IE 10+
doc/ci/migration_to_omnibus/README.md deleted 100644 → 0
+ 0
− 29
View file @ f2d89023
## Migrating to packaged CI
Since version 5.1 GitLab CI is shipping as part of the GitLab omnibus package. This guide describes how to migrate GitLab CI from a source installation to an Omnibus package.
### 1. Update GitLab
Update GitLab CI manually to the version that you will install using the omnibus package (at least 7.11). Follow the update [manual for installation from sourse](update/README.md)
### 2. Backup
```
sudo -u gitlab_ci -H bundle exec rake backup:create RAILS_ENV=production
```
This command will create a backup file in the tmp folder
(`/home/gitlab_ci/gitlab_ci/tmp/backups/*_gitlab_ci_backup.tar.gz`). You can read more in the [GitLab CI backup/restore documentation](https://gitlab.com/gitlab-org/gitlab-ci/blob/master/doc/raketasks/backup_restore.md)
### 2. Install a packaged GitLab CI
This process is described in the [instruction for enabling GitLab CI](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/gitlab-ci/README.md)
### 4. Restore backup
Put backup file to directory `/var/opt/gitlab/backups`.
Run the restore command:
```
sudo gitlab-ci-rake backup:restore
```
doc/ci/quick_start/README.md
+ 3
− 3
View file @ 1b464bfe
@@ -16,7 +16,7 @@ Push your application to that repository.
## 3. Add project to CI
 
The next part is to login to GitLab CI.
Point your browser to the URL you have set GitLab CI or use [ci.gitlab.com](http://ci.gitlab.com/) that is linked to [GitLab.com](http://GitLab.com/).
Point your browser to the URL you have set GitLab or use [gitlab.com/ci](http://gitlab.com/ci/).
 
On the first screen you will see a list of GitLab's projects that you have access to:
 
@@ -88,7 +88,7 @@ More information about different runner types can be found in [Configuring runne
To check if you have runners assigned to your project go to **Runners**. You will find there information how to setup project specific runner:
 
1. Install GitLab Runner software. Checkout the [GitLab Runner](https://about.gitlab.com/gitlab-ci/#gitlab-runner) section to install it.
1. Specify following URL during runner setup: https://ci.gitlab.com/
1. Specify following URL during runner setup: https://gitlab.com/ci/
1. Use the following registration token during setup: TOKEN
 
If you do it correctly your runner should be shown under **Runners activated for this project**:
@@ -97,7 +97,7 @@ If you do it correctly your runner should be shown under **Runners activated for
 
### Shared runners
 
If you use [ci.gitlab.com](http://ci.gitlab.com/) you can use **Shared runners** provided by GitLab Inc.
If you use [gitlab.com/ci](http://gitlab.com/ci/) you can use **Shared runners** provided by GitLab Inc.
These are special virtual machines that are run on GitLab's infrastructure that can build any project.
To enable **Shared runners** you have to go to **Runners** and click **Enable shared runners** for this project.
 
doc/ci/raketasks/README.md deleted 100644 → 0
+ 0
− 3
View file @ f2d89023
# Rake Tasks
+ [Backup/Restore](backup_restore.md)
\ No newline at end of file
doc/ci/raketasks/backup_restore.md deleted 100644 → 0
+ 0
− 237
View file @ f2d89023
# Backup restore
## Create a backup of the GitLab CI
A backup creates an archive file that contains the database and builds files.
This archive will be saved in backup_path (see `config/application.yml`).
The filename will be `[TIMESTAMP]_gitlab_ci_backup.tar.gz`. This timestamp can be used to restore an specific backup.
You can only restore a backup to exactly the same version of GitLab CI that you created it on, for example 7.10.1.
*If you are interested in the GitLab backup please follow to the [GitLab backup documentation](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/raketasks/backup_restore.md)*
```
# use this command if you've installed GitLab CI with the Omnibus package
sudo gitlab-ci-rake backup:create
# if you've installed GitLab from source
sudo -u gitlab_ci -H bundle exec rake backup:create RAILS_ENV=production
```
Example output:
```
Dumping database ...
Dumping PostgreSQL database gitlab_ci_development ... [DONE]
done
Dumping builds ...
done
Creating backup archive: 1430930060_gitlab_ci_backup.tar.gz ... done
Uploading backup archive to remote storage ... skipped
Deleting tmp directories ... done
done
Deleting old backups ... skipping
```
## Upload backups to remote (cloud) storage
You can let the backup script upload the '.tar.gz' file it creates.
It uses the [Fog library](http://fog.io/) to perform the upload.
In the example below we use Amazon S3 for storage.
But Fog also lets you use [other storage providers](http://fog.io/storage/).
For omnibus packages:
```ruby
gitlab_ci['backup_upload_connection'] = {
'provider' => 'AWS',
'region' => 'eu-west-1',
'aws_access_key_id' => 'AKIAKIAKI',
'aws_secret_access_key' => 'secret123'
}
gitlab_ci['backup_upload_remote_directory'] = 'my.s3.bucket'
gitlab_ci['backup_multipart_chunk_size'] = 104857600
```
For installations from source:
```yaml
backup:
# snip
upload:
# Fog storage connection settings, see http://fog.io/storage/ .
connection:
provider: AWS
region: eu-west-1
aws_access_key_id: AKIAKIAKI
aws_secret_access_key: 'secret123'
# The remote 'directory' to store your backups. For S3, this would be the bucket name.
remote_directory: 'my.s3.bucket'
multipart_chunk_size: 104857600
```
If you are uploading your backups to S3 you will probably want to create a new
IAM user with restricted access rights. To give the upload user access only for
uploading backups create the following IAM profile, replacing `my.s3.bucket`
with the name of your bucket:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1412062044000",
"Effect": "Allow",
"Action": [
"s3:AbortMultipartUpload",
"s3:GetBucketAcl",
"s3:GetBucketLocation",
"s3:GetObject",
"s3:GetObjectAcl",
"s3:ListBucketMultipartUploads",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::my.s3.bucket/*"
]
},
{
"Sid": "Stmt1412062097000",
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListAllMyBuckets"
],
"Resource": [
"*"
]
},
{
"Sid": "Stmt1412062128000",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::my.s3.bucket"
]
}
]
}
```
## Storing configuration files
Please be informed that a backup does not store your configuration and secret files.
If you use an Omnibus package please see the [instructions in the readme to backup your configuration](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#backup-and-restore-omnibus-gitlab-configuration).
If you have a cookbook installation there should be a copy of your configuration in Chef.
If you have an installation from source:
1. please backup `config/secrets.yml` file that contains key to encrypt variables in database,
but don't store it in the same place as your database backups.
Otherwise your secrets are exposed in case one of your backups is compromised.
1. please consider backing up your `application.yml` file,
1. any SSL keys and certificates,
1. and your [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079).
## Restore a previously created backup
You can only restore a backup to exactly the same version of GitLab CI that you created it on, for example 7.10.1.
### Installation from source
```
sudo -u gitlab_ci -H bundle exec rake backup:restore RAILS_ENV=production
```
Options
```
BACKUP=timestamp_of_backup (required if more than one backup exists)
```
### Omnibus package installation
We will assume that you have installed GitLab CI from an omnibus package and run
`sudo gitlab-ctl reconfigure` at least once.
First make sure your backup tar file is in `/var/opt/gitlab/backups`.
```shell
sudo cp 1393513186_gitlab_ci_backup.tar.gz /var/opt/gitlab/backups/
```
Next, restore the backup by running the restore command. You need to specify the
timestamp of the backup you are restoring.
```shell
# Stop processes that are connected to the database
sudo gitlab-ctl stop ci-unicorn
sudo gitlab-ctl stop ci-sidekiq
# This command will overwrite the contents of your GitLab CI database!
sudo gitlab-ci-rake backup:restore BACKUP=1393513186
# Start GitLab
sudo gitlab-ctl start
```
If there is a GitLab version mismatch between your backup tar file and the installed
version of GitLab, the restore command will abort with an error. Install a package for
the [required version](https://www.gitlab.com/downloads/archives/) and try again.
## Configure cron to make daily backups
### For installation from source:
```
cd /home/git/gitlab
sudo -u gitlab_ci -H editor config/application.yml # Enable keep_time in the backup section to automatically delete old backups
sudo -u gitlab_ci crontab -e # Edit the crontab for the git user
```
Add the following lines at the bottom:
```
# Create a backup of the GitLab CI every day at 4am
0 4 * * * cd /home/gitlab_ci/gitlab_ci && PATH=/usr/local/bin:/usr/bin:/bin bundle exec rake backup:create RAILS_ENV=production CRON=1
```
The `CRON=1` environment setting tells the backup script to suppress all progress output if there are no errors.
This is recommended to reduce cron spam.
### Omnibus package installation
To schedule a cron job that backs up your GitLab CI, use the root user:
```
sudo su -
crontab -e
```
There, add the following line to schedule the backup for everyday at 2 AM:
```
0 2 * * * /opt/gitlab/bin/gitlab-ci-rake backup:create CRON=1
```
You may also want to set a limited lifetime for backups to prevent regular
backups using all your disk space. To do this add the following lines to
`/etc/gitlab/gitlab.rb` and reconfigure:
```
# limit backup lifetime to 7 days - 604800 seconds
gitlab_ci['backup_keep_time'] = 604800
```
NOTE: This cron job does not [backup your omnibus-gitlab configuration](#backup-and-restore-omnibus-gitlab-configuration).
## Known issues
If you’ve been using GitLab CI since 7.11 or before using MySQL and the official installation guide, you will probably get the following error while making a backup: `Dumping MySQL database gitlab_ci_production ... mysqldump: Got error: 1044: Access denied for user 'gitlab_ci'@'localhost' to database 'gitlab_ci_production' when using LOCK TABLES` .This can be resolved by adding a LOCK TABLES permission to the gitlab_ci MySQL user. Add this permission with:
```
$ mysql -u root -p
mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, LOCK TABLES ON `gitlab_ci_production`.* TO 'gitlab_ci'@'localhost';
```
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment