Skip to content
Snippets Groups Projects
Commit 291678d6 authored by Achilleas Pipinellis's avatar Achilleas Pipinellis
Browse files

Merge branch 'ci-services-docs' into 'master' 

Introduce CI documentation for services and languages

This introduces guide for PHP, a bunch of services (databases) and cleans up some docker references.

With this there are bunch of example projects in https://gitlab.com/groups/gitlab-examples.

/cc @JobV @axil 

See merge request !1917
parents ee0ab46d 1e7156ed
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 910 additions and 114 deletions
doc/README.md
+ 12
− 0
View file @ 291678d6
@@ -24,9 +24,21 @@
- [Using Docker Images](ci/docker/using_docker_images.md)
- [Using Docker Build](ci/docker/using_docker_build.md)
- [Using Variables](ci/variables/README.md)
- [Using SSH keys](ci/ssh_keys/README.md)
- [User permissions](ci/permissions/README.md)
- [API](ci/api/README.md)
 
### CI Languages
+ [Testing PHP](ci/languages/php.md)
### CI Services
+ [Using MySQL](ci/services/mysql.md)
+ [Using PostgreSQL](ci/services/postgres.md)
+ [Using Redis](ci/services/redis.md)
+ [Using Other Services](ci/docker/using_docker_images.md#how-to-use-other-images-as-services)
### CI Examples
 
- [Test and deploy Ruby applications to Heroku](ci/examples/test-and-deploy-ruby-application-to-heroku.md)
doc/ci/README.md
+ 12
− 0
View file @ 291678d6
@@ -9,6 +9,18 @@
+ [Using Docker Images](docker/using_docker_images.md)
+ [Using Docker Build](docker/using_docker_build.md)
+ [Using Variables](variables/README.md)
+ [Using SSH keys](ssh_keys/README.md)
### Languages
+ [Testing PHP](languages/php.md)
### Services
+ [Using MySQL](services/mysql.md)
+ [Using PostgreSQL](services/postgres.md)
+ [Using Redis](services/redis.md)
+ [Using Other Services](docker/using_docker_images.md#how-to-use-other-images-as-services)
 
### Examples
 
doc/ci/docker/using_docker_images.md
+ 171
− 114
View file @ 291678d6
# Using Docker Images
GitLab CI can use [Docker Engine](https://www.docker.com/) to build projects.
 
Docker is an open-source project that allows to use predefined images to run applications
in independent "containers" that are run within a single Linux instance.
[Docker Hub](https://registry.hub.docker.com/) have rich database of built images that can be used to build applications.
GitLab CI in conjuction with [GitLab Runner](../runners/README.md) can use
[Docker Engine](https://www.docker.com/) to test and build any application.
 
Docker when used with GitLab CI runs each build in separate and isolated container using predefined image and always from scratch.
It makes it easier to have simple and reproducible build environment that can also be run on your workstation.
This allows you to test all commands from your shell, rather than having to test them on a CI server.
Docker is an open-source project that allows you to use predefined images to
run applications in independent "containers" that are run within a single Linux
instance. [Docker Hub][hub] has a rich database of prebuilt images that can be
used to test and build your applications.
 
### Register Docker runner
To use GitLab Runner with Docker you need to register new runner to use `docker` executor:
Docker, when used with GitLab CI, runs each build in a separate and isolated
container using the predefined image that is set up in
[`.gitlab-ci.yml`](../yaml/README.md).
This makes it easier to have a simple and reproducible build environment that
can also run on your workstation. The added benefit is that you can test all
the commands that we will explore later from your shell, rather than having to
test them on a dedicated CI server.
## Register docker runner
To use GitLab Runner with docker you need to register a new runner to use the
`docker` executor:
 
```bash
gitlab-ci-multi-runner register \
gitlab-runner register \
--url "https://gitlab.com/" \
--registration-token "PROJECT_REGISTRATION_TOKEN" \
--description "docker-ruby-2.1" \
@@ -23,101 +33,79 @@ gitlab-ci-multi-runner register \
--docker-mysql latest
```
 
**The registered runner will use `ruby:2.1` image and will run two services (`postgres:latest` and `mysql:latest`) that will be accessible for time of the build.**
The registered runner will use the `ruby:2.1` docker image and will run two
services, `postgres:latest` and `mysql:latest`, both of which will be
accessible during the build process.
 
### What is image?
The image is the name of any repository that is present in local Docker Engine or any repository that can be found at [Docker Hub](https://registry.hub.docker.com/).
For more information about the image and Docker Hub please read the [Docker Fundamentals](https://docs.docker.com/introduction/understanding-docker/).
## What is image
 
### What is service?
Service is just another image that is run for time of your build and is linked to your build. This allows you to access the service image during build time.
The service image can run any application, but most common use case is to run some database container, ie.: `mysql`.
It's easier and faster to use existing image, run it as additional container than install `mysql` every time project is built.
The `image` keyword is the name of the docker image that is present in the
local Docker Engine (list all images with `docker images`) or any image that
can be found at [Docker Hub][hub]. For more information about images and Docker
Hub please read the [Docker Fundamentals][] documentation.
 
#### How is service linked to the build?
There's good document that describes how Docker linking works: [Linking containers together](https://docs.docker.com/userguide/dockerlinks/).
To summarize: if you add `mysql` as service to your application, the image will be used to create container that is linked to build container.
The service container for MySQL will be accessible under hostname `mysql`.
So, **to access your database service you have to connect to host: `mysql` instead of socket or `localhost`**.
In short, with `image` we refer to the docker image, which will be used to
create a container on which your build will run.
 
### How to use other images as services?
You are not limited to have only database services.
You can hand modify `config.toml` to add any image as service found at [Docker Hub](https://registry.hub.docker.com/).
Look for `[runners.docker]` section:
```
[runners.docker]
image = "ruby:2.1"
services = ["mysql:latest", "postgres:latest"]
```
## What is service
 
For example you need `wordpress` instance to test some API integration with `Wordpress`.
You can for example use this image: [tutum/wordpress](https://registry.hub.docker.com/u/tutum/wordpress/).
This is image that have fully preconfigured `wordpress` and have `MySQL` server built-in:
```
[runners.docker]
image = "ruby:2.1"
services = ["mysql:latest", "postgres:latest", "tutum/wordpress:latest"]
```
The `services` keyword defines just another docker image that is run during
your build and is linked to the docker image that the `image` keyword defines.
This allows you to access the service image during build time.
 
Next time when you run your application the `tutum/wordpress` will be started
and you will have access to it from your build container under hostname: `tutum__wordpress`.
The service image can run any application, but the most common use case is to
run a database container, eg. `mysql`. It's easier and faster to use an
existing image and run it as an additional container than install `mysql` every
time the project is built.
 
Alias hostname for the service is made from the image name:
1. Everything after `:` is stripped,
2. '/' is replaced with `__`.
You can see some widely used services examples in the relevant documentation of
[CI services examples](../services/README.md).
 
### Configuring services
Many services accept environment variables, which allow you to easily change database names or set account names depending on the environment.
### How is service linked to the build
 
GitLab Runner 0.5.0 and up passes all YAML-defined variables to created service containers.
To better understand how the container linking works, read
[Linking containers together](https://docs.docker.com/userguide/dockerlinks/).
 
1. To configure database name for [postgres](https://registry.hub.docker.com/u/library/postgres/) service,
you need to set POSTGRES_DB.
To summarize, if you add `mysql` as service to your application, the image will
then be used to create a container that is linked to the build container.
 
```yaml
services:
- postgres
variables:
POSTGRES_DB: gitlab
```
The service container for MySQL will be accessible under the hostname `mysql`.
So, in order to access your database service you have to connect to the host
named `mysql` instead of a socket or `localhost`.
 
1. To use [mysql](https://registry.hub.docker.com/u/library/mysql/) service with empty password for time of build,
you need to set MYSQL_ALLOW_EMPTY_PASSWORD.
## Overwrite image and services
 
```yaml
services:
- mysql
variables:
MYSQL_ALLOW_EMPTY_PASSWORD: "yes"
```
See [How to use other images as services](#how-to-use-other-images-as-services).
 
For other possible configuration variables check the
https://registry.hub.docker.com/u/library/mysql/ or https://registry.hub.docker.com/u/library/postgres/
or README page for any other Docker image.
## How to use other images as services
 
**Note: All variables will passed to all service containers. It's not designed to distinguish which variable should go where.**
You are not limited to have only database services. You can add as many
services you need to `.gitlab-ci.yml` or manually modify `config.toml`.
Any image found at [Docker Hub][hub] can be used as a service.
 
### Overwrite image and services
It's possible to overwrite `docker-image` and specify services from `.gitlab-ci.yml`.
If you add to your YAML the `image` and the `services` these parameters
be used instead of the ones that were specified during runner's registration.
```
## Define image and services from `.gitlab-ci.yml`
You can simply define an image that will be used for all jobs and a list of
services that you want to use during build time.
```yaml
image: ruby:2.2
services:
- postgres:9.3
before_install:
before_script:
- bundle install
test:
script:
- bundle exec rake spec
```
 
It's possible to define image and service per-job:
```
before_install:
It is also possible to define different images and services per job:
```yaml
before_script:
- bundle install
 
test:2.1:
@@ -135,34 +123,73 @@ test:2.2:
- bundle exec rake spec
```
 
#### How to enable overwriting?
To enable overwriting you have to **enable it first** (it's disabled by default for security reasons).
You can do that by hand modifying runner configuration: `config.toml`.
Please go to section where is `[runners.docker]` definition for your runner.
Add `allowed_images` and `allowed_services` to specify what images are allowed to be picked from `.gitlab-ci.yml`:
## Define image and services in `config.toml`
Look for the `[runners.docker]` section:
```
[runners.docker]
image = "ruby:2.1"
allowed_images = ["ruby:*", "python:*"]
allowed_services = ["mysql:*", "redis:*"]
services = ["mysql:latest", "postgres:latest"]
```
This enables you to use in your `.gitlab-ci.yml` any image that matches above wildcards.
You will be able to pick only `ruby` and `python` images.
The same rule can be applied to limit services.
 
If you are courageous enough, you can make it fully open and accept everything:
```
[runners.docker]
image = "ruby:2.1"
allowed_images = ["*", "*/*"]
allowed_services = ["*", "*/*"]
The image and services defined this way will be added to all builds run by
that runner.
## Accessing the services
Let's say that you need a Wordpress instance to test some API integration with
your application.
You can then use for example the [tutum/wordpress][] image in your
`.gitlab-ci.yml`:
```yaml
services:
- tutum/wordpress:latest
```
 
**It the feature is not enabled, or image isn't allowed the error message will be put into the build log.**
When the build is run, `tutum/wordpress` will be started and you will have
access to it from your build container under the hostname `tutum__wordpress`.
The alias hostname for the service is made from the image name following these
rules:
1. Everything after `:` is stripped
2. Backslash (`/`) is replaced with double underscores (`__`)
## Configuring services
Many services accept environment variables which allow you to easily change
database names or set account names depending on the environment.
GitLab Runner 0.5.0 and up passes all YAML-defined variables to the created
service containers.
For all possible configuration variables check the documentation of each image
provided in their corresponding Docker hub page.
*Note: All variables will be passed to all services containers. It's not
designed to distinguish which variable should go where.*
### PostgreSQL service example
See the specific documentation for
[using PostgreSQL as a service](../services/postgres.md).
### MySQL service example
See the specific documentation for
[using MySQL as a service](../services/mysql.md).
## How Docker integration works
Below is a high level overview of the steps performed by docker during build
time.
 
### How Docker integration works
1. Create any service container: `mysql`, `postgresql`, `mongodb`, `redis`.
1. Create cache container to store all volumes as defined in `config.toml` and `Dockerfile` of build image (`ruby:2.1` as in above example).
1. Create cache container to store all volumes as defined in `config.toml` and
`Dockerfile` of build image (`ruby:2.1` as in above example).
1. Create build container and link any service container to build container.
1. Start build container and send build script to the container.
1. Run build script.
@@ -171,33 +198,63 @@ If you are courageous enough, you can make it fully open and accept everything:
1. Check exit status of build script.
1. Remove build container and all created service containers.
 
### How to debug a build locally
1. Create a file with build script:
## How to debug a build locally
*Note: The following commands are run without root privileges. You should be
able to run docker with your regular user account.*
First start with creating a file named `build script`:
```bash
$ cat <<EOF > build_script
cat <<EOF > build_script
git clone https://gitlab.com/gitlab-org/gitlab-ci-multi-runner.git /builds/gitlab-org/gitlab-ci-multi-runner
cd /builds/gitlab-org/gitlab-ci-multi-runner
make <- or any other build step
make
EOF
```
 
1. Create service containers:
Here we use as an example the GitLab Runner repository which contains a
Makefile, so running `make` will execute the commands defined in the Makefile.
Your mileage may vary, so instead of `make` you could run the command which
is specific to your project.
Then create some service containers:
```
$ docker run -d -n service-mysql mysql:latest
$ docker run -d -n service-postgres postgres:latest
docker run -d -n service-mysql mysql:latest
docker run -d -n service-postgres postgres:latest
```
This will create two service containers (MySQL and PostgreSQL).
 
1. Create a build container and execute script in its context:
This will create two service containers, named `service-mysql` and
`service-postgres` which use the latest MySQL and PostgreSQL images
respectively. They will both run in the background (`-d`).
Finally, create a build container by executing the `build_script` file we
created earlier:
```
$ docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script
docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script
```
This will create build container that has two service containers linked.
The build_script is piped using STDIN to bash interpreter which executes the build script in container.
 
1. At the end remove all containers:
The above command will create a container named `build` that is spawned from
the `ruby:2.1` image and has two services linked to it. The `build_script` is
piped using STDIN to the bash interpreter which in turn executes the
`build_script` in the `build` container.
When you finish testing and no longer need the containers, you can remove them
with:
```
docker rm -f -v build service-mysql service-postgres
```
This will forcefully (the `-f` switch) remove build container and service containers
and all volumes (the `-v` switch) that were created with the container creation.
This will forcefully (`-f`) remove the `build` container, the two service
containers as well as all volumes (`-v`) that were created with the container
creation.
[Docker Fundamentals]: https://docs.docker.com/engine/introduction/understanding-docker/
[hub]: https://hub.docker.com/
[linking-containers]: https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/
[tutum/wordpress]: https://registry.hub.docker.com/u/tutum/wordpress/
[postgres-hub]: https://registry.hub.docker.com/u/library/postgres/
[mysql-hub]: https://registry.hub.docker.com/u/library/mysql/
doc/ci/languages/README.md 0 → 100644
+ 7
− 0
View file @ 291678d6
### Languages
This is a list of languages you can test with GitLab CI. Each section has
comprehensive documentation and comes with a test repository hosted on
GitLab.com
+ [Testing PHP](php.md)
doc/ci/languages/php.md 0 → 100644
+ 284
− 0
View file @ 291678d6
# Testing PHP projects
This guide covers basic building instructions for PHP projects.
There are covered two cases: testing using the Docker executor and testing
using the Shell executor.
## Test PHP projects using the Docker executor
While it is possible to test PHP apps on any system, this would require manual
configuration from the developer. To overcome this we will be using the
official [PHP docker image][php-hub] that can be found in Docker Hub.
This will allow us to test PHP projects against different versions of PHP.
However, not everything is plug 'n' play, you still need to onfigure some
things manually.
As with every build, you need to create a valid `.gitlab-ci.yml` describing the
build environment.
Let's first specify the PHP image that will be used for the build process
(you can read more about what an image means in the Runner's lingo reading
about [Using Docker images](../docker/using_docker_images.md#what-is-image)).
Start by adding the image to your `.gitlab-ci.yml`:
```yaml
image: php:5.6
```
The official images are great, but they lack a few useful tools for testing.
We need to first prepare the build environment. A way to overcome this is to
create a script which installs all prerequisites prior the actual testing is
done.
Let's create a `ci/docker_install.sh` file in the root directory of our
repository with the following content:
```bash
#!/bin/bash
# We need to install dependencies only for Docker
[[ ! -e /.dockerinit ]] && exit 0
set -xe
# Install git (the php image doesn't have it) which is required by composer
apt-get update -yqq
apt-get install git -yqq
# Install phpunit, the tool that we will use for testing
curl -o /usr/local/bin/phpunit https://phar.phpunit.de/phpunit.phar
chmod +x /usr/local/bin/phpunit
# Install mysql driver
# Here you can install any other extension that you need
docker-php-ext-install pdo_mysql
```
You might wonder what `docker-php-ext-install` is. In short, it is a script
provided by the official php docker image that you can use to easilly install
extensions. For more information read the the documentation at
<https://hub.docker.com/_/php/>.
Now that we created the script that contains all prerequisites for our build
environment, let's add it in `.gitlab-ci.yml`:
```yaml
...
before_script:
- bash ci/docker_install.sh > /dev/null
...
```
Last step, run the actual tests using `phpunit`:
```yaml
...
test:app:
script:
- phpunit --configuration phpunit_myapp.xml
...
```
Finally, commit your files and push them to GitLab to see your build succeeding
(or failing).
The final `.gitlab-ci.yml` should look similar to this:
```yaml
# Select image from https://hub.docker.com/_/php/
image: php:5.6
before_script:
# Install dependencies
- ci/docker_install.sh > /dev/null
test:app:
script:
- phpunit --configuration phpunit_myapp.xml
```
### Test against different PHP versions in Docker builds
Testing against multiple versions of PHP is super easy. Just add another job
with a different docker image version and the runner will do the rest:
```yaml
before_script:
# Install dependencies
- ci/docker_install.sh > /dev/null
# We test PHP5.6
test:5.6:
image: php:5.6
script:
- phpunit --configuration phpunit_myapp.xml
# We test PHP7.0 (good luck with that)
test:7.0:
image: php:7.0
script:
- phpunit --configuration phpunit_myapp.xml
```
### Custom PHP configuration in Docker builds
There are times where you will need to customise your PHP environment by
putting your `.ini` file into `/usr/local/etc/php/conf.d/`. For that purpose
add a `before_script` action:
```yaml
before_script:
- cp my_php.ini /usr/local/etc/php/conf.d/test.ini
```
Of course, `my_php.ini` must be present in the root directory of your repository.
## Test PHP projects using the Shell executor
The shell executor runs your builds in a terminal session on your server.
Thus, in order to test your projects you first need to make sure that all
dependencies are installed.
For example, in a VM running Debian 8 we first update the cache, then we
install `phpunit` and `php5-mysql`:
```bash
sudo apt-get update -y
sudo apt-get install -y phpunit php5-mysql
```
Next, add the following snippet to your `.gitlab-ci.yml`:
```yaml
test:app:
script:
- phpunit --configuration phpunit_myapp.xml
```
Finally, push to GitLab and let the tests begin!
### Test against different PHP versions in Shell builds
The [phpenv][] project allows you to easily manage different versions of PHP
each with its own config. This is specially usefull when testing PHP projects
with the Shell executor.
You will have to install it on your build machine under the `gitlab-runner`
user following [the upstream installation guide][phpenv-installation].
Using phpenv also allows to easily configure the PHP environment with:
```
phpenv config-add my_config.ini
```
*__Important note:__ It seems `phpenv/phpenv`
[is abandoned](https://github.com/phpenv/phpenv/issues/57). There is a fork
at [madumlao/phpenv](https://github.com/madumlao/phpenv) that tries to bring
the project back to life. [CHH/phpenv](https://github.com/CHH/phpenv) also
seems like a good alternative. Picking any of the mentioned tools will work
with the basic phpenv commands. Guiding you to choose the right phpenv is out
of the scope of this tutorial.*
### Install custom extensions
Since this is a pretty bare installation of the PHP environment, you may need
some extensions that are not currently present on the build machine.
To install additional extensions simply execute:
```bash
pecl install <extension>
```
It's not advised to add this to `.gitlab-ci.yml`. You should execute this
command once, only to setup the build environment.
## Extend your tests
### Using atoum
Instead of PHPUnit, you can use any other tool to run unit tests. For example
you can use [atoum](https://github.com/atoum/atoum):
```yaml
before_script:
- wget http://downloads.atoum.org/nightly/mageekguy.atoum.phar
test:atoum:
script:
- php mageekguy.atoum.phar
```
### Using Composer
The majority of the PHP projects use Composer for managing their PHP packages.
In order to execute Composer before running your tests, simply add the
following in your `.gitlab-ci.yml`:
```yaml
...
# Composer stores all downloaded packages in the vendor/ directory.
# Do not use the following if the vendor/ directory is commited to
# your git repository.
cache:
paths:
- vendor/
before_script:
# Install composer dependencies
- curl -sS https://getcomposer.org/installer | php
- php composer.phar install
...
```
## Access private packages / dependencies
If your test suite needs to access a private repository, you need to configure
[the SSH keys](../ssh_keys/README.md) in order to be able to clone it.
## Use databases or other services
Most of the time you will need a running database in order for your tests to
run. If you are using the Docker executor you can leverage Docker's ability to
link to other containers. In GitLab Runner lingo, this can be achieved by
defining a `service`.
This functionality is covered in [the CI services](../services/README.md)
documentation.
## Testing things locally
With GitLab Runner 1.0 you can also test any changes locally. From your
terminal execute:
```bash
# Check using docker executor
gitlab-runner exec docker test:app
# Check using shell executor
gitlab-runner exec shell test:app
```
## Example project
We have set up an [Example PHP Project][php-example-repo] for your convenience
that runs on [GitLab.com](https://gitlab.com) using our publicly available
[shared runners](../runners/README.md).
Want to hack on it? Simply fork it, commit and push your changes. Within a few
moments the changes will be picked by a public runner and the build will begin.
[php-hub]: https://hub.docker.com/_/php/
[phpenv]: https://github.com/phpenv/phpenv
[phpenv-installation]: https://github.com/phpenv/phpenv#installation
[php-example-repo]: https://gitlab.com/gitlab-examples/php
doc/ci/services/README.md 0 → 100644
+ 9
− 0
View file @ 291678d6
## GitLab CI Services
GitLab CI uses the `services` keyword to define what docker containers should be
linked with your base image. Below is a list of examples you may use.
+ [Using MySQL](mysql.md)
+ [Using PostgreSQL](postgres.md)
+ [Using Redis](redis.md)
+ [Using Other Services](../docker/using_docker_images.md#how-to-use-other-images-as-services)
doc/ci/services/docker-services.md 0 → 100644
+ 5
− 0
View file @ 291678d6
## GitLab CI Services
+ [Using MySQL](mysql.md)
+ [Using PostgreSQL](postgres.md)
+ [Using Redis](redis.md)
doc/ci/services/mysql.md 0 → 100644
+ 118
− 0
View file @ 291678d6
# Using MySQL
As many applications depend on MySQL as their database, you will eventually
need it in order for your tests to run. Below you are guided how to do this
with the Docker and Shell executors of GitLab Runner.
## Use MySQL with the Docker executor
If you are using [GitLab Runner](../runners/README.md) with the Docker executor
you basically have everything set up already.
First, in your `.gitlab-ci.yml` add:
```yaml
services:
- mysql:latest
variables:
# Configure mysql environment variables (https://hub.docker.com/_/mysql/)
MYSQL_DATABASE: el_duderino
MYSQL_ROOT_PASSWORD: mysql_strong_password
```
And then configure your application to use the database, for example:
```yaml
Host: mysql
User: root
Password: mysql_strong_password
Database: el_duderino
```
If you are wondering why we used `mysql` for the `Host`, read more at
[How is service linked to the build](../docker/using_docker_images.md#how-is-service-linked-to-the-build).
You can also use any other docker image available on [Docker Hub][hub-mysql].
For example, to use MySQL 5.5 the service becomes `mysql:5.5`.
The `mysql` image can accept some environment variables. For more details
check the documentation on [Docker Hub][hub-mysql].
## Use MySQL with the Shell executor
You can also use MySQL on manually configured servers that are using
GitLab Runner with the Shell executor.
First install the MySQL server:
```bash
sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev
```
Pick a MySQL root password (can be anything), and type it twice when asked.
*Note: As a security measure you can run `mysql_secure_installation` to
remove anonymous users, drop the test database and disable remote logins with
the root user.*
The next step is to create a user, so login to MySQL as root:
```bash
mysql -u root -p
```
Then create a user (in our case `runner`) which will be used by your
application. Change `$password` in the command below to a real strong password.
*Note: Do not type `mysql>`, this is part of the MySQL prompt.*
```bash
mysql> CREATE USER 'runner'@'localhost' IDENTIFIED BY '$password';
```
Create the database:
```bash
mysql> CREATE DATABASE IF NOT EXISTS `el_duderino` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`;
```
Grant the necessary permissions on the database:
```bash
mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES ON `el_duderino`.* TO 'runner'@'localhost';
```
If all went well you can now quit the database session:
```bash
mysql> \q
```
Now, try to connect to the newly created database to check that everything is
in place:
```bash
mysql -u runner -p -D el_duderino
```
As a final step, configure your application to use the database, for example:
```bash
Host: localhost
User: runner
Password: $password
Database: el_duderino
```
## Example project
We have set up an [Example MySQL Project][mysql-example-repo] for your
convenience that runs on [GitLab.com](https://gitlab.com) using our publicly
available [shared runners](../runners/README.md).
Want to hack on it? Simply fork it, commit and push your changes. Within a few
moments the changes will be picked by a public runner and the build will begin.
[hub-mysql]: https://hub.docker.com/_/mysql/
[mysql-example-repo]: https://gitlab.com/gitlab-examples/mysql
doc/ci/services/postgres.md 0 → 100644
+ 114
− 0
View file @ 291678d6
# Using PostgreSQL
As many applications depend on PostgreSQL as their database, you will
eventually need it in order for your tests to run. Below you are guided how to
do this with the Docker and Shell executors of GitLab Runner.
## Use PostgreSQL with the Docker executor
If you are using [GitLab Runner](../runners/README.md) with the Docker executor
you basically have everything set up already.
First, in your `.gitlab-ci.yml` add:
```yaml
services:
- postgres:latest
variables:
POSTGRES_DB: nice_marmot
POSTGRES_USER: runner
POSTGRES_PASSWORD: ""
```
And then configure your application to use the database, for example:
```yaml
Host: postgres
User: runner
Password:
Database: nice_marmot
```
If you are wondering why we used `postgres` for the `Host`, read more at
[How is service linked to the build](../docker/using_docker_images.md#how-is-service-linked-to-the-build).
You can also use any other docker image available on [Docker Hub][hub-pg].
For example, to use PostgreSQL 9.3 the service becomes `postgres:9.3`.
The `postgres` image can accept some environment variables. For more details
check the documentation on [Docker Hub][hub-pg].
## Use PostgreSQL with the Shell executor
You can also use PostgreSQL on manually configured servers that are using
GitLab Runner with the Shell executor.
First install the PostgreSQL server:
```bash
sudo apt-get install -y postgresql postgresql-client libpq-dev
```
The next step is to create a user, so login to PostgreSQL:
```bash
sudo -u postgres psql -d template1
```
Then create a user (in our case `runner`) which will be used by your
application. Change `$password` in the command below to a real strong password.
*__Note:__ Do not type `template1=#`, this is part of the PostgreSQL prompt.*
```bash
template1=# CREATE USER runner WITH PASSWORD '$password' CREATEDB;
```
*__Note:__ Notice that we created the user with the privilege to be able to
create databases (`CREATEDB`). In the following steps we will create a database
explicitly for that user but having that privilege can be useful if in your
testing framework you have tools that drop and create databases.*
Create the database and grant all privileges on it for the user `runner`:
```bash
template1=# CREATE DATABASE nice_marmot OWNER runner;
```
If all went well you can now quit the database session:
```bash
template1=# \q
```
Now, try to connect to the newly created database with the user `runner` to
check that everything is in place.
```bash
psql -U runner -h localhost -d nice_marmot -W
```
*__Note:__ We are explicitly telling `psql` to connect to localhost in order
to use the md5 authentication. If you omit this step you will be denied access.*
Finally, configure your application to use the database, for example:
```yaml
Host: localhost
User: runner
Password: $password
Database: nice_marmot
```
## Example project
We have set up an [Example PostgreSQL Project][postgres-example-repo] for your
convenience that runs on [GitLab.com](https://gitlab.com) using our publicly
available [shared runners](../runners/README.md).
Want to hack on it? Simply fork it, commit and push your changes. Within a few
moments the changes will be picked by a public runner and the build will begin.
[hub-pg]: https://hub.docker.com/_/postgres/
[postgres-example-repo]: https://gitlab.com/gitlab-examples/postgres
doc/ci/services/redis.md 0 → 100644
+ 69
− 0
View file @ 291678d6
# Using Redis
As many applications depend on Redis as their key-value store, you will
eventually need it in order for your tests to run. Below you are guided how to
do this with the Docker and Shell executors of GitLab Runner.
## Use Redis with the Docker executor
If you are using [GitLab Runner](../runners/README.md) with the Docker executor
you basically have everything set up already.
First, in your `.gitlab-ci.yml` add:
```yaml
services:
- redis:latest
```
Then you need to configure your application to use the Redis database, for
example:
```yaml
Host: redis
```
And that's it. Redis will now be available to be used within your testing
framework.
You can also use any other docker image available on [Docker Hub][hub-redis].
For example, to use Redis 2.8 the service becomes `redis:2.8`.
## Use Redis with the Shell executor
Redis can also be used on manually configured servers that are using GitLab
Runner with the Shell executor.
In your build machine install the Redis server:
```bash
sudo apt-get install redis-server
```
Verify that you can connect to the server with the `gitlab-runner` user:
```bash
# Try connecting the the Redis server
sudo -u gitlab-runner -H redis-cli
# Quit the session
127.0.0.1:6379> quit
```
Finally, configure your application to use the database, for example:
```yaml
Host: localhost
```
## Example project
We have set up an [Example Redis Project][redis-example-repo] for your convenience
that runs on [GitLab.com](https://gitlab.com) using our publicly available
[shared runners](../runners/README.md).
Want to hack on it? Simply fork it, commit and push your changes. Within a few
moments the changes will be picked by a public runner and the build will begin.
[hub-redis]: https://hub.docker.com/_/redis/
[redis-example-repo]: https://gitlab.com/gitlab-examples/redis
doc/ci/ssh_keys/README.md 0 → 100644
+ 109
− 0
View file @ 291678d6
# Using SSH keys
GitLab currently doesn't have built-in support for managing SSH keys in a build
environment.
The SSH keys can be useful when:
1. You want to checkout internal submodules
2. You want to download private packages using your package manager (eg. bundler)
3. You want to deploy your application to eg. Heroku or your own server
4. You want to execute SSH commands from the build server to the remote server
5. You want to rsync files from your build server to the remote server
If anything of the above rings a bell, then you most likely need an SSH key.
## Inject keys in your build server
The most widely supported method is to inject an SSH key into your build
environment by extending your `.gitlab-ci.yml`.
This is the universal solution which works with any type of executor
(docker, shell, etc.).
### How it works
1. Create a new SSH key pair with [ssh-keygen][]
2. Add the private key as a **Secret Variable** to the project
3. Run the [ssh-agent][] during build to load the private key.
## SSH keys when using the Docker executor
You will first need to create an SSH key pair. For more information, follow the
instructions to [generate an SSH key](../ssh/README.md).
Then, create a new **Secret Variable** in your project settings on GitLab
following **Settings > Variables**. As **Key** add the name `SSH_PRIVATE_KEY`
and in the **Value** field paste the content of your _private_ key that you
created earlier.
Next you need to modify your `.gitlab-ci.yml` with a `before_script` action.
Add it to the top:
```
before_script:
# Install ssh-agent if not already installed, it is required by Docker.
# (change apt-get to yum if you use a CentOS-based image)
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
# Run ssh-agent (inside the build environment)
- eval $(ssh-agent -s)
# Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
- ssh-add <(echo "$SSH_PRIVATE_KEY")
# For Docker builds disable host key checking. Be aware that by adding that
# you are suspectible to man-in-the-middle attacks.
# WARNING: Use this only with the Docker executor, if you use it with shell
# you will overwrite your user's SSH config.
- mkdir -p ~/.ssh
- '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config`
```
As a final step, add the _public_ key from the one you created earlier to the
services that you want to have an access to from within the build environment.
If you are accessing a private GitLab repository you need to add it as a
[deploy key](../ssh/README.md#deploy-keys).
That's it! You can now have access to private servers or repositories in your
build environment.
## SSH keys when using the Shell executor
If you are using the Shell executor and not Docker, it is easier to set up an
SSH key.
You can generate the SSH key from the machine that GitLab Runner is installed
on, and use that key for all projects that are run on this machine.
First, you need to login to the server that runs your builds.
Then from the terminal login as the `gitlab-runner` user and generate the SSH
key pair as described in the [SSH keys documentation](../ssh/README.md).
As a final step, add the _public_ key from the one you created earlier to the
services that you want to have an access to from within the build environment.
If you are accessing a private GitLab repository you need to add it as a
[deploy key](../ssh/README.md#deploy-keys).
Once done, try to login to the remote server in order to accept the fingerprint:
```bash
ssh <address-of-my-server>
```
For accessing repositories on GitLab.com, the `<address-of-my-server>` would be
`git@gitlab.com`.
## Example project
We have set up an [Example SSH Project][ssh-example-repo] for your convenience
that runs on [GitLab.com](https://gitlab.com) using our publicly available
[shared runners](../runners/README.md).
Want to hack on it? Simply fork it, commit and push your changes. Within a few
moments the changes will be picked by a public runner and the build will begin.
[ssh-keygen]: http://linux.die.net/man/1/ssh-keygen
[ssh-agent]: http://linux.die.net/man/1/ssh-agent
[ssh-example-repo]: https://gitlab.com/gitlab-examples/ssh-private-key/
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment