Use shorter service names for private registry images or add an alternate name field to services specified in .gitlab-ci.yml

Description

Hypothetically, you don't want to use docker:latest and docker:dind for your CI deployment, but you want to use your own images built off those. Here is my story of failing to do that and becoming exasperated.

I thought, naively, that I could simply create a project, say global/docker, set up CI in that repository, and create my own alternative docker:dind. So I did, and I created the tagged image registry.example.com/global/docker:dind. Then I specified in another project's configuration:

  services:
  - registry.example.com/global/docker:dind

But I got:

Cannot connect to the Docker daemon. Is the docker daemon running on this host?

Hm, looks like it's not finding the Docker daemon. Must not be linking it in properly? I look at the Dockerfile for docker:1.12 ~ docker:latest and see that it uses the hostname docker by default. The spec for services says:

Define an image from a private Docker registry

Starting with GitLab Runner 0.6.0, you are able to define images located to private registries that could also require authentication.

All you have to do is be explicit on the image definition in .gitlab-ci.yml. image:

my.registry.tld:5000/namepace/image:tag

In the example above, GitLab Runner will look at my.registry.tld:5000 for the image namespace/image:tag.

If the repository is private you need to authenticate your GitLab Runner in the registry. Learn how to do that on GitLab Runner's documentation.

Accessing the services

Let's say that you need a Wordpress instance to test some API integration with your application.

You can then use for example the tutum/wordpress image in your .gitlab-ci.yml: services:

- tutum/wordpress:latest

When the build is run, tutum/wordpress will be started and you will have access to it from your build container under the hostname tutum__wordpress.

The alias hostname for the service is made from the image name following these rules:

1. Everything after : is stripped
2. Slash (/) is replaced with double underscores (__)

In particular, this leads me to believe the "image name" is the stuff after the registry:

In the example above, GitLab Runner will look at my.registry.tld:5000 for the image namespace/image:tag.

And this leads me to believe the alias hostname for the service is made from the "image name":

The alias hostname for the service is made from the image name following these rules:

I'll just create my own variation of docker:latest and patch the URL to tcp://global__docker. Perfect. Error:

An error occurred trying to connect: Post http://global__docker:2375/v1.24/auth: dial tcp: lookup global__docker on 168.63.129.16:53: no such host

Oh. So the image name isn't what's used. Let's look at what the CI runner linked using rekcod and some quick fingers on running a build and capturing the commands it used to create some containers.

--link runner-f0c82b7f-project-3-concurrent-0-registry.example.com__global__docker:registry.example.com-global-docker 
--link runner-f0c82b7f-project-3-concurrent-0-registry.example.com__global__docker:registry.example.com__global__docker 

That's horrible. So I have to hardcode my registry path into my docker image (or my build scripts) or mess with variables and pre-scripts just to get this to work, and the CI runner created two different names, one of which isn't even to the specification.

Proposal

1.

When using a private registry url, e.g.: registry.example.com/foo/bar:baz, add alternate service names for foo__bar and bar if there are no conflicts for each. If there are, it should fail the lint and not run to avoid ambiguity.

2.

Allow specifying an alternate name for a service. .gitlab-ci.yml is YAML, this would permit spinning up duplicate services under different names.

services: 
  bar_1: registry.example.com/foo/bar:baz
  bar_2: registry.example.com/foo/bar:baz

This seems necessary for testing scale-out services, e.g.: testing a Redis cluster.

It would also satisfy Liskov's substitution principle, e.g.:

services:
  - wordpress:latest

Can be replaced with:

services:
  wordpress: tutum/wordpress

That would simplify changing the service used to one line of code. Swap one redis build for another, switch MySQL with MariaDB in one line. Etc. (I don't actually know if wordpress and tutum/wordpress are substitutable, but bear with me for the sake of example.)

3.

Clean up the specification to avoid the hazard I ran into. It is just not clear at all what happens with services from private registries.

added ~19173 container registry labels

added feature proposal label

mentioned in issue #38047

@aaron.friel It seems like you are after the alias option which is already available, https://docs.gitlab.com/ce/ci/docker/using_docker_images.html#available-settings-for-services (also see https://docs.gitlab.com/runner/executors/docker.html#accessing-the-services)

test:
  stage: test
  services:
    - name: selenium/standalone-chrome:3.4.0-francium
      alias: some-selenium-server
    - name: mysql:latest
      alias: my-sql-server

closed