Skip to content
GitLab
    • GitLab: the DevOps platform
    • Explore GitLab
    • Install GitLab
    • How GitLab compares
    • Get started
    • GitLab docs
    • GitLab Learn
  • Pricing
  • Talk to an expert
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
    • Switch to GitLab Next
    Projects Groups Topics Snippets
  • Register
  • Sign in
  • gitlab-runner gitlab-runner
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Issues 972
    • Issues 972
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Jira
    • Jira
  • Merge requests 88
    • Merge requests 88
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Artifacts
    • Schedules
    • Test cases
  • Deployments
    • Deployments
    • Environments
  • Packages and registries
    • Packages and registries
    • Container Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • Code review
    • Insights
    • Issue
  • Activity
  • Create a new issue
  • Jobs
  • Issue Boards
Collapse sidebar

Do not update/delete: Banner broadcast message test data

Do not update/delete: Notification broadcast message test data

  • GitLab.orgGitLab.org
  • gitlab-runnergitlab-runner
  • Issues
  • #196
Closed
Open
Issue created Sep 25, 2015 by username-removed-99119@artem-sidorenko

gitlab-ci-multi-runnner isn't closing the sockets to docker?

We use gitlab-ci-multi-runner with docker. Both are installed on the system and communication runs via 127.0.0.1:2376 with TLS authentication. Yesterday all CI jobs started to fail with "Too many files open", the analyze showed that too much sockets were open. It looks like gitlab-ci-multi-runner doesn't close the connection to docker after job is finished (the containers are removed still).

$ netstat -anp | grep ESTAB | grep gitlab-ci
tcp        0      0 127.0.0.1:60089         127.0.0.1:2376          ESTABLISHED 19157/gitlab-ci-mul 
tcp        0      0 127.0.0.1:60166         127.0.0.1:2376          ESTABLISHED 19157/gitlab-ci-mul 
tcp        0      0 127.0.0.1:60033         127.0.0.1:2376          ESTABLISHED 19157/gitlab-ci-mul
# Now I trigger some single&simple CI job and wait for it
# and now the stats after the run (docker shows no running containers)
$ netstat -anp | grep ESTAB | grep gitlab-ci
tcp        0      0 127.0.0.1:60089         127.0.0.1:2376          ESTABLISHED 19157/gitlab-ci-mul 
tcp        0      0 127.0.0.1:60498         127.0.0.1:2376          ESTABLISHED 19157/gitlab-ci-mul 
tcp        0      0 127.0.0.1:60166         127.0.0.1:2376          ESTABLISHED 19157/gitlab-ci-mul 
tcp        0      0 127.0.0.1:60033         127.0.0.1:2376          ESTABLISHED 19157/gitlab-ci-mul

I did the upgrade to the latest available version of gitlab-ci-multi-runner and still see the same behavior like above.

Following versions and environment are used:

$ cat /etc/*release*
NAME="Red Hat Enterprise Linux Server"
VERSION="7.1 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="7.1"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.1 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.1:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.1
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.1"
Red Hat Enterprise Linux Server release 7.1 (Maipo)
Red Hat Enterprise Linux Server release 7.1 (Maipo)
cpe:/o:redhat:enterprise_linux:7.1:ga:server
$ rpm -qa | grep gitlab
gitlab-ci-multi-runner-0.5.5_1_g69bc934-1.x86_64
$ docker --version
Docker version 1.7.1, build 3043001/1.7.1

Following configuration file is used:

concurrent = 10

[[runners]]
  name = "centos-chefdk"
  url = "<URL>"
  token = "<TOKEN>"
  limit = 10
  executor = "docker"
  builds_dir = ""
  clean_environment = true
  shell = ""
  disable_verbose = false
  [runners.docker]
    host = "tcp://127.0.0.1:2376"
    tls_cert_path = "/home/gitlab_ci_multi_runner/.docker"
    hostname = ""
    image = "centos-chefdk"
    privileged = false
    disable_cache = true
    disable_pull = true
    volumes = []
    cache_dir = ""
    registry = ""

[[runners]]
  name = "centos-chefdk-vagrant"
  url = "<URL>"
  token = "<TOKEN>"
  limit = 1
  executor = "docker"
  builds_dir = ""
  clean_environment = true
  shell = ""
  disable_verbose = false
  [runners.docker]
    host = "tcp://127.0.0.1:2376"
    tls_cert_path = "/home/gitlab_ci_multi_runner/.docker"
    hostname = ""
    image = "centos-chefdk-vagrant"
    privileged = false
    disable_cache = true
    disable_pull = true
    volumes = []
    cache_dir = ""
    registry = ""
Assignee
Assign to
Time tracking