Skip to content
Snippets Groups Projects
New issue look: Off

Save disk space and download time by stripping binaries on Linux and FreeBSD

    • Closed Issue created by username-removed-210164 @swills

    You can make the binaries for Linux and FreeBSD about 20mb smaller if you strip debugging symbols from them. This would save both download time and disk space.

    Designs

    Child items ...

    • Activity

    • Tomasz Maczukin
      Tomasz Maczukin @tmaczukin ·
      Maintainer

      Interesting. I was able to reduce the size of Linux amd64 binary from 66984 KiB to 47072 KiB by adding only -s -w to -ldflags list. This is 19MiB less while pprof and panic stack traces still have all important data.

      @ayufan WDYT? Do we need debugging symbols in binaries?

    • Kamil Trzcińśki
      Kamil Trzcińśki @ayufan ·
      Maintainer

      Does it affect pprof and stack traces? If not, we can strip them. The nice thing today is that we have exact stack trace with line number, will it be gone?

      Edited by Kamil Trzcińśki
    • Tomasz Maczukin
      Tomasz Maczukin @tmaczukin ·
      Maintainer

      As far as I tested - no, it doesn't affect stack traces and pprof. But I can try it again and confirm.

    • Tomasz Maczukin
      Tomasz Maczukin @tmaczukin ·
      Maintainer

      without -s -w flags

      size

      $ du -k out/binaries/gitlab-ci-multi-runner
65720   out/binaries/gitlab-ci-multi-runner

      strace

      $ gitlab-runner run --config ~/.gitlab-runner/config.toml
Starting multi-runner from /home/tmaczukin/.gitlab-runner/config.toml ...  builds=0
WARNING: Running in user-mode.                     
WARNING: Use sudo for system-mode:                 
WARNING: $ sudo gitlab-runner...                   
                                                   
Configuration loaded                                builds=0
Metrics server listening at 127.0.0.1:6061         
panic: test

goroutine 46 [running]:
gitlab.com/gitlab-org/gitlab-ci-multi-runner/network.(*GitLabClient).RequestJob(0xc42034e810, 0xc4200f8b40, 0x11, 0x0, 0x0, 0x0, 0xc4200f8be0, 0x15, 0xc42033b1d0, 0x1e, ...)
        /home/tmaczukin/Projekty/go/src/gitlab.com/gitlab-org/gitlab-ci-multi-runner/network/gitlab.go:226 +0x269
gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands.(*RunCommand).requestJob(0xc4202ded00, 0xc4200b2dc0, 0x0, 0xc4200b2d00)
        /home/tmaczukin/Projekty/go/src/gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands/multi.go:107 +0xff
gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands.(*RunCommand).processRunner(0xc4202ded00, 0x0, 0xc4200b2dc0, 0xc4200721e0, 0x0, 0x0)
        /home/tmaczukin/Projekty/go/src/gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands/multi.go:134 +0x3b3
gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands.(*RunCommand).processRunners(0xc4202ded00, 0x0, 0xc4201e0060, 0xc4200721e0)
        /home/tmaczukin/Projekty/go/src/gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands/multi.go:183 +0x211
created by gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands.(*RunCommand).startWorkers
        /home/tmaczukin/Projekty/go/src/gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands/multi.go:199 +0x9f

      pprof

      $ curl -s http://127.0.0.1:6061/debug/pprof/goroutine?debug=1 | head -n 20
goroutine profile: total 32
7 @ 0x42cd8a 0x43c184 0x43adec 0x9aa89b 0x459ee1
#       0x9aa89a        gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands.(*RunCommand).processRunners+0x3ba        /home/tmaczukin/Projekty/go/src/gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands/multi.go:181

4 @ 0x42cd8a 0x427ea7 0x4274e9 0x613278 0x6132e4 0x614a37 0x627920 0x6c02e8 0x6c0854 0x6c455d 0x77b4fb 0x5066f7 0x5068b7 0x77c1a6 0x459ee1
#       0x4274e8        net.runtime_pollWait+0x58               /usr/local/go/src/runtime/netpoll.go:164
#       0x613277        net.(*pollDesc).wait+0x37               /usr/local/go/src/net/fd_poll_runtime.go:75
#       0x6132e3        net.(*pollDesc).waitRead+0x33           /usr/local/go/src/net/fd_poll_runtime.go:80
#       0x614a36        net.(*netFD).Read+0x1b6                 /usr/local/go/src/net/fd_unix.go:250
#       0x62791f        net.(*conn).Read+0x6f                   /usr/local/go/src/net/net.go:181
#       0x6c02e7        crypto/tls.(*block).readFromUntil+0x97  /usr/local/go/src/crypto/tls/conn.go:488
#       0x6c0853        crypto/tls.(*Conn).readRecord+0xc3      /usr/local/go/src/crypto/tls/conn.go:590
#       0x6c455c        crypto/tls.(*Conn).Read+0x11c           /usr/local/go/src/crypto/tls/conn.go:1134
#       0x77b4fa        net/http.(*persistConn).Read+0x14a      /usr/local/go/src/net/http/transport.go:1316
#       0x5066f6        bufio.(*Reader).fill+0x116              /usr/local/go/src/bufio/bufio.go:97
#       0x5068b6        bufio.(*Reader).Peek+0x66               /usr/local/go/src/bufio/bufio.go:129
#       0x77c1a5        net/http.(*persistConn).readLoop+0x195  /usr/local/go/src/net/http/transport.go:1474

4 @ 0x42cd8a 0x43c184 0x43adec 0x77ddfa 0x459ee1
#       0x77ddf9        net/http.(*persistConn).writeLoop+0x439 /usr/local/go/src/net/http/transport.go:1704

      with -s -w flags

      size

      $ du -k out/binaries/gitlab-ci-multi-runner
47148   out/binaries/gitlab-ci-multi-runner

      strace

      $ gitlab-runner run --config ~/.gitlab-runner/config.toml
Starting multi-runner from /home/tmaczukin/.gitlab-runner/config.toml ...  builds=0
WARNING: Running in user-mode.                     
WARNING: Use sudo for system-mode:                 
WARNING: $ sudo gitlab-runner...                   
                                                   
Configuration loaded                                builds=0
Metrics server listening at 127.0.0.1:6061         
panic: test

goroutine 22 [running]:
gitlab.com/gitlab-org/gitlab-ci-multi-runner/network.(*GitLabClient).RequestJob(0xc42038ee50, 0xc4200902a0, 0x11, 0x0, 0x0, 0x0, 0xc420090340, 0x15, 0xc42011df20, 0x1e, ...)
        /home/tmaczukin/Projekty/go/src/gitlab.com/gitlab-org/gitlab-ci-multi-runner/network/gitlab.go:226 +0x269
gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands.(*RunCommand).requestJob(0xc420302d00, 0xc4200b2dc0, 0x0, 0xc4200b2d00)
        /home/tmaczukin/Projekty/go/src/gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands/multi.go:107 +0xff
gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands.(*RunCommand).processRunner(0xc420302d00, 0x0, 0xc4200b2dc0, 0xc420073b00, 0x0, 0x0)
        /home/tmaczukin/Projekty/go/src/gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands/multi.go:134 +0x3b3
gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands.(*RunCommand).processRunners(0xc420302d00, 0x0, 0xc4202f8060, 0xc420073b00)
        /home/tmaczukin/Projekty/go/src/gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands/multi.go:183 +0x211
created by gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands.(*RunCommand).startWorkers
        /home/tmaczukin/Projekty/go/src/gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands/multi.go:199 +0x9f

      pprof

      goroutine profile: total 26
9 @ 0x42cd8a 0x43c184 0x43adec 0x9aa89b 0x459ee1
#       0x9aa89a        gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands.(*RunCommand).processRunners+0x3ba        /home/tmaczukin/Projekty/go/src/gitlab.com/gitlab-org/gitlab-ci-multi-runner/commands/multi.go:181

2 @ 0x42cd8a 0x427ea7 0x4274e9 0x613278 0x6132e4 0x614a37 0x627920 0x6c02e8 0x6c0854 0x6c455d 0x77b4fb 0x5066f7 0x5068b7 0x77c1a6 0x459ee1
#       0x4274e8        net.runtime_pollWait+0x58               /usr/local/go/src/runtime/netpoll.go:164
#       0x613277        net.(*pollDesc).wait+0x37               /usr/local/go/src/net/fd_poll_runtime.go:75
#       0x6132e3        net.(*pollDesc).waitRead+0x33           /usr/local/go/src/net/fd_poll_runtime.go:80
#       0x614a36        net.(*netFD).Read+0x1b6                 /usr/local/go/src/net/fd_unix.go:250
#       0x62791f        net.(*conn).Read+0x6f                   /usr/local/go/src/net/net.go:181
#       0x6c02e7        crypto/tls.(*block).readFromUntil+0x97  /usr/local/go/src/crypto/tls/conn.go:488
#       0x6c0853        crypto/tls.(*Conn).readRecord+0xc3      /usr/local/go/src/crypto/tls/conn.go:590
#       0x6c455c        crypto/tls.(*Conn).Read+0x11c           /usr/local/go/src/crypto/tls/conn.go:1134
#       0x77b4fa        net/http.(*persistConn).Read+0x14a      /usr/local/go/src/net/http/transport.go:1316
#       0x5066f6        bufio.(*Reader).fill+0x116              /usr/local/go/src/bufio/bufio.go:97
#       0x5068b6        bufio.(*Reader).Peek+0x66               /usr/local/go/src/bufio/bufio.go:129
#       0x77c1a5        net/http.(*persistConn).readLoop+0x195  /usr/local/go/src/net/http/transport.go:1474

2 @ 0x42cd8a 0x43c184 0x43adec 0x77ddfa 0x459ee1
#       0x77ddf9        net/http.(*persistConn).writeLoop+0x439 /usr/local/go/src/net/http/transport.go:1704
    • Tomasz Maczukin
      Tomasz Maczukin @tmaczukin ·
      Maintainer

      So even with -s -w we still have file names and line numbers in both stack trace and pprof.

    • Tomasz Maczukin changed milestone to %9.5

      changed milestone to %9.5

    • Tomasz Maczukin mentioned in merge request !643 (merged)

      mentioned in merge request !643 (merged)

    • Nick Thomas closed via merge request !643 (merged)

      closed via merge request !643 (merged)

    Please register or sign in to reply