Skip to content
Snippets Groups Projects

Reduce binary size with removing debugging symbols

Merged Tomasz Maczukin requested to merge 2616-reduce-binary-size into master
All threads resolved!

What does this MR do?

Reduces binary size by ~28% by removing debugging symbols.

Why was this MR needed?

Please read #2616 (closed) for a reference. And here - https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/2616#note_36662389 - we have a comparison of two versions - with and without debugging symbols.

Are there points in the code the reviewer needs to double check?

Does this MR meet the acceptance criteria?

  • Documentation created/updated
  • Tests
    • Added for this feature/bug
    • All builds are passing
  • Branch has no merge conflicts with master (if you do - rebase it please)

What are the relevant issue numbers?

Closes #2616 (closed)

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • @tmaczukin LGTM, but what do you think to the UPX option described in https://blog.filippo.io/shrink-your-go-binaries-with-this-one-weird-trick/ in addition? does it save us anything, considering the bulk of our binary size is go-bindata embedded docker images?

  • assigned to @tmaczukin

  • Nick Thomas resolved all discussions

    resolved all discussions

  • Author Maintainer

    @nick.thomas UPX looks interesting and it makes binaries much smaller (more than 50% of initial size) even with embedded docker images in executors/docker/bindata.go. However there is one problem - it doesn't work for freebsd/amd64 and freebsd/arm:

    $ for name in gitlab-ci-multi-runner-*; do size_after=$(du -b $name | awk '{print $1}'); size_before=$(du -b original-$name | awk '{print $1}'); final_size_ratio=$(echo "scale=2; $size_after/$size_before" | bc); echo -e "$name \t $size_after : $size_before >> $final_size_ratio"; done
    
    gitlab-ci-multi-runner-darwin-386           27570176 : 61497084 >> .44
    gitlab-ci-multi-runner-darwin-amd64         27987968 : 68213216 >> .41
    gitlab-ci-multi-runner-freebsd-386          27558896 : 61576286 >> .44
    gitlab-ci-multi-runner-freebsd-amd64        68442317 : 68442317 >> 1.00
    gitlab-ci-multi-runner-freebsd-arm          62929380 : 62929380 >> 1.00
    gitlab-ci-multi-runner-linux-386            27587284 : 61924835 >> .44
    gitlab-ci-multi-runner-linux-amd64          28004596 : 68629175 >> .40
    gitlab-ci-multi-runner-linux-arm            26783972 : 63015648 >> .42
    gitlab-ci-multi-runner-windows-386.exe      27010560 : 62339584 >> .43
    gitlab-ci-multi-runner-windows-amd64.exe    27982848 : 68946944 >> .40

    Also it adds a big slowdown for process start:

    $ time ./gitlab-ci-multi-runner-linux-amd64 --version >/dev/null
    
    real    0m1,900s
    user    0m1,856s
    sys     0m0,048s
    
    $ time for i in {1..10}; do ./gitlab-ci-multi-runner-linux-amd64 --version >/dev/null; done
    
    real    0m18,870s
    user    0m18,536s
    sys     0m0,300s
    
    $ time ./original-gitlab-ci-multi-runner-linux-amd64 --version >/dev/null
    
    real    0m0,034s
    user    0m0,036s
    sys     0m0,004s
    
    $ time for i in {1..10}; do ./original-gitlab-ci-multi-runner-linux-amd64 --version >/dev/null; done
    
    real    0m0,131s
    user    0m0,116s
    sys     0m0,032s

    So it's 55 times more for one execution and 140 times more for multiple executions (in comparison to execution of binary without UPX compression). From what I've tested the it's slower only on process start (the binary needs to be decompressed) and after this it works as usual, but still - one could expect that Runner after calling gitlab-runner run will just work. Also in this case we're saving disk space but the cost is taken on additional CPU usage while generally disk space is cheaper than CPU.

    The last thing - I've checked quickly how linux/amd64 binary works after compression with UPX and it looks good. But I have no idea how it works on other platforms that we support.

    So as a summary:

    1. We could use UPX (so after ~25% less thanks to -ldflags "-s -w" we will have another ~50-60% less), but UPX will not handle all platforms. Which IMO is not a problem.
    2. Before we decide to do it, we need to decide if we're OK with slowdown on start.
    3. If we will decide that we're OK also with the above, then we need to check if binaries for all officially supported platforms are working properly after UPX compression.
  • @tmaczukin OK, let's merge this as-is :thumbsup:

  • Nick Thomas approved this merge request

    approved this merge request

  • merged

  • Please register or sign in to reply
    Loading