Nightly builds using wrong version prefix

Looking at https://packages.gitlab.com/gitlab/nightly-builds, I see filenames such as gitlab-ce_8.0.1+XXX. Should these be on 8.0.4 now? Where does the nightly build get this version? VERSION is set to 8.1.0.pre in master, and 8.0.4 in 8-0-stable.

This has a side effect of looking like a downgrade:

dpkg: warning: downgrading gitlab-ce from 8.0.3-ce.1 to 8.0.1+20151001013044.git.98.9757575-ce.1

When we build the package we build it out of stable branch and a tag on it.

The problem here is that the commits on master are not tied to any tag and git describe then defaults to describing the relation of the latest commit to the latest git tag it can find, it forms the string in the form of MOST_RECENT_TAG-COMMITS_SINCE-gGIT_SHA.

Omnibus-gitlab doesn't know about VERSION file of GitLab-CE, that would require a separate clone of the CE repo and parsing of the VERSION file during the steps before build is triggered.

We could consider creating a VERSION file for the omnibus-gitlab repository which we would update the same way we do with the CE repo and then build the version based on the contents of that file. I will play around a bit with it to see how that could work.

mentioned in issue #1007

mentioned in merge request !578 (closed)

Added release label

Added Enhancement label

mentioned in issue #1500 (closed)

mentioned in merge request !1611 (merged)

mentioned in issue gitlab-com/infrastructure#1944

I think we can resurrect this issue. If we reduce the scope of the package naming change, we can get away with a change.

My proposal is the following:

1. Do not touch stable releases naming conventions (tag) or RC's (tag)
2. Do the change only for nightlies (branch) and possibly just all branch builds

Change on branch:

• We could reuse the latest_tag sorting we do for Docker images to always get the latest released version.
• We can then do some processing to make sure that any nightly package is considered an upgrade: 9.4.2-nightly+TIMESTAMP.COMMITS.PIPELINE_ID . We don't need to do anything special with edition because package will be named gitlab-ee/ce.

And because this will be on non stable release, we don't have to take special care about differences between RHEL and Debian, we can optimise it only for the OS we currently care about.

One of the problems with using nightly is that, it is smaller than rc. We need nightlies to be the largest. Maybe use a different term, say recent or release or something that will be greater than rc. Or even add a z prefix.

$ dpkg --compare-versions 9.4.0-nightly+1234.5678.9865.ce.0 ge 9.4.0+rc3.ce.0
$ echo $?
1

Good point. We could say rnightly as in release nightly. And we should definitely exclude .ce.0 from the package version.

mentioned in issue #1625

Yay ANSI-C ordering

Or znightly as pronounced, "zee nightly build". :)

added Scheduled label

changed milestone to %10.0

mentioned in merge request !1835 (merged)

@marin The problem with using timestamp in the version is that, we are no longer to compute the package file name in Docker build step as it will be different due to difference in timestamp. Then, we have two options

1. Pass the package as an artifact to the Docker build job. But, we will not be using this package; we just need the name of the file to compute the URL. So this "may" be a wasted resource. Or,
2. Access the trace file of the Ubuntu 16.04 job via API and parse it to get the download URL

Considering we don't push these packages to packagecloud repo, we won't be having any chance of package collision. So I don't think we need to even consider timestamp. IMO, we can go with +nightly.<pipeline id>.<current commit sha>. Pipeline id will only be increasing, which will take care of newer builds having later version strings. WDYT @marin ?

assigned to @balasankarc

@balasankarc Nightlies are pushed to the package repository, or is that not what you meant?

Pipeline id is fine as a replacement of timestamp but how will that solve the issue with the Docker build?

@marin Let's take this discussion to https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1835#note_37070857.

To answer your query, pipeline ID is same for the whole pipeline and will not differ between stages. So, Docker stage can compute it. So it won't need artifacts to be passed.

Regarding nightlies being pushed to the package repo: Package collisions often occur when a successful job is being retried. We rarely do that for nightlies. Actually, we are not even using timestamps now so if it were to be an issue, it would've been one already.

closed via merge request !1835 (merged)

mentioned in commit 4dc2f3e2