Review apps as a GitLab Runner jobs

We have great auto-scaling mechanism which seems to be great to not only build and test our projects, but also to have a review apps a disposable, short living, pre-seed versions of applications that you can test live.

My proposal is to make GitLab Runner to be able to expose ports if possible on used executors and callback with an user accessible URL to each of the mappings.

GitLab and GitLab Runner would be responsible for stopping a running app after some period of time defined by the timeout. We already have this mechanism implemented as cancel. GitLab Runner already can auto-scale infrastructure, so it would allow us to pretty much server unlimited number of review apps and provision new machines for running regular builds. This does work well with integrated docker and docker-machine executor and planned kubernetes executor.

The responsibly of a developer would be to add a additional job to .gitlab-ci.yml:

review-app:
  script:
  - bin/rails server -p 8080
  ports:
  - 8080
  timeout: 30 minutes

With recently added manual actions this allows us to have a review apps to be automatically started or be manually executed with a manually defined action. This makes it possible for example on Merge Request page to have button Run this Merge Request.

All technology stack is already there. We only require some improvements to our configuration (adding a ports), a call home mechanism to know on what URL our application is served and a place where we can expose this information. Everything else is already done as part of the CI. The lifecycle of review apps would be managed by GitLab in a way how we currently manage the builds (or actually jobs).

People would still have they own deployment procedures for staging and production environments, but this would be great help for having a small testable applications running quickly somewhere.

@JobV @sytses @markpundsack @grzesiek @tmaczukin What do you think?