Dynamic environment tear down artifacts

Description

Regarding dynamic environments/review app destruction. Currently all teardown has to happen via the yml script array since the branch had been deleted.

Its problematic for complex environments such as Amazon when your environment might consist of ec2 instances, iam entries, security groups, route53 entries, secrets etc..

ideally it would be helpful if we could access some scripts to handle teardown, currently turning the git strategy to none because the branch had been deleted prevents access to scripts in the repo

Proposal

Artifacts seem to be a good solution, we could upload the teardown script as an artifact in the deploy stage, then access it on the stop environment stage.

I thought that artifacts might work like this already but I haven't been able to get the configuration to work, I'm guessing that disabling the git checkout is affecting artifact download?

moved from gitlab-ci-multi-runner#2079

added ~19173 label

/cc @grzesiek

added 147988 gitlab-foss14213 labels

found a working solution for the time being, the teardown stage needs to be in a separate stage

stages:
  - review
  - teardown

deploy:review:
  stage: review
  artifacts:
    name: "$CI_BUILD_REF_NAME"
    paths:
      - scripts/destroy_dynamic_env.sh
  script:
  - source ./scripts/create_dynamic_env.sh
  - docker-compose pull
  - docker-compose up -d
  when: always
  environment:
    name: review/$CI_BUILD_REF_NAME
    url: https://$CI_BUILD_REF_NAME.example.com
    on_stop: auto:teardown
  tags:
    - docker
  only:
    - branches
  except:
    - master

auto:teardown:
  stage: teardown
  variables:
    GIT_STRATEGY: none
  script:
    - echo "Remove review app"
    - source ./scripts/destroy_dynamic_env.sh
  when: manual
  environment:
    name: review/$CI_BUILD_REF_NAME
    action: stop
  dependencies:
    - deploy:review

@gkeys I agree that GIT_STRATEGY=none is really problematic in this case. This is a known drawback! We plan to improve that in the future.

/cc @ayufan

added deploy in GitLab FOSS and removed ~147988 labels

I also have the same issue in that I would like to be able to use code in the repo to teardown my environment (in my case I'm using Terraform to spin up and tear down S3 buckets).

I was thinking of trying exactly what @gkeys did but the docs seem to suggest that that won't work for auto stopping an environment:

When you have an environment that has a stop action defined (typically when the environment describes a review app), GitLab will automatically trigger a stop action when the associated branch is deleted. The stop_review job must be in the same stage as the deploy_review one in order for the environment to automatically stop.

Is that the case?

An easy fix for me would be for there to be a GIT_STRATEGY: master that fetched master but still keeping the CI_COMMIT_REF_SLUG etc variables for the branch that is being stopped. Then I would need to have the teardown scripts on master which isn't 100% perfect but does solve this slightly more elegantly.

EDIT:

Looking at https://docs.gitlab.com/ee/ci/yaml/#git-checkout it looks like I might be able to get away with something like:

variables:
  GIT_STRATEGY: clone
  GIT_CHECKOUT: false
before_script:
  - git checkout master
script:
  - make destroy

Which should force it to clone the full repo and then checkout master only. Is that a better way of achieving this for now? It's a bit ugly but I'm not too worried about that.

EDIT2: Using the above snippet fails (at least for 9.1.2) with:

Error: jobs:stop_review:variables config should be a hash of key value pairs

Despite the docs showing that exact syntax. Wrapping false in quotes (GIT_CHECKOUT: "false") passes the linter. Do the docs need updating or is that fine in later versions of Gitlab (CI)?

moved from gitlab-foss#27207 (moved)