allow variable substitution in CI tag names

Description

I wrote the following .gitlab-ci.yml:

deploy:
  stage: deploy
  environment:
    name: $CI_BUILD_REF_NAME
  only:
    - /^env-.*$/
  tags:
    - deploy
    - $CI_BUILD_REF_NAME

I was expecting a deploy to run on a runner with a tag name the same as the name of my branch (to have the proper environment's deployment variables securely injected).

Instead, I got a hung build where it was looking for a runner with the tag $CI_BUILD_REF_NAME.

Proposal

When evaluating a .gitlab-ci.yml file to schedule a job, first perform variable substitution prior to the logic used for selecting a runner.

Additionally consider implementing regex-style variable manipulation as proposed in gitlab-foss#24044 (closed). These two improvements combined would be especially powerful.

Hi @DanielDent. I think that your proposal is covered by the following issue:

• Add support for variable expansion - https://gitlab.com/gitlab-org/gitlab-ce/issues/18758

Please could you take a look?

Added ~65372 label

@markglenfletcher This looks like a related but different issue.

Note that $CI_BUILD_REF_NAME is a variable injected by the gitlab CI process, and my concern is how it gets used by the component of gitlab which decides where a job is scheduled. gitlab-foss#18758 (moved) appears to be about what happens at the runner level with a scheduled job. It's in fact already possible to use variables in execution scripts and I'm unclear exactly what the user in gitlab-foss#18758 (moved) is trying to achieve that can't already be done.

Added 19173 gitlab-foss14213 and removed ~65372 labels

mentioned in issue gitlab-foss#30819 (moved)

added ci variables in GitLab FOSS label

I'd also like to see support for job tags variable expansion.

In our use case, each person on the project has their own separate runner. I was hoping to use the $GITLAB_USER_EMAIL variable in a job tag to know which committer's runner to use based on the user email. I think there's other ways around this with some conditional scripts but I did find this proposal interesting/useful.

https://gitlab.zendesk.com/agent/tickets/78493 (GitLab internal only)

Yes we are looking for a similar feature , where we can do variable substitution (not only limited to inbuilt gitlab variables but user defined variables) for TAGS & ENVIRONMENT section of gitlab CI. It will be a really handy feature & you can create a dynamic pipeline using this feature.

added customer in GitLab FOSS label

@bikebilly Are you able to assess this request?

@bikebilly My problem here is that we need to sit to the drawing table and group variables into ones that could be used and not.

Some of the variables that we have, are dynamic (job id), some of them are static (job name). If we would have to evaluate dynamic ones in that context it creates a lot of potential problems.

If we figure for what uses cases we could use only: with variables it would allow us to better understand if this is something that we could do.

It is hard to estimate, it depends on what variables would have to be supported. Fortunately, it would allow us to clean up port of configuration processing, so I expect it to be Medium-sized task.

@bikebilly Feel free to remove the label

moved from gitlab-foss#24207 (moved)