Allow gitlab-runner exec to build uncommited working tree

Hey Guys Right now I have to commit before i can to local build with gitlab-runner exec. The point of exec is to quickly compile and try out thing. It would be nice if we can have flag which allow exec to send uncommitted workingtree to be sent to build. If thr is already this option, that I might overlooked plz let me know. I would even argue that this should be default behaviour.

Jas

@ayufan @tmaczukin

Changed title: Allow gitlab-runner exec to build uncommit working tree → Allow gitlab-runner exec to build uncommited working tree

@ayufan @tmaczukin

Hey guys

Since you didn't reply yet; I was wondering if I was clear enough in my question/feature_request; I would be happy to rephrase it.

Thx

Sorry, yes we understand that, but this is not possible currently. It's due to the fact that we do a normal git clone inside a created container so it's impossible to use a working directory changes.

Given that it's not possible to easily change that :(

Added feature proposal label

Added wontfix label

Status changed to closed

Milestone changed to %Backlog

@jas99 I'm curious why you want to run tests locally with exec? Why don't just execute them normally - however it is done with the technology you're using :)

Do you want to test CI configuration or do you want to test your application? For me usage of exec is more to check if my .gitlab-ci.yml and all build scripts are prepared properly, not for running application tests.

For example - in GitLab CE project we have our CI configuration and we have a full tests suite. I would use gitlab-runner exec to check if my changes in .gitlab-ci.yml are working properly. But for that I need only to do changes in this one file even without commiting. If I execute some build scripts in job definition (for example some ./ci/prepare), then I would commit only those files to check if they are executed properly when running the job.

But if I want to run tests for a new GitLab CE feature that I'm working - which is the day-to-day task done by the developer - currently I just execute bundle exec rspec spec/features/my_feature_spec.rb or bundle exec rake spinach or else. There is no need to use gitlab-runner exec ... here. As a developer you have the full technology stack on your computer that is able to execute tests from this application.

As @ayufan wrote - when using exec we are doing a full git clone inside of the job. And in my opinion this is very important to leave this as it is. With exec we are simulating the normal Runner's behavior - we are just setting git source to a local path, but rest is done like in a normal build:

• there is a "fake", predefined GetBuildResponse structure created with all important data,
• the RepoURL field is set to local git path (e.g. /home/projects/my-project/.git)
• the runner is used as in "normal" situation, only this predefined structure is set instead of data received from GitLab.

Thanks to this you can test you build behavior without pushing changes to GitLab, registering the Runner in the project etc.

Hey @tmaczukin In my setup I am more interested testing my application. From testing I just mean running my application; not running the test suite. I have my build script written in .gitlab-ci.yml itself and like it that way. I have python script which further calls gitlab-runner exec with various configuration.
Below attached files might give you little glimpse how I am setup.
.gitlab-ci.yml
gitlab-ci.py

Hey @ayufan
I completely understood and knew about the clone argument.
Let me propose an approach which I will use in my script in case you reject my proposal (which is totally cool )

Before running clone

• stash working tree
• create temporary branch with name of real branch but with some suffix or prefix
• pop stash on real branch
• clone and delete temporary branch

This guess that will do it

Thx Jas

i valid workaround for me testing the ci stuff before pushing to gitlab itself:

git commit ...

Then test

git commit --amend ...

iteration and amend again if it fails ...

then you have the final commit at the end and everything is fine ... but please do not push to gitlab before last amend

Do you want to test CI configuration or do you want to test your application? For me usage of exec is more to check if my .gitlab-ci.yml and all build scripts are prepared properly, not for running application tests.

Sometimes something fails in CI, you run it locally via the "normal" method and it doesn't fail - being able to quickly iterate over changes and execute the tests with exec is often necessary here to figure out what's different when executing under GitLab CI.

@kaystrobach This is the only work around at the moment, but I wouldn't call it valid. It adds a (seemingly unnecessary) step to each iteration and creates a decent risk of someone pushing a commit full of debug and/or broken code.

I'm curious why you want to run tests locally with exec? Why don't just execute them normally - however it is done with the technology you're using :)

I don't know how people even work locally, seems i the only one in the internet who still trying to work on localhost ...

I need this feature to testing my Dockerfile(s) and other non-gitlab configs before commiting them into repo.

For example: Making some changes to Dockerfile, now i need to test it (build and run). To test my changes i need to build it like it will build for testing stage (in gitlab/CI). So i need to execute something like gitlab-runner exec shell build_images and gitlab-runner exec shell run_images, so it will build and run all tasks locally. If i see error - i change configs/dockerfiles/startup scrips/whatever and rerun build + run. Now i have tested my changes - i commit it to repo, knowing it works.

If i need to commit any changes - it takes some time and add some complexity to my workflow, more interaction with git... Why cant i just gitlab-runner exec ... in my workdir and have result? What i am doing wrong?

@caravus agree on that, esp. with complex pipelines localtesting can be a pain in the ass.

my current workaround is having a .gitlab folder with shell scripts, which are then also used in the .gitlab-ci.yml file. but this is kind of a weird workaround.