Detect (and time) new tests introduced by a feature
What
System to find new tests introduced by an MR.
We can then run these tests in their own job and time how long they take.
Probably just for the gitlab-ce/ee projects rather than as a generic feature.
Why
- Provide data so we realize when an MR is introducing a large amount of additional test time
- We might be able to use this to spot tests written at the wrong level, e.g. as long running integration tests instead of unit tests
- We can use this to run new tests in isolation so we can spot failures quicker instead of waiting for all tests to complete
Ideas how
rspec --format json --color --dry-run | jq '.examples[] | .full_description'
- Could get list of names of tests before/after an MR and have special job to run only new tests
- Could store metadata for which tests exist for a given commit, or could checkout previous commit and run
Edited by James EJ