Skip to content

rspec command exits with non zero status, 0 failures

gitlab-qa-bot requested to merge correct_rake_task_exit_code into master

Created by: JonRowe

When running our work projects test suite using rspec, the process sometimes (not always) will return an exit status of 1, even though there was no failures, the output on CI looks like:

Finished in 1 minute 44.48 seconds
2237 examples, 0 failures, 2 pending

Randomized with seed 63250

RSpec Finished with status: 1 # this is printing $?

The command being run is: ruby -S rspec spec/ --failure-exit-code 2

Rspec is version: 2.14.7, ruby 2.0.0p353

I've put some logging around the at_exit method to see if any processes were fiddling with the status, I'm printing the status before and after the at_exit is run, the only registered hook is sinatra, this one here: https://github.com/sinatra/sinatra/blob/master/lib/sinatra/main.rb#L25 but i'm logging the exit status before and after the block is executing and it's a 0 both before and after.

I'm also logging all calls to abort exit and exit! and none of them seem to be getting called causing the exit.

I'm not sure where else to look to hunt down the issue? It's hard to reproduce as running a subset of the tests is always fine and it happens on some machines more often than others.

The only other way I can think that the script would return a status of 1 is if there were an exception thrown, but there's no output to indicate that's happening (although it could be). Maybe there's a way I could log if there's an uncaught exception to check?

Merge request reports