Skip to content

Add 'reproduction-order' flag for printing spec order in summary

Created by: maschwenk

When running specs using the random ordering, it can often be useful to see the example order that the seed generated.

This can be useful when you are doing a bisect over a large number of failing specs yet you know the first failing example should allow you to reproduce the order-dependent failure with smaller bisect-able bundle. With a large amount of failures, bisect tends to churn for hours and hours.

In the past I had used a combination of --fail-fast plus the --format json flag to parse out the actual spec ordering after the random seeding had taken place. The alternative is running in documentation mode to infer which examples ran in which order (kinda terrible).

Other folks have ran into similar issues and their solutions are similarly hacky

I plan to write specs/documentation if this seems like a palatable change.

Example output

rspec spec/speccy_the_spectacle.rb --order random --reproduction-order

Randomized with seed 21471
...

Finished in 0.21412 seconds (files took 7.1 seconds to load)
3 examples, 0 failures

Reproduce this spec order with:
rspec --seed ordered ./spec/speccy_the_spectacle.rb:50 ./spec/speccy_the_spectacle.rb:10 ./spec/speccy_the_spectacle.rb:20

Merge request reports