Make `—order random` more deterministic.
Created by: myronmarston
When you’re troubleshooting an order dependent failure, you want to get the repro case down to a minimal run that loads and runs as few specs as possible. With the old random ordering implementation, that was hard to achieve because while rerunning with a given seed produced the same order when the exact same set of examples were loaded, the ordering would be completely different when a subset was loaded.
By ordering by md5(seed + example_id)
it ensures
that the ordering of any two examples should stay
consistent regardless of how many other examples are
loaded.
MD5 is significantly slower than shuffle
, and it
loads an additional part of our stdlib (which we
want to minimize) but I think the tradeoff is worth it
here.
This will help us be able to implement the --bisect
option discussed in #1767 (closed).
What do others think? Is the tradeoff worth it? Is there some other hashing function we should consider using? If there was a hashing function that had similar distribution characteristics to MD5 and was faster and easy to use w/o loading part of the stdlib, that would be ideal, but I can't think of one.
/cc @rspec/rspec