Fix flaky examples tracking

EE MR: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3065

What does this MR do?

From the commits:

Ensure RSpecFlaky doesn't automatically update flaky examples

Previously, instantiating a RspecFlaky::FlakyExample object would automatically update its first_flaky_at, last_flaky_at and last_flaky_job (even if the example was not flaky in the current spec run!).

That was wrong because we would overwrite every time the suite report with this false data.

We now:

• Get the suite report and only read from it
• Write only the currently detected flaky examples in the report, so that the final report is only updated with flaky examples that were actually detected in each job. Before, job1 could overwrite the legit report from job2!
• Write the newly detected flaky examples by rejecting the already tracked flaky specs instead of using another hash.

Are there points in the code the reviewer needs to double check?

I've improved the specs and added more specs so everything should be better.

I've changed the name of the reports so that the detection will start from scratch since we cannot trust the data we currently have in report-master.json.

Why was this MR needed?

Because, the first_flaky_at, last_flaky_at and last_flaky_job in the report-master.json file are not accurate.

Does this MR meet the acceptance criteria?

• Tests added for this feature/bug
• Review
  • Has been reviewed by Backend
• Conform by the merge request performance guides
• Conform by the style guides
• Squashed related commits together

changed the description

assigned to @grzesiek

@rymai I left just a single question! I think this is getting quite complex, and we can benefit from thinking about extracting some classes.

assigned to @rymai

assigned to @grzesiek

resolved all discussions

@rymai lib/rspec_flaky/flaky_examples_collection.rb:5:1: C: Layout/EmptyLinesAroundClassBody: Extra empty line detected at class body beginning.

added 1 commit

• da5aa64f - Add new RSpecFlaky::FlakyExamplesCollection and RSpecFlaky::Config classes

Compare with previous version

added 1 commit

• da5aa64f - Add new RSpecFlaky::FlakyExamplesCollection and RSpecFlaky::Config classes

Compare with previous version

marked the checklist item Has been reviewed by Backend as completed

approved this merge request

merged

mentioned in commit 05054803

Thanks @rymai! Let's try it!