Skip to content

Skip reporting Net::ReadTimeout failures

George Koltsov requested to merge ml-skip-reporting-transient-failures into master

What does this MR do and why?

Currently we open a failure issue for any failure in monitored pipelines. This change makes it not open an issue if the failure is Net::ReadTimeout because in most cases that's a transient failure that we can't (or don't need to) do anything about.

This shouldn't create a risk that we miss a real problem because if a lot of tests fail with the same reason, or one test fails consistently, we'll see that in the pipelines themselves, or in test session reports.

Closes https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1283

Screenshots or screen recordings

↳ ./exe/gitlab-qa-report --dry-run --report-results failures.json --results-issue-project=gitlab-org/quality/testcases --test-case-project=gitlab-org/gitlab

Reporting test results in `failures.json` as test cases in project `gitlab-org/gitlab` and issues in project `gitlab-org/quality/testcases` via the API at `https://gitlab.com/api/v4`.
Reporting tests in failures.json
Reporting test: ./qa/specs/features/browser_ui/1_manage/project/import_github_repo_spec.rb | Manage Project import imports a GitHub repo
Issue update skipped because the error was `Net::ReadTimeout`

How to set up and validate locally

Run the command above where the contents of failures.json is:

{
  "examples": [
    {
      "id": "./qa/specs/features/browser_ui/1_manage/project/import_github_repo_spec.rb[1:1:1]",
      "description": "imports a GitHub repo",
      "full_description": "Manage Project import imports a GitHub repo",
      "status": "failed",
      "file_path": "./qa/specs/features/browser_ui/1_manage/project/import_github_repo_spec.rb",
      "exceptions": [
        {
          "class": "Net::ReadTimeout",
          "message": "",
          "message_lines": [],
          "backtrace": []
        }
      ]
    }
  ]
}

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by George Koltsov

Merge request reports