Skip to content

Assign DRI to failure issues

George Koltsov requested to merge am-assign-dri-to-failures into master

What does this MR do and why?

Describe in detail what your merge request does and why.

Create a mechanism to assign SET counterparts and due dates on failure issue creation. Part of https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/1284

Please note that currently only Source Code tests have a product_group label assigned so after this MR is merged just that group will start getting assignments and will serve as a test subject. If all is successful, we will be rolling out other labels in batches to ensure accuracy and limit disruption.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. Create the following rspec-3101390531.json file in your local gitlab-qa repository:
Click to expand 
{
  "examples": [
  {
    "id": "./qa/specs/features/browser_ui/3_create/repository/file/delete_file_via_web_spec.rb[1:1:1]",
    "description": "user deletes a file via the Web",
    "full_description": "Create File management user deletes a file via the Web",
    "status": "failed",
    "file_path": "./qa/specs/features/browser_ui/3_create/repository/file/delete_file_via_web_spec.rb",
    "line_number": 15,
    "run_time": 18.077729056,
    "pending_message": null,
    "testcase": "https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases/347731",
    "quarantine": null,
    "screenshot": null,
    "product_group": "source_code",
    "ci_job_url": "https://gitlab.com/gitlab-org/gitlab/-/jobs/3101390531",
    "exceptions": [
      {
        "class": "RuntimeError",
        "message": "failure to create and assign a new issue for",
        "message_lines": [
          "Failure/Error: raise \"failure to create and assign a new issue for\"",
          "",
          "RuntimeError:",
          "  failure to create and assign a new issue for"
        ],
        "backtrace": [
          "./qa/specs/features/browser_ui/3_create/repository/file/delete_file_via_web_spec.rb:16:in `block (3 levels) in <module:QA>'",
          "./qa/specs/spec_helper.rb:149:in `block (2 levels) in <top (required)>'",
          "./qa/specs/knapsack_runner.rb:15:in `run'",
          "./qa/specs/runner.rb:55:in `perform'",
          "./qa/scenario/template.rb:10:in `block in perform'",
          "./qa/scenario/template.rb:8:in `tap'",
          "./qa/scenario/template.rb:8:in `perform'",
          "./qa/scenario/template.rb:50:in `perform'",
          "./qa/scenario/template.rb:10:in `block in perform'",
          "./qa/scenario/template.rb:8:in `tap'",
          "./qa/scenario/template.rb:8:in `perform'",
          "./qa/scenario/bootable.rb:51:in `launch!'"
        ]
      }
    ]
  }
]
}

  1. Hardcode pipeline_name_label value in report_as_issue.rb as the script won't be able to retrieve it otherwise.

  2. Run the following command:

GITLAB_QA_ACCESS_TOKEN="your-token" bundle exec gitlab-qa-report --relate-failure-issue "rspec-3101390531.json" --project "gitlab-org/gitlab" --max-diff-ratio "0.15"

NB This will create a failure issue in gitlab-org/gitlab project and assign it to Source Code SET (@a_mcdonald) with a due date. Please close the issue after testing is complete.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by George Koltsov

Merge request reports