Skip to content
Snippets Groups Projects
Unverified Commit 3d5a1ea0 authored by Tiffany Rea's avatar Tiffany Rea Committed by GitLab
Browse files

Merge branch 'q1685-documentation-organization' into 'master' 

[Doc Reorg] Updates beginner's guide to include references to additional key concepts

See merge request https://gitlab.com/gitlab-org/gitlab/-/merge_requests/167992



Merged-by: default avatarTiffany Rea <trea@gitlab.com>
Approved-by: default avatarVishal Patel <vpatel@gitlab.com>
Approved-by: default avatarTiffany Rea <trea@gitlab.com>
Co-authored-by: default avatarRichard Chong <rchong@gitlab.com>
Co-authored-by: default avatarTomas Bulva <tbulva@gitlab.com>
parents 09460027 633eea24
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 1253 additions and 1239 deletions
doc/development/adding_service_component.md
+ 1
− 1
View file @ 3d5a1ea0
@@ -19,7 +19,7 @@ The following outline re-uses the [maturity metric](https://handbook.gitlab.com/
- [Handling service dependencies](#handling-service-dependencies)
- Viable
- [Bundled with GitLab installations](#bundling-a-service-with-gitlab)
- [End-to-end testing in GitLab QA](testing_guide/end_to_end/beginners_guide.md)
- [End-to-end testing in GitLab QA](testing_guide/end_to_end/beginners_guide/index.md)
- [Release management](#release-management)
- [Enabled on GitLab.com](feature_flags/controls.md#enabling-a-feature-for-gitlabcom)
- Complete
doc/development/testing_guide/end_to_end/beginners_guide.md
+ 8
− 367
View file @ 3d5a1ea0
---
stage: none
group: unassigned
info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review.
redirect_to: 'beginners_guide/index.md'
remove_date: '2025-01-03'
---
<!-- markdownlint-disable -->
This document was moved to [another location](beginners_guide/index.md).
 
# Beginner's guide to writing end-to-end tests
This tutorial walks you through the creation of end-to-end (_e2e_) tests
for [GitLab Community Edition](https://about.gitlab.com/install/?version=ce) and
[GitLab Enterprise Edition](https://about.gitlab.com/install/).
By the end of this tutorial, you can:
- Determine whether an end-to-end test is needed.
- Understand the directory structure within `qa/`.
- Write a basic end-to-end test that validates login features.
- Develop any missing [page object](page_objects.md) libraries.
## Before you write a test
Before you write tests, your
[GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit)
must be configured to run the specs. The end-to-end tests:
- Are contained within the `qa/` directory.
- Should be independent and
[idempotent](https://en.wikipedia.org/wiki/Idempotence#Computer_science_meaning).
- Create [resources](resources.md) (such as project, issue, user) on an ad-hoc basis.
- Test the UI and API interfaces, and use the API to efficiently set up the UI tests.
NOTE:
For more information, see [End-to-end testing Best Practices](best_practices.md).
## Determine if end-to-end tests are needed
Check the code coverage of a specific feature before writing end-to-end tests
for the [GitLab](https://gitlab-org.gitlab.io/gitlab/coverage-ruby/#_AllFiles) project.
Does sufficient test coverage exist at the unit, feature, or integration levels?
If you answered *yes*, then you *don't* need an end-to-end test.
For information about the distribution of tests per level in GitLab, see
[Testing Levels](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/testing_guide/testing_levels.md).
- See the
[How to test at the correct level?](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/testing_guide/testing_levels.md#how-to-test-at-the-correct-level)
section of the [Testing levels](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/testing_guide/testing_levels.md) document.
- Review how often the feature changes. Stable features that don't change very often
might not be worth covering with end-to-end tests if they are already covered
in lower level tests.
- Finally, discuss the proposed test with the developers involved in implementing
the feature and the lower-level tests.
WARNING:
Check the [GitLab](https://gitlab-org.gitlab.io/gitlab/coverage-ruby/#_AllFiles) coverage project
for previously written tests for this feature. To analyze code coverage,
you must understand which application files implement specific features.
In this tutorial we're writing a login end-to-end test, even though it has been
sufficiently covered by lower-level testing, because it's the first step for most
end-to-end flows, and is easiest to understand.
## Identify the DevOps stage
The GitLab QA end-to-end tests are organized by the different
[stages in the DevOps lifecycle](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/specs/features/browser_ui).
Determine where the test should be placed by
[stage](https://handbook.gitlab.com/handbook/product/categories/#devops-stages),
determine which feature the test belongs to, and then place it in a subdirectory
under the stage.
![DevOps lifecycle by stages](img/gl-devops-lifecycle-by-stage_v12_10.png)
If the test is Enterprise Edition only, the test is created in the `features/ee`
directory, but follow the same DevOps lifecycle format.
## Create a skeleton test
In the first part of this tutorial we are testing login, which is owned by the
Manage stage. Inside `qa/specs/features/browser_ui/1_manage/login`, create a
file `basic_login_spec.rb`.
### The outer `context` block
See the [`RSpec.describe` outer block](#the-outer-rspecdescribe-block)
WARNING:
The outer `context` [was deprecated](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/550) in `13.2`
in adherence to RSpec 4.0 specifications. Use `RSpec.describe` instead.
### The outer `RSpec.describe` block
Specs have an outer `RSpec.describe` indicating the DevOps stage.
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Manage' do
end
end
```
### The `describe` block
Inside of our outer `RSpec.describe`, describe the feature to test. In this case, `Login`.
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Manage' do
describe 'Login' do
end
end
end
```
### The `product_group` metadata
Assign `product_group` metadata and specify what product group this test belongs to. In this case, `authentication_and_authorization`.
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Manage' do
describe 'Login', product_group: :authentication do
end
end
end
```
### The `it` blocks (examples)
Every test suite contains at least one `it` block (example). A good way to start
writing end-to-end tests is to write test case descriptions as `it` blocks:
```ruby
module QA
RSpec.describe 'Manage' do
describe 'Login', product_group: :authentication do
it 'can login' do
end
it 'can logout' do
end
end
end
end
```
## Write the test
An important question is "What do we test?" and even more importantly, "How do we test?"
Begin by logging in.
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Manage' do
describe 'Login', product_group: :authentication do
it 'can login' do
Flow::Login.sign_in
end
it 'can logout' do
Flow::Login.sign_in
end
end
end
end
```
After [running the spec](#run-the-spec), our test should login and end; then we
should answer the question "What do we test?"
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Manage' do
describe 'Login', product_group: :authentication do
it 'can login' do
Flow::Login.sign_in
Page::Main::Menu.perform do |menu|
expect(menu).to be_signed_in
end
end
it 'can logout' do
Flow::Login.sign_in
Page::Main::Menu.perform do |menu|
menu.sign_out
expect(menu).not_to be_signed_in
end
end
end
end
end
```
**What do we test?**
1. Can we sign in?
1. Can we sign out?
**How do we test?**
1. Check if the user avatar appears in the left sidebar.
1. Check if the user avatar *does not* appear in the left sidebar.
Behind the scenes, `be_signed_in` is a
[predicate matcher](https://rspec.info/features/3-12/rspec-expectations/built-in-matchers/predicates/)
that [implements checking the user avatar](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/page/main/menu.rb#L92).
## De-duplicate your code
Refactor your test to use a `before` block for test setup, since it's duplicating
a call to `sign_in`.
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Manage' do
describe 'Login', product_group: :authentication do
before do
Flow::Login.sign_in
end
it 'can login' do
Page::Main::Menu.perform do |menu|
expect(menu).to be_signed_in
end
end
it 'can logout' do
Page::Main::Menu.perform do |menu|
menu.sign_out
expect(menu).not_to be_signed_in
end
end
end
end
end
```
The `before` block is essentially a `before(:each)` and is run before each example,
ensuring we now sign in at the beginning of each test.
## Test setup using resources and page objects
Next, let's test something other than Login. Let's test Issues, which are owned by the Plan
stage and the Project Management Group, so [create a file](#identify-the-devops-stage) in
`qa/specs/features/browser_ui/2_plan/issue` called `issues_spec.rb`.
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Plan' do
describe 'Issues', product_group: :project_management do
let(:issue) { create(:issue) }
before do
Flow::Login.sign_in
issue.visit!
end
it 'can close an issue' do
Page::Project::Issue::Show.perform do |show|
show.click_close_issue_button
expect(show).to be_closed
end
end
end
end
end
```
Note the following important points:
- At the start of our example, we are at the `page/issue/show.rb` [page](page_objects.md).
- Our test fabricates only what it needs, when it needs it.
- The issue is fabricated through the API to save time.
- GitLab prefers `let()` over instance variables. See
[best practices](../best_practices.md#subject-and-let-variables).
- `be_closed` is not implemented in `page/project/issue/show.rb` yet, but is
implemented in the next step.
The issue is fabricated as a [Resource](resources.md), which is a GitLab entity
you can create through the UI or API. Other examples include:
- A [Merge Request](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/resource/merge_request.rb).
- A [User](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/resource/user.rb).
- A [Project](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/resource/project.rb).
- A [Group](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/resource/group.rb).
## Write the page object
A [Page Object](page_objects.md) is a class in our suite that represents a page
within GitLab. The **Login** page would be one example. Since our page object for
the **Issue Show** page already exists, add the `closed?` method.
```ruby
module Page::Project::Issue
class Show
view 'app/views/projects/issues/show.html.haml' do
element 'closed-status-box'
end
def closed?
has_element?('closed-status-box')
end
end
end
```
Next, define the element `closed-status-box` within your view, so your Page Object
can see it.
```haml
-#=> app/views/projects/issues/show.html.haml
.issuable-status-box.status-box.status-box-issue-closed{ ..., data: { testid: 'closed-status-box' } }
```
## Run the spec
Before running the spec, make sure that:
- GDK is installed.
- GDK is running locally on port 3000.
- No additional [RSpec metadata tags](rspec_metadata_tests.md) have been applied.
- Your working directory is `qa/` within your GDK GitLab installation.
- Your GitLab instance-level settings are default. If you changed the default settings, some tests might have unexpected results.
- Because the GDK requires a password change on first login, you must include the GDK password for `root` user
To run the spec, run the following command:
```shell
GITLAB_PASSWORD=<GDK root password> bundle exec rspec <test_file>
```
Where `<test_file>` is:
- `qa/specs/features/browser_ui/1_manage/login/log_in_spec.rb` when running the Login example.
- `qa/specs/features/browser_ui/2_plan/issue/create_issue_spec.rb` when running the Issue example.
Additional information on test execution and possible options are described in ["QA framework README"](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/README.md#run-the-end-to-end-tests-in-a-local-development-environment)
## End-to-end test merge request template
When submitting a new end-to-end test, use the ["New End to End Test"](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/merge_request_templates/New%20End%20To%20End%20Test.md)
merge request description template for additional
steps that are required prior a successful merge.
<!-- This redirect file can be deleted after <2025-01-03>. -->
<!-- Redirects that point to other docs in the same project expire in three months. -->
<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
doc/development/testing_guide/end_to_end/beginners_guide/flows.md 0 → 100644
+ 62
− 0
View file @ 3d5a1ea0
---
stage: none
group: unassigned
info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review.
---
# Flows in GitLab QA
Flows are frequently used sequences of actions. They are a higher level
of abstraction than page objects. Flows can include multiple page objects,
or any other relevant code.
For example, the sign in flow encapsulates two steps that are included
in every browser UI test.
```ruby
# QA::Flow::Login
def sign_in(as: nil)
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.perform { |login| login.sign_in_using_credentials(user: as) }
end
# When used in a test
it 'performs a test after signing in as the default user' do
Flow::Login.sign_in
# Perform the test
end
```
`QA::Flow::Login` provides an even more useful flow, allowing a test to easily switch users.
```ruby
# QA::Flow::Login
def while_signed_in(as: nil)
Page::Main::Menu.perform(&:sign_out_if_signed_in)
sign_in(as: as)
yield
Page::Main::Menu.perform(&:sign_out)
end
# When used in a test
it 'performs a test as one user and verifies as another' do
user1 = Resource::User.fabricate_or_use(Runtime::Env.gitlab_qa_username_1, Runtime::Env.gitlab_qa_password_1)
user2 = Resource::User.fabricate_or_use(Runtime::Env.gitlab_qa_username_2, Runtime::Env.gitlab_qa_password_2)
Flow::Login.while_signed_in(as: user1) do
# Perform some setup as user1
end
Flow::Login.sign_in(as: user2)
# Perform the rest of the test as user2
end
```
doc/development/testing_guide/end_to_end/beginners_guide/index.md 0 → 100644
+ 340
− 0
View file @ 3d5a1ea0
---
stage: none
group: unassigned
info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review.
---
# Beginner's guide to writing end-to-end tests
This tutorial walks you through the creation of end-to-end (_e2e_) tests for [GitLab Community Edition](https://about.gitlab.com/install/?version=ce) and [GitLab Enterprise Edition](https://about.gitlab.com/install/).
By the end of this tutorial, you can:
- Determine whether an end-to-end test is needed.
- Understand the directory structure within `qa/`.
- Write a basic end-to-end test that validates login features.
- Develop any missing [page object](page_objects.md) libraries.
## Before you write a test
Before you write tests, your [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit) must be configured to run the specs. The end-to-end tests:
- Are contained within the `qa/` directory.
- Should be independent and [idempotent](https://en.wikipedia.org/wiki/Idempotence#Computer_science_meaning).
- Create [resources](resources.md) (such as project, issue, user) on an ad-hoc basis.
- Test the UI and API interfaces, and use the API to efficiently set up the UI tests.
## Determine if end-to-end tests are needed
Check the code coverage of a specific feature before writing end-to-end tests for the [GitLab](https://gitlab-org.gitlab.io/gitlab/coverage-ruby/#_AllFiles) project. Does sufficient test coverage exist at the unit, feature, or integration levels? If you answered *yes*, then you *don't* need an end-to-end test.
For information about the distribution of tests per level in GitLab, see [Testing Levels](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/testing_guide/testing_levels.md).
- See the [How to test at the correct level?](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/testing_guide/testing_levels.md#how-to-test-at-the-correct-level) section of the [Testing levels](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/testing_guide/testing_levels.md) document.
- Review how often the feature changes. Stable features that don't change very often might not be worth covering with end-to-end tests if they are already covered in lower level tests.
- Finally, discuss the proposed test with the developers involved in implementing the feature and the lower-level tests.
WARNING:
Check the [GitLab](https://gitlab-org.gitlab.io/gitlab/coverage-ruby/#_AllFiles) coverage project for previously written tests for this feature. To analyze code coverage, you must understand which application files implement specific features.
In this tutorial we're writing a login end-to-end test, even though it has been sufficiently covered by lower-level testing, because it's the first step for most end-to-end flows, and is easiest to understand.
## Identify the DevOps stage
The GitLab QA end-to-end tests are organized by the different [stages in the DevOps lifecycle](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/specs/features/browser_ui). Determine where the test should be placed by [stage](https://handbook.gitlab.com/handbook/product/categories/#devops-stages), determine which feature the test belongs to, and then place it in a subdirectory under the stage.
![DevOps lifecycle by stages](../img/gl-devops-lifecycle-by-stage_v12_10.png)
If the test is Enterprise Edition only, the test is created in the `features/ee` directory, but follow the same DevOps lifecycle format.
## Create a skeleton test
In the first part of this tutorial we are testing login, which is owned by the Manage stage. Inside `qa/specs/features/browser_ui/1_manage/login`, create a file `basic_login_spec.rb`.
### The outer `context` block
See the [`RSpec.describe` outer block](#the-outer-rspecdescribe-block)
WARNING:
The outer `context` [was deprecated](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/550) in `13.2` in adherence to RSpec 4.0 specifications. Use `RSpec.describe` instead.
### The outer `RSpec.describe` block
Specs have an outer `RSpec.describe` indicating the DevOps stage.
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Manage' do
end
end
```
### The `describe` block
Inside of our outer `RSpec.describe`, describe the feature to test. In this case, `Login`.
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Manage' do
describe 'Login' do
end
end
end
```
### The `product_group` metadata
Assign `product_group` metadata and specify what product group this test belongs to. In this case, `authentication_and_authorization`.
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Manage' do
describe 'Login', product_group: :authentication do
end
end
end
```
### The `it` blocks (examples)
Every test suite contains at least one `it` block (example). A good way to start writing end-to-end tests is to write test case descriptions as `it` blocks:
```ruby
module QA
RSpec.describe 'Manage' do
describe 'Login', product_group: :authentication do
it 'can login' do
end
it 'can logout' do
end
end
end
end
```
## Write the test
An important question is "What do we test?" and even more importantly, "How do we test?"
Begin by logging in.
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Manage' do
describe 'Login', product_group: :authentication do
it 'can login' do
Flow::Login.sign_in
end
it 'can logout' do
Flow::Login.sign_in
end
end
end
end
```
NOTE:
For more information on Flows, see [Flows](flows.md)
After [running the spec](#run-the-spec), our test should login and end; then we should answer the question "What do we test?"
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Manage' do
describe 'Login', product_group: :authentication do
it 'can login' do
Flow::Login.sign_in
Page::Main::Menu.perform do |menu|
expect(menu).to be_signed_in
end
end
it 'can logout' do
Flow::Login.sign_in
Page::Main::Menu.perform do |menu|
menu.sign_out
expect(menu).not_to be_signed_in
end
end
end
end
end
```
**What do we test?**
1. Can we sign in?
1. Can we sign out?
**How do we test?**
1. Check if the user avatar appears in the left sidebar.
1. Check if the user avatar *does not* appear in the left sidebar.
Behind the scenes, `be_signed_in` is a [predicate matcher](https://rspec.info/features/3-12/rspec-expectations/built-in-matchers/predicates/) that [implements checking the user avatar](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/page/main/menu.rb#L92).
## De-duplicate your code
Refactor your test to use a `before` block for test setup, since it's duplicating a call to `sign_in`.
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Manage' do
describe 'Login', product_group: :authentication do
before do
Flow::Login.sign_in
end
it 'can login' do
Page::Main::Menu.perform do |menu|
expect(menu).to be_signed_in
end
end
it 'can logout' do
Page::Main::Menu.perform do |menu|
menu.sign_out
expect(menu).not_to be_signed_in
end
end
end
end
end
```
The `before` block is essentially a `before(:each)` and is run before each example, ensuring we now sign in at the beginning of each test.
## Test setup using resources and page objects
Next, let's test something other than Login. Let's test Issues, which are owned by the Plan stage and the Project Management Group, so [create a file](#identify-the-devops-stage) in `qa/specs/features/browser_ui/2_plan/issue` called `issues_spec.rb`.
```ruby
# frozen_string_literal: true
module QA
RSpec.describe 'Plan' do
describe 'Issues', product_group: :project_management do
let(:issue) { create(:issue) }
before do
Flow::Login.sign_in
issue.visit!
end
it 'can close an issue' do
Page::Project::Issue::Show.perform do |show|
show.click_close_issue_button
expect(show).to be_closed
end
end
end
end
end
```
Note the following important points:
- At the start of our example, we are at the `page/issue/show.rb` [page](page_objects.md).
- Our test fabricates only what it needs, when it needs it.
- The issue is fabricated through the API to save time.
- GitLab prefers `let()` over instance variables. See [best practices](../../best_practices.md#subject-and-let-variables).
- `be_closed` is not implemented in `page/project/issue/show.rb` yet, but is implemented in the next step.
The issue is fabricated as a [Resource](resources.md), which is a GitLab entity you can create through the UI or API. Other examples include:
- A [Merge Request](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/resource/merge_request.rb).
- A [User](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/resource/user.rb).
- A [Project](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/resource/project.rb).
- A [Group](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/resource/group.rb).
## Write the page object
A [Page Object](page_objects.md) is a class in our suite that represents a page within GitLab. The **Login** page would be one example. Since our page object for the **Issue Show** page already exists, add the `closed?` method.
```ruby
module Page::Project::Issue
class Show
view 'app/views/projects/issues/show.html.haml' do
element 'closed-status-box'
end
def closed?
has_element?('closed-status-box')
end
end
end
```
Next, define the element `closed-status-box` within your view, so your Page Object can see it.
```haml
-#=> app/views/projects/issues/show.html.haml
.issuable-status-box.status-box.status-box-issue-closed{ ..., data: { testid: 'closed-status-box' } }
```
## Run the spec
Before running the spec, make sure that:
- GDK is installed.
- GDK is running locally on port 3000.
- No additional [RSpec metadata tags](../rspec_metadata_tests.md) have been applied.
- Your working directory is `qa/` within your GDK GitLab installation.
- Your GitLab instance-level settings are default. If you changed the default settings, some tests might have unexpected results.
- Because the GDK requires a password change on first login, you must include the GDK password for `root` user
To run the spec, run the following command:
```shell
GITLAB_PASSWORD=<GDK root password> bundle exec rspec <test_file>
```
Where `<test_file>` is:
- `qa/specs/features/browser_ui/1_manage/login/log_in_spec.rb` when running the Login example.
- `qa/specs/features/browser_ui/2_plan/issue/create_issue_spec.rb` when running the Issue example.
Additional information on test execution and possible options are described in ["QA framework README"](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/README.md#run-the-end-to-end-tests-in-a-local-development-environment)
## Preparing test for code review
Before submitting the test for code review, there are a few housecleaning tasks to do:
1. Ensure that the test name follows the recommended [naming convention](../best_practices.md#test-naming).
1. Ensure that the spec is [linked to a test case](../best_practices.md#link-a-test-to-its-test-case).
1. Ensure that the spec has the correct `product_group` metadata. See [Product sections, stages, groups, and categories](https://handbook.gitlab.com/handbook/product/categories/) for the comprehensive list of groups.
1. Ensure that the relevant [RSpec metadata](../rspec_metadata_tests.md) are added to the spec.
1. Ensure the page object elements are named according to the [recommended naming convention](../style_guide.md#element-naming-convention).
NOTE:
For more information, see [End-to-end testing best practices](../best_practices.md) and [End-to-end testing style guide](../style_guide.md).
## End-to-end test merge request template
When submitting a new end-to-end test, use the ["New End to End Test"](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/merge_request_templates/New%20End%20To%20End%20Test.md) merge request description template for additional steps that are required prior a successful merge.
doc/development/testing_guide/end_to_end/beginners_guide/page_objects.md 0 → 100644
+ 304
− 0
View file @ 3d5a1ea0
---
stage: none
group: unassigned
info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review.
---
# Page objects in GitLab QA
In GitLab QA we are using a known pattern, called _Page Objects_.
This means that we have built an abstraction for all pages in GitLab that we use
to drive GitLab QA scenarios. Whenever we do something on a page, like filling
in a form or selecting a button, we do that only through a page object
associated with this area of GitLab.
For example, when GitLab QA test harness signs in into GitLab, it needs to fill
in user login and user password. To do that, we have a class, called
`Page::Main::Login` and `sign_in_using_credentials` methods, that is the only
piece of the code, that reads the `user-login` and `user-password`
fields.
## Why do we need that?
We need page objects because we need to reduce duplication and avoid problems
whenever someone changes some selectors in the GitLab source code.
Imagine that we have a hundred specs in GitLab QA, and we need to sign in to
GitLab each time, before we make assertions. Without a page object, one would
need to rely on volatile helpers or invoke Capybara methods directly. Imagine
invoking `fill_in 'user-login'` in every `*_spec.rb` file / test example.
When someone later changes `t.text_field 'login'` in the view associated with
this page to `t.text_field 'username'` it generates a different field
identifier, what would effectively break all tests.
Because we are using `Page::Main::Login.perform(&:sign_in_using_credentials)`
everywhere, when we want to sign in to GitLab, the page object is the single
source of truth, and we must update `fill_in 'user-login'`
to `fill_in 'user-username'` only in one place.
## What problems did we have in the past?
We do not run QA tests for every commit, because of performance reasons, and
the time it would take to build packages and test everything.
That is why when someone changes `t.text_field 'login'` to
`t.text_field 'username'` in the _new session_ view we don't know about this
change until our GitLab QA nightly pipeline fails, or until someone triggers
`package-and-qa` action in their merge request.
Such a change would break all tests. We call this problem a _fragile
tests problem_.
To make GitLab QA more reliable and robust, we had to solve this
problem by introducing coupling between GitLab CE / EE views and GitLab QA.
## How did we solve fragile tests problem?
Currently, when you add a new `Page::Base` derived class, you must also
define all selectors that your page objects depend on.
Whenever you push your code to CE / EE repository, `qa:selectors` sanity test
job runs as a part of a CI pipeline.
This test validates all page objects that we have implemented in
`qa/page` directory. When it fails, it notifies you about missing
or invalid views/selectors definition.
## How to properly implement a page object?
We have built a DSL to define coupling between a page object and GitLab views
it is actually implemented by. See an example below.
```ruby
module Page
module Main
class Login < Page::Base
view 'app/views/devise/passwords/edit.html.haml' do
element 'password-field'
element 'password-confirmation'
element 'change-password-button'
end
view 'app/views/devise/sessions/_new_base.html.haml' do
element 'login-field'
element 'password-field'
element 'sign-in-button'
end
# ...
end
end
end
```
### Defining Elements
The `view` DSL method corresponds to the Rails view, partial, or Vue component that renders the elements.
The `element` DSL method in turn declares an element for which a corresponding
`testid=element-name` data attribute must be added, if not already, to the view file.
You can also define a value (String or Regexp) to match to the actual view
code but **this is deprecated** in favor of the above method for two reasons:
- Consistency: there is only one way to define an element
- Separation of concerns: Tests use dedicated `data-testid` attributes instead of reusing code
or classes used by other components (for example, `js-*` classes etc.)
```ruby
view 'app/views/my/view.html.haml' do
### Good ###
# Implicitly require the CSS selector `[data-testid="logout-button"]` to be present in the view
element 'logout-button'
### Bad ###
## This is deprecated and forbidden by the `QA/ElementWithPattern` RuboCop cop.
# Require `f.submit "Sign in"` to be present in `my/view.html.haml
element :my_button, 'f.submit "Sign in"' # rubocop:disable QA/ElementWithPattern
## This is deprecated and forbidden by the `QA/ElementWithPattern` RuboCop cop.
# Match every line in `my/view.html.haml` against
# `/link_to .* "My Profile"/` regexp.
element :profile_link, /link_to .* "My Profile"/ # rubocop:disable QA/ElementWithPattern
end
```
### Adding Elements to a View
Given the following elements...
```ruby
view 'app/views/my/view.html.haml' do
element 'login-field'
element 'password-field'
element 'sign-in-button'
end
```
To add these elements to the view, you must change the Rails view, partial, or Vue component by adding a `data-testid` attribute
for each element defined.
In our case, `data-testid="login-field"`, `data-testid="password-field"` and `data-testid="sign-in-button"`
`app/views/my/view.html.haml`
```haml
= f.text_field :login, class: "form-control top", autofocus: "autofocus", autocapitalize: "off", autocorrect: "off", required: true, title: "This field is required.", data: { testid: 'login_field' }
= f.password_field :password, class: "form-control bottom", required: true, title: "This field is required.", data: { testid: 'password_field' }
= f.submit "Sign in", class: "btn btn-confirm", data: { testid: 'sign_in_button' }
```
Things to note:
- The name of the element and the `data-testid` must match and be kebab cased
- If the element appears on the page unconditionally, add `required: true` to the element. See
[Dynamic element validation](../dynamic_element_validation.md)
- You should not see `data-qa-selector` classes in Page Objects.
We should use the [`data-testid`](#data-testid-vs-data-qa-selector)
method of definition
### `data-testid` vs `data-qa-selector`
> - Introduced in GitLab 16.1
Any existing `data-qa-selector` class should be considered deprecated
and we should use the `data-testid` method of definition.
### Dynamic element selection
A common occurrence in automated testing is selecting a single "one-of-many" element.
In a list of several items, how do you differentiate what you are selecting on?
The most common workaround for this is via text matching. Instead, a better practice is
by matching on that specific element by a unique identifier, rather than by text.
We got around this by adding the `data-qa-*` extensible selection mechanism.
#### Examples
**Example 1**
Given the following Rails view (using GitLab Issues as an example):
```haml
%ul.issues-list
- @issues.each do |issue|
%li.issue{data: { testid: 'issue', qa_issue_title: issue.title } }= link_to issue
```
We can select on that specific issue by matching on the Rails model.
```ruby
class Page::Project::Issues::Index < Page::Base
def has_issue?(issue)
has_element?(:issue, issue_title: issue)
end
end
```
In our test, we can validate that this particular issue exists.
```ruby
describe 'Issue' do
it 'has an issue titled "hello"' do
Page::Project::Issues::Index.perform do |index|
expect(index).to have_issue('hello')
end
end
end
```
**Example 2**
*By an index...*
```haml
%ol
- @some_model.each_with_index do |model, idx|
%li.model{ data: { testid: 'model', qa_index: idx } }
```
```ruby
expect(the_page).to have_element(:model, index: 1) #=> select on the first model that appears in the list
```
### Exceptions
In some cases, it might not be possible or worthwhile to add a selector.
Some UI components use external libraries, including some maintained by third parties.
Even if a library is maintained by GitLab, the selector sanity test only runs
on code within the GitLab project, so it's not possible to specify the path for
the view for code in a library.
In such rare cases it's reasonable to use CSS selectors in page object methods,
with a comment explaining why an `element` can't be added.
### Define Page concerns
Some pages share common behaviors, and/or are prepended with EE-specific modules that adds EE-specific methods.
These modules must:
1. Extend from the `QA::Page::PageConcern` module, with `extend QA::Page::PageConcern`.
1. Override the `self.prepended` method if they need to `include`/`prepend` other modules themselves, and/or define
`view` or `elements`.
1. Call `super` as the first thing in `self.prepended`.
1. Include/prepend other modules and define their `view`/`elements` in a `base.class_eval` block to ensure they're
defined in the class that prepends the module.
These steps ensure the sanity selectors check detect problems properly.
For example, `qa/qa/ee/page/merge_request/show.rb` adds EE-specific methods to `qa/qa/page/merge_request/show.rb` (with
`QA::Page::MergeRequest::Show.prepend_mod_with('Page::MergeRequest::Show', namespace: QA)`) and following is how it's implemented
(only showing the relevant part and referring to the 4 steps described above with inline comments):
```ruby
module QA
module EE
module Page
module MergeRequest
module Show
extend QA::Page::PageConcern # 1.
def self.prepended(base) # 2.
super # 3.
base.class_eval do # 4.
prepend Page::Component::LicenseManagement
view 'app/assets/javascripts/vue_merge_request_widget/components/states/sha_mismatch.vue' do
element 'head-mismatch', "The source branch HEAD has recently changed."
end
[...]
end
end
end
end
end
end
end
```
## Running the test locally
During development, you can run the `qa:selectors` test by running
```shell
bin/qa Test::Sanity::Selectors
```
from within the `qa` directory.
## Where to ask for help?
If you need more information, ask for help on `#test-platform` channel on Slack
(internal, GitLab Team only).
If you are not a Team Member, and you still need help to contribute,
open an issue in GitLab CE issue tracker with the `~QA` label.
doc/development/testing_guide/end_to_end/beginners_guide/resources.md 0 → 100644
+ 507
− 0
View file @ 3d5a1ea0
---
stage: none
group: unassigned
info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review.
---
# Resource classes in GitLab QA
Resources are primarily created using Browser UI steps, but can also be created via the API or the CLI.
## How to properly implement a resource class?
All resource classes should inherit from `Resource::Base`.
There is only one mandatory method to implement to define a resource class.
This is the `#fabricate!` method, which is used to build the resource via the
browser UI. Note that you should only use [Page objects](page_objects.md) to
interact with a Web page in this method.
Here is an imaginary example:
```ruby
module QA
module Resource
class Shirt < Base
attr_accessor :name
def fabricate!
Page::Dashboard::Index.perform do |dashboard_index|
dashboard_index.go_to_new_shirt
end
Page::Shirt::New.perform do |shirt_new|
shirt_new.set_name(name)
shirt_new.create_shirt!
end
end
end
end
end
```
### Define API implementation
A resource class may also implement the three following methods to be able to
create the resource via the public GitLab API:
- `#api_get_path`: The `GET` path to fetch an existing resource.
- `#api_post_path`: The `POST` path to create a new resource.
- `#api_post_body`: The `POST` body (as a Ruby hash) to create a new resource.
> Be aware that many API resources are [paginated](../../../../api/rest/index.md#pagination).
> If you don't find the results you expect, check if there is more that one page of results.
Let's take the `Shirt` resource class, and add these three API methods:
```ruby
module QA
module Resource
class Shirt < Base
attr_accessor :name
def fabricate!
# ... same as before
end
def api_get_path
"/shirt/#{name}"
end
def api_post_path
"/shirts"
end
def api_post_body
{
name: name
}
end
end
end
end
```
The `Project` resource is a good real example of Browser
UI and API implementations.
#### Resource attributes
A resource may need another resource to exist first. For instance, a project
needs a group to be created in.
To define a resource attribute, you can use the `attribute` method with a
block using the other resource class to fabricate the resource.
That allows access to the other resource from your resource object's
methods. You would usually use it in `#fabricate!`, `#api_get_path`,
`#api_post_path`, `#api_post_body`.
Let's take the `Shirt` resource class, and add a `project` attribute to it:
```ruby
module QA
module Resource
class Shirt < Base
attr_accessor :name
attribute :project do
Project.fabricate! do |resource|
resource.name = 'project-to-create-a-shirt'
end
end
def fabricate!
project.visit!
Page::Project::Show.perform do |project_show|
project_show.go_to_new_shirt
end
Page::Shirt::New.perform do |shirt_new|
shirt_new.set_name(name)
shirt_new.create_shirt!
end
end
def api_get_path
"/project/#{project.path}/shirt/#{name}"
end
def api_post_path
"/project/#{project.path}/shirts"
end
def api_post_body
{
name: name
}
end
end
end
end
```
**Note that all the attributes are lazily constructed. This means if you want
a specific attribute to be fabricated first, you must call the
attribute method first even if you're not using it.**
#### Product data attributes
Once created, you may want to populate a resource with attributes that can be
found in the Web page, or in the API response.
For instance, once you create a project, you may want to store its repository
SSH URL as an attribute.
Again we could use the `attribute` method with a block, using a page object
to retrieve the data on the page.
Let's take the `Shirt` resource class, and define a `:brand` attribute:
```ruby
module QA
module Resource
class Shirt < Base
attr_accessor :name
attribute :project do
Project.fabricate! do |resource|
resource.name = 'project-to-create-a-shirt'
end
end
# Attribute populated from the Browser UI (using the block)
attribute :brand do
Page::Shirt::Show.perform do |shirt_show|
shirt_show.fetch_brand_from_page
end
end
# ... same as before
end
end
end
```
**Note again that all the attributes are lazily constructed. This means if
you call `shirt.brand` after moving to the other page, it doesn't properly
retrieve the data because we're no longer on the expected page.**
Consider this:
```ruby
shirt =
QA::Resource::Shirt.fabricate! do |resource|
resource.name = "GitLab QA"
end
shirt.project.visit!
shirt.brand # => FAIL!
```
The above example fails because now we're on the project page, trying to
construct the brand data from the shirt page, however we moved to the project
page already. There are two ways to solve this, one is that we could try to
retrieve the brand before visiting the project again:
```ruby
shirt =
QA::Resource::Shirt.fabricate! do |resource|
resource.name = "GitLab QA"
end
shirt.brand # => OK!
shirt.project.visit!
shirt.brand # => OK!
```
The attribute is stored in the instance, therefore all the following calls
are fine, using the data previously constructed. If we think that this
might be too brittle, we could eagerly construct the data right before
ending fabrication:
```ruby
module QA
module Resource
class Shirt < Base
# ... same as before
def fabricate!
project.visit!
Page::Project::Show.perform do |project_show|
project_show.go_to_new_shirt
end
Page::Shirt::New.perform do |shirt_new|
shirt_new.set_name(name)
shirt_new.create_shirt!
end
populate(:brand) # Eagerly construct the data
end
end
end
end
```
The `populate` method iterates through its arguments and call each
attribute. Here `populate(:brand)` has the same effect as
just `brand`. Using the populate method makes the intention clearer.
With this, it ensures we construct the data right after we create the
shirt. The drawback is that this always constructs the data when the
resource is fabricated even if we don't need to use the data.
Alternatively, we could just make sure we're on the right page before
constructing the brand data:
```ruby
module QA
module Resource
class Shirt < Base
attr_accessor :name
attribute :project do
Project.fabricate! do |resource|
resource.name = 'project-to-create-a-shirt'
end
end
# Attribute populated from the Browser UI (using the block)
attribute :brand do
back_url = current_url
visit!
Page::Shirt::Show.perform do |shirt_show|
shirt_show.fetch_brand_from_page
end
visit(back_url)
end
# ... same as before
end
end
end
```
This ensures it's on the shirt page before constructing brand, and
move back to the previous page to avoid breaking the state.
#### Define an attribute based on an API response
Sometimes, you want to define a resource attribute based on the API response
from its `GET` or `POST` request. For instance, if the creation of a shirt via
the API returns
```ruby
{
brand: 'a-brand-new-brand',
style: 't-shirt',
materials: [[:cotton, 80], [:polyamide, 20]]
}
```
you may want to store `style` as-is in the resource, and fetch the first value
of the first `materials` item in a `main_fabric` attribute.
Let's take the `Shirt` resource class, and define a `:style` and a
`:main_fabric` attributes:
```ruby
module QA
module Resource
class Shirt < Base
# ... same as before
# @style from the instance if present,
# or fetched from the API response if present,
# or a QA::Resource::Base::NoValueError is raised otherwise
attribute :style
# If @main_fabric is not present,
# and if the API does not contain this field, this block will be
# used to construct the value based on the API response, and
# store the result in @main_fabric
attribute :main_fabric do
api_response.&dig(:materials, 0, 0)
end
# ... same as before
end
end
end
```
**Notes on attributes precedence:**
- resource instance variables have the highest precedence
- attributes from the API response take precedence over attributes from the
block (usually from Browser UI)
- attributes without a value raises a `QA::Resource::Base::NoValueError` error
## Creating resources in your tests
To create a resource in your tests, you can call the `.fabricate!` method on
the resource class, or use the [factory](#factories) to create it.
Note that if the resource class supports API fabrication, this uses this
fabrication by default.
Here is an example that uses the API fabrication method under the hood
since it's supported by the `Shirt` resource class:
```ruby
my_shirt = Resource::Shirt.fabricate! do |shirt|
shirt.name = 'my-shirt'
end
expect(page).to have_text(my_shirt.name) # => "my-shirt" from the resource's instance variable
expect(page).to have_text(my_shirt.brand) # => "a-brand-new-brand" from the API response
expect(page).to have_text(my_shirt.style) # => "t-shirt" from the API response
expect(page).to have_text(my_shirt.main_fabric) # => "cotton" from the API response via the block
```
If you explicitly want to use the Browser UI fabrication method, you can call
the `.fabricate_via_browser_ui!` method instead:
```ruby
my_shirt = Resource::Shirt.fabricate_via_browser_ui! do |shirt|
shirt.name = 'my-shirt'
end
expect(page).to have_text(my_shirt.name) # => "my-shirt" from the resource's instance variable
expect(page).to have_text(my_shirt.brand) # => the brand name fetched from the `Page::Shirt::Show` page via the block
expect(page).to have_text(my_shirt.style) # => QA::Resource::Base::NoValueError will be raised because no API response nor a block is provided
expect(page).to have_text(my_shirt.main_fabric) # => QA::Resource::Base::NoValueError will be raised because no API response and the block didn't provide a value (because it's also based on the API response)
```
You can also explicitly use the API fabrication method, by calling the
`.fabricate_via_api!` method:
```ruby
my_shirt = Resource::Shirt.fabricate_via_api! do |shirt|
shirt.name = 'my-shirt'
end
```
In this case, the result is similar to calling `Resource::Shirt.fabricate!`.
### Factories
You may also use [FactoryBot](https://github.com/thoughtbot/factory_bot/) invocations to create, build, and fetch resources within your tests.
```ruby
# create a project via the API to use in the test
let(:project) { create(:project) }
# create an issue belonging to a project via the API to use in the test
let(:issue) { create(:issue, project: project) }
# create a private project via the API with a specific name
let(:project) { create(:project, :private, name: 'my-project-name', add_name_uuid: false) }
# create one commit in a project that performs three actions
let(:commit) do
create(:commit, commit_message: 'my message', project: project, actions: [
{ action: 'create', file_path: 'README.md', content: '# Welcome!' },
{ action: 'update', file_path: 'README.md', content: '# Updated' },
{ action: 'delete', file_path: 'README.md' }
])
end
###
# instantiate an Issue but don't create it via API yet
let(:issue) { build(:issue) }
# instantiate a Project and perform some actions before creating
let(:project) do
build(:project) do |p|
p.name = 'Test'
p.add_name_uuid = false
end
end
# fetch an existing issue via the API with attributes
let(:existing_issue) { build(:issue, project: project, iid: issue.iid).reload! }
```
All factories are defined in [`qa/qa/factories`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/factories/) and are representative of
their respective `QA::Resource::Base` class.
For example, a factory `:issue` can be found in `qa/resource/issue.rb`. A factory `:project` can be found in `qa/resource/project.rb`.
#### Create a new Factory
Given a resource:
```ruby
# qa/resource/shirt.rb
module QA
module Resource
class Shirt < Base
attr_accessor :name
attr_reader :read_only
attribute :brand
def api_post_body
{ name: name, brand: brand }
end
end
end
end
```
Define a factory with defaults and overrides:
```ruby
# qa/factories/shirts.rb
module QA
FactoryBot.define do
factory :shirt, class: 'QA::Resource::Shirt' do
brand { 'BrandName' }
trait :with_name do
name { 'Shirt Name' }
end
end
end
end
```
In the test, create the resource via the API:
```ruby
let(:my_shirt) { create(:shirt, brand: 'AnotherBrand') } #<Resource::Shirt @brand="AnotherBrand" @name=nil>
let(:named_shirt) { create(:shirt, :with_name) } #<Resource::Shirt @brand="Brand Name" @name="Shirt Name">
let(:invalid_shirt) { create(:shirt, read_only: true) } # NoMethodError
it 'creates a shirt' do
expect(my_shirt.brand).to eq('AnotherBrand')
expect(named_shirt.name).to eq('Shirt Name')
expect(invalid_shirt).to raise_error(NoMethodError) # tries to call Resource::Shirt#read_only=
end
```
### Resources cleanup
We have a mechanism to [collect](https://gitlab.com/gitlab-org/gitlab/-/blob/44345381e89d6bbd440f7b4c680d03e8b75b86de/qa/qa/tools/test_resource_data_processor.rb#L32)
all resources created during test executions, and another to [handle](https://gitlab.com/gitlab-org/gitlab/-/blob/44345381e89d6bbd440f7b4c680d03e8b75b86de/qa/qa/tools/test_resources_handler.rb#L44)
these resources. On [dotcom environments](https://handbook.gitlab.com/handbook/engineering/infrastructure/environments/#environments), after a test suite finishes in the [QA pipelines](https://handbook.gitlab.com/handbook/engineering/infrastructure/test-platform/debugging-qa-test-failures/#qa-test-pipelines), resources from all passing test are
automatically deleted in the same pipeline run. Resources from all failed tests are reserved for investigation,
and won't be deleted until the following Saturday by a scheduled pipeline. When introducing new resources,
also make sure to add any resource that cannot be deleted to the [IGNORED_RESOURCES](https://gitlab.com/gitlab-org/gitlab/-/blob/44345381e89d6bbd440f7b4c680d03e8b75b86de/qa/qa/tools/test_resources_handler.rb#L29)
list.
## Where to ask for help?
If you need more information, ask for help on `#test-platform` channel on Slack
(internal, GitLab Team only).
If you are not a Team Member, and you still need help to contribute,
open an issue in GitLab CE issue tracker with the `~QA` label.
doc/development/testing_guide/end_to_end/best_practices.md
+ 2
− 2
View file @ 3d5a1ea0
@@ -143,7 +143,7 @@ Resources should be fabricated via the API wherever possible.
 
We can save both time and money by fabricating resources that our test will need via the API.
 
[Learn more](resources.md) about resources.
[Learn more](beginners_guide/resources.md) about resources.
 
## Avoid superfluous expectations
 
@@ -446,7 +446,7 @@ except(page).to have_no_text('hidden')
```
 
Unfortunately, that's not automatically the case for the predicate methods that we add to our
[page objects](page_objects.md). We need to [create our own negatable matchers](https://rspec.info/features/3-12/rspec-expectations/custom-matchers/define-matcher/).
[page objects](beginners_guide/page_objects.md). We need to [create our own negatable matchers](https://rspec.info/features/3-12/rspec-expectations/custom-matchers/define-matcher/).
 
The initial example uses the `have_job` matcher which is derived from the
[`has_job?` predicate method of the `Page::Project::Pipeline::Show` page object](https://gitlab.com/gitlab-org/gitlab/-/blob/87864b3047c23b4308f59c27a3757045944af447/qa/qa/page/project/pipeline/show.rb#L53).
doc/development/testing_guide/end_to_end/feature_flags.md
+ 2
− 2
View file @ 3d5a1ea0
@@ -103,11 +103,11 @@ the feature flag enabled. To ensure tests pass in both scenarios:
1. Create another selector inside the new component or file.
1. Give it the same name as the old one.
 
Selectors are connected to a specific frontend file in the [page object](page_objects.md),
Selectors are connected to a specific frontend file in the [page object](beginners_guide/page_objects.md),
and checked for availability inside our `qa:selectors` test. If the mentioned selector
is missing inside that frontend file, the test fails. To ensure selectors are
available when a feature flag is enabled or disabled, add the new selector to the
[page object](page_objects.md), leaving the old selector in place.
[page object](beginners_guide/page_objects.md), leaving the old selector in place.
The test uses the correct selector and still detects missing selectors.
 
If a new feature changes an existing frontend file that already has a selector,
doc/development/testing_guide/end_to_end/flows.md
+ 8
− 59
View file @ 3d5a1ea0
---
stage: none
group: unassigned
info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review.
redirect_to: 'beginners_guide/flows.md'
remove_date: '2025-01-03'
---
<!-- markdownlint-disable -->
This document was moved to [another location](beginners_guide/flows.md).
 
# Flows in GitLab QA
Flows are frequently used sequences of actions. They are a higher level
of abstraction than page objects. Flows can include multiple page objects,
or any other relevant code.
For example, the sign in flow encapsulates two steps that are included
in every browser UI test.
```ruby
# QA::Flow::Login
def sign_in(as: nil)
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.perform { |login| login.sign_in_using_credentials(user: as) }
end
# When used in a test
it 'performs a test after signing in as the default user' do
Flow::Login.sign_in
# Perform the test
end
```
`QA::Flow::Login` provides an even more useful flow, allowing a test to easily switch users.
```ruby
# QA::Flow::Login
def while_signed_in(as: nil)
Page::Main::Menu.perform(&:sign_out_if_signed_in)
sign_in(as: as)
yield
Page::Main::Menu.perform(&:sign_out)
end
# When used in a test
it 'performs a test as one user and verifies as another' do
user1 = Resource::User.fabricate_or_use(Runtime::Env.gitlab_qa_username_1, Runtime::Env.gitlab_qa_password_1)
user2 = Resource::User.fabricate_or_use(Runtime::Env.gitlab_qa_username_2, Runtime::Env.gitlab_qa_password_2)
Flow::Login.while_signed_in(as: user1) do
# Perform some setup as user1
end
Flow::Login.sign_in(as: user2)
# Perform the rest of the test as user2
end
```
<!-- This redirect file can be deleted after <2025-01-03>. -->
<!-- Redirects that point to other docs in the same project expire in three months. -->
<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
doc/development/testing_guide/end_to_end/index.md
+ 2
− 2
View file @ 3d5a1ea0
@@ -292,11 +292,11 @@ We should follow these best practices for end-to-end tests:
 
Continued reading:
 
- [Beginner's Guide](beginners_guide.md)
- [Beginner's Guide](beginners_guide/index.md)
- [Style Guide](style_guide.md)
- [Best Practices](best_practices.md)
- [Testing with feature flags](feature_flags.md)
- [Flows](flows.md)
- [Flows](beginners_guide/flows.md)
- [RSpec metadata/tags](rspec_metadata_tests.md)
- [Execution context selection](execution_context_selection.md)
- [Troubleshooting](troubleshooting.md)
doc/development/testing_guide/end_to_end/page_objects.md
+ 8
− 301
View file @ 3d5a1ea0
---
stage: none
group: unassigned
info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review.
redirect_to: 'beginners_guide/page_objects.md'
remove_date: '2025-01-03'
---
<!-- markdownlint-disable -->
This document was moved to [another location](beginners_guide/page_objects.md).
 
# Page objects in GitLab QA
In GitLab QA we are using a known pattern, called _Page Objects_.
This means that we have built an abstraction for all pages in GitLab that we use
to drive GitLab QA scenarios. Whenever we do something on a page, like filling
in a form or selecting a button, we do that only through a page object
associated with this area of GitLab.
For example, when GitLab QA test harness signs in into GitLab, it needs to fill
in user login and user password. To do that, we have a class, called
`Page::Main::Login` and `sign_in_using_credentials` methods, that is the only
piece of the code, that reads the `user-login` and `user-password`
fields.
## Why do we need that?
We need page objects because we need to reduce duplication and avoid problems
whenever someone changes some selectors in the GitLab source code.
Imagine that we have a hundred specs in GitLab QA, and we need to sign in to
GitLab each time, before we make assertions. Without a page object, one would
need to rely on volatile helpers or invoke Capybara methods directly. Imagine
invoking `fill_in 'user-login'` in every `*_spec.rb` file / test example.
When someone later changes `t.text_field 'login'` in the view associated with
this page to `t.text_field 'username'` it generates a different field
identifier, what would effectively break all tests.
Because we are using `Page::Main::Login.perform(&:sign_in_using_credentials)`
everywhere, when we want to sign in to GitLab, the page object is the single
source of truth, and we must update `fill_in 'user-login'`
to `fill_in 'user-username'` only in one place.
## What problems did we have in the past?
We do not run QA tests for every commit, because of performance reasons, and
the time it would take to build packages and test everything.
That is why when someone changes `t.text_field 'login'` to
`t.text_field 'username'` in the _new session_ view we don't know about this
change until our GitLab QA nightly pipeline fails, or until someone triggers
`package-and-qa` action in their merge request.
Such a change would break all tests. We call this problem a _fragile
tests problem_.
To make GitLab QA more reliable and robust, we had to solve this
problem by introducing coupling between GitLab CE / EE views and GitLab QA.
## How did we solve fragile tests problem?
Currently, when you add a new `Page::Base` derived class, you must also
define all selectors that your page objects depend on.
Whenever you push your code to CE / EE repository, `qa:selectors` sanity test
job runs as a part of a CI pipeline.
This test validates all page objects that we have implemented in
`qa/page` directory. When it fails, it notifies you about missing
or invalid views/selectors definition.
## How to properly implement a page object?
We have built a DSL to define coupling between a page object and GitLab views
it is actually implemented by. See an example below.
```ruby
module Page
module Main
class Login < Page::Base
view 'app/views/devise/passwords/edit.html.haml' do
element 'password-field'
element 'password-confirmation'
element 'change-password-button'
end
view 'app/views/devise/sessions/_new_base.html.haml' do
element 'login-field'
element 'password-field'
element 'sign-in-button'
end
# ...
end
end
end
```
### Defining Elements
The `view` DSL method corresponds to the Rails view, partial, or Vue component that renders the elements.
The `element` DSL method in turn declares an element for which a corresponding
`testid=element-name` data attribute must be added, if not already, to the view file.
You can also define a value (String or Regexp) to match to the actual view
code but **this is deprecated** in favor of the above method for two reasons:
- Consistency: there is only one way to define an element
- Separation of concerns: Tests use dedicated `data-testid` attributes instead of reusing code
or classes used by other components (for example, `js-*` classes etc.)
```ruby
view 'app/views/my/view.html.haml' do
### Good ###
# Implicitly require the CSS selector `[data-testid="logout-button"]` to be present in the view
element 'logout-button'
### Bad ###
## This is deprecated and forbidden by the `QA/ElementWithPattern` RuboCop cop.
# Require `f.submit "Sign in"` to be present in `my/view.html.haml
element :my_button, 'f.submit "Sign in"' # rubocop:disable QA/ElementWithPattern
## This is deprecated and forbidden by the `QA/ElementWithPattern` RuboCop cop.
# Match every line in `my/view.html.haml` against
# `/link_to .* "My Profile"/` regexp.
element :profile_link, /link_to .* "My Profile"/ # rubocop:disable QA/ElementWithPattern
end
```
### Adding Elements to a View
Given the following elements...
```ruby
view 'app/views/my/view.html.haml' do
element 'login-field'
element 'password-field'
element 'sign-in-button'
end
```
To add these elements to the view, you must change the Rails view, partial, or Vue component by adding a `data-testid` attribute
for each element defined.
In our case, `data-testid="login-field"`, `data-testid="password-field"` and `data-testid="sign-in-button"`
`app/views/my/view.html.haml`
```haml
= f.text_field :login, class: "form-control top", autofocus: "autofocus", autocapitalize: "off", autocorrect: "off", required: true, title: "This field is required.", data: { testid: 'login_field' }
= f.password_field :password, class: "form-control bottom", required: true, title: "This field is required.", data: { testid: 'password_field' }
= f.submit "Sign in", class: "btn btn-confirm", data: { testid: 'sign_in_button' }
```
Things to note:
- The name of the element and the `data-testid` must match and be kebab cased
- If the element appears on the page unconditionally, add `required: true` to the element. See
[Dynamic element validation](dynamic_element_validation.md)
- You should not see `data-qa-selector` classes in Page Objects.
We should use the [`data-testid`](#data-testid-vs-data-qa-selector)
method of definition
### `data-testid` vs `data-qa-selector`
> - Introduced in GitLab 16.1
Any existing `data-qa-selector` class should be considered deprecated
and we should use the `data-testid` method of definition.
### Dynamic element selection
A common occurrence in automated testing is selecting a single "one-of-many" element.
In a list of several items, how do you differentiate what you are selecting on?
The most common workaround for this is via text matching. Instead, a better practice is
by matching on that specific element by a unique identifier, rather than by text.
We got around this by adding the `data-qa-*` extensible selection mechanism.
#### Examples
**Example 1**
Given the following Rails view (using GitLab Issues as an example):
```haml
%ul.issues-list
- @issues.each do |issue|
%li.issue{data: { testid: 'issue', qa_issue_title: issue.title } }= link_to issue
```
We can select on that specific issue by matching on the Rails model.
```ruby
class Page::Project::Issues::Index < Page::Base
def has_issue?(issue)
has_element?(:issue, issue_title: issue)
end
end
```
In our test, we can validate that this particular issue exists.
```ruby
describe 'Issue' do
it 'has an issue titled "hello"' do
Page::Project::Issues::Index.perform do |index|
expect(index).to have_issue('hello')
end
end
end
```
**Example 2**
*By an index...*
```haml
%ol
- @some_model.each_with_index do |model, idx|
%li.model{ data: { testid: 'model', qa_index: idx } }
```
```ruby
expect(the_page).to have_element(:model, index: 1) #=> select on the first model that appears in the list
```
### Exceptions
In some cases, it might not be possible or worthwhile to add a selector.
Some UI components use external libraries, including some maintained by third parties.
Even if a library is maintained by GitLab, the selector sanity test only runs
on code within the GitLab project, so it's not possible to specify the path for
the view for code in a library.
In such rare cases it's reasonable to use CSS selectors in page object methods,
with a comment explaining why an `element` can't be added.
### Define Page concerns
Some pages share common behaviors, and/or are prepended with EE-specific modules that adds EE-specific methods.
These modules must:
1. Extend from the `QA::Page::PageConcern` module, with `extend QA::Page::PageConcern`.
1. Override the `self.prepended` method if they need to `include`/`prepend` other modules themselves, and/or define
`view` or `elements`.
1. Call `super` as the first thing in `self.prepended`.
1. Include/prepend other modules and define their `view`/`elements` in a `base.class_eval` block to ensure they're
defined in the class that prepends the module.
These steps ensure the sanity selectors check detect problems properly.
For example, `qa/qa/ee/page/merge_request/show.rb` adds EE-specific methods to `qa/qa/page/merge_request/show.rb` (with
`QA::Page::MergeRequest::Show.prepend_mod_with('Page::MergeRequest::Show', namespace: QA)`) and following is how it's implemented
(only showing the relevant part and referring to the 4 steps described above with inline comments):
```ruby
module QA
module EE
module Page
module MergeRequest
module Show
extend QA::Page::PageConcern # 1.
def self.prepended(base) # 2.
super # 3.
base.class_eval do # 4.
prepend Page::Component::LicenseManagement
view 'app/assets/javascripts/vue_merge_request_widget/components/states/sha_mismatch.vue' do
element 'head-mismatch', "The source branch HEAD has recently changed."
end
[...]
end
end
end
end
end
end
end
```
## Running the test locally
During development, you can run the `qa:selectors` test by running
```shell
bin/qa Test::Sanity::Selectors
```
from within the `qa` directory.
## Where to ask for help?
If you need more information, ask for help on `#test-platform` channel on Slack
(internal, GitLab Team only).
If you are not a Team Member, and you still need help to contribute,
open an issue in GitLab CE issue tracker with the `~QA` label.
<!-- This redirect file can be deleted after <2025-01-03>. -->
<!-- Redirects that point to other docs in the same project expire in three months. -->
<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
doc/development/testing_guide/end_to_end/resources.md
+ 8
− 504
View file @ 3d5a1ea0
---
stage: none
group: unassigned
info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review.
redirect_to: 'beginners_guide/resources.md'
remove_date: '2025-01-03'
---
<!-- markdownlint-disable -->
This document was moved to [another location](beginners_guide/resources.md).
 
# Resource classes in GitLab QA
Resources are primarily created using Browser UI steps, but can also be created via the API or the CLI.
## How to properly implement a resource class?
All resource classes should inherit from `Resource::Base`.
There is only one mandatory method to implement to define a resource class.
This is the `#fabricate!` method, which is used to build the resource via the
browser UI. Note that you should only use [Page objects](page_objects.md) to
interact with a Web page in this method.
Here is an imaginary example:
```ruby
module QA
module Resource
class Shirt < Base
attr_accessor :name
def fabricate!
Page::Dashboard::Index.perform do |dashboard_index|
dashboard_index.go_to_new_shirt
end
Page::Shirt::New.perform do |shirt_new|
shirt_new.set_name(name)
shirt_new.create_shirt!
end
end
end
end
end
```
### Define API implementation
A resource class may also implement the three following methods to be able to
create the resource via the public GitLab API:
- `#api_get_path`: The `GET` path to fetch an existing resource.
- `#api_post_path`: The `POST` path to create a new resource.
- `#api_post_body`: The `POST` body (as a Ruby hash) to create a new resource.
> Be aware that many API resources are [paginated](../../../api/rest/index.md#pagination).
> If you don't find the results you expect, check if there is more that one page of results.
Let's take the `Shirt` resource class, and add these three API methods:
```ruby
module QA
module Resource
class Shirt < Base
attr_accessor :name
def fabricate!
# ... same as before
end
def api_get_path
"/shirt/#{name}"
end
def api_post_path
"/shirts"
end
def api_post_body
{
name: name
}
end
end
end
end
```
The `Project` resource is a good real example of Browser
UI and API implementations.
#### Resource attributes
A resource may need another resource to exist first. For instance, a project
needs a group to be created in.
To define a resource attribute, you can use the `attribute` method with a
block using the other resource class to fabricate the resource.
That allows access to the other resource from your resource object's
methods. You would usually use it in `#fabricate!`, `#api_get_path`,
`#api_post_path`, `#api_post_body`.
Let's take the `Shirt` resource class, and add a `project` attribute to it:
```ruby
module QA
module Resource
class Shirt < Base
attr_accessor :name
attribute :project do
Project.fabricate! do |resource|
resource.name = 'project-to-create-a-shirt'
end
end
def fabricate!
project.visit!
Page::Project::Show.perform do |project_show|
project_show.go_to_new_shirt
end
Page::Shirt::New.perform do |shirt_new|
shirt_new.set_name(name)
shirt_new.create_shirt!
end
end
def api_get_path
"/project/#{project.path}/shirt/#{name}"
end
def api_post_path
"/project/#{project.path}/shirts"
end
def api_post_body
{
name: name
}
end
end
end
end
```
**Note that all the attributes are lazily constructed. This means if you want
a specific attribute to be fabricated first, you must call the
attribute method first even if you're not using it.**
#### Product data attributes
Once created, you may want to populate a resource with attributes that can be
found in the Web page, or in the API response.
For instance, once you create a project, you may want to store its repository
SSH URL as an attribute.
Again we could use the `attribute` method with a block, using a page object
to retrieve the data on the page.
Let's take the `Shirt` resource class, and define a `:brand` attribute:
```ruby
module QA
module Resource
class Shirt < Base
attr_accessor :name
attribute :project do
Project.fabricate! do |resource|
resource.name = 'project-to-create-a-shirt'
end
end
# Attribute populated from the Browser UI (using the block)
attribute :brand do
Page::Shirt::Show.perform do |shirt_show|
shirt_show.fetch_brand_from_page
end
end
# ... same as before
end
end
end
```
**Note again that all the attributes are lazily constructed. This means if
you call `shirt.brand` after moving to the other page, it doesn't properly
retrieve the data because we're no longer on the expected page.**
Consider this:
```ruby
shirt =
QA::Resource::Shirt.fabricate! do |resource|
resource.name = "GitLab QA"
end
shirt.project.visit!
shirt.brand # => FAIL!
```
The above example fails because now we're on the project page, trying to
construct the brand data from the shirt page, however we moved to the project
page already. There are two ways to solve this, one is that we could try to
retrieve the brand before visiting the project again:
```ruby
shirt =
QA::Resource::Shirt.fabricate! do |resource|
resource.name = "GitLab QA"
end
shirt.brand # => OK!
shirt.project.visit!
shirt.brand # => OK!
```
The attribute is stored in the instance, therefore all the following calls
are fine, using the data previously constructed. If we think that this
might be too brittle, we could eagerly construct the data right before
ending fabrication:
```ruby
module QA
module Resource
class Shirt < Base
# ... same as before
def fabricate!
project.visit!
Page::Project::Show.perform do |project_show|
project_show.go_to_new_shirt
end
Page::Shirt::New.perform do |shirt_new|
shirt_new.set_name(name)
shirt_new.create_shirt!
end
populate(:brand) # Eagerly construct the data
end
end
end
end
```
The `populate` method iterates through its arguments and call each
attribute. Here `populate(:brand)` has the same effect as
just `brand`. Using the populate method makes the intention clearer.
With this, it ensures we construct the data right after we create the
shirt. The drawback is that this always constructs the data when the
resource is fabricated even if we don't need to use the data.
Alternatively, we could just make sure we're on the right page before
constructing the brand data:
```ruby
module QA
module Resource
class Shirt < Base
attr_accessor :name
attribute :project do
Project.fabricate! do |resource|
resource.name = 'project-to-create-a-shirt'
end
end
# Attribute populated from the Browser UI (using the block)
attribute :brand do
back_url = current_url
visit!
Page::Shirt::Show.perform do |shirt_show|
shirt_show.fetch_brand_from_page
end
visit(back_url)
end
# ... same as before
end
end
end
```
This ensures it's on the shirt page before constructing brand, and
move back to the previous page to avoid breaking the state.
#### Define an attribute based on an API response
Sometimes, you want to define a resource attribute based on the API response
from its `GET` or `POST` request. For instance, if the creation of a shirt via
the API returns
```ruby
{
brand: 'a-brand-new-brand',
style: 't-shirt',
materials: [[:cotton, 80], [:polyamide, 20]]
}
```
you may want to store `style` as-is in the resource, and fetch the first value
of the first `materials` item in a `main_fabric` attribute.
Let's take the `Shirt` resource class, and define a `:style` and a
`:main_fabric` attributes:
```ruby
module QA
module Resource
class Shirt < Base
# ... same as before
# @style from the instance if present,
# or fetched from the API response if present,
# or a QA::Resource::Base::NoValueError is raised otherwise
attribute :style
# If @main_fabric is not present,
# and if the API does not contain this field, this block will be
# used to construct the value based on the API response, and
# store the result in @main_fabric
attribute :main_fabric do
api_response.&dig(:materials, 0, 0)
end
# ... same as before
end
end
end
```
**Notes on attributes precedence:**
- resource instance variables have the highest precedence
- attributes from the API response take precedence over attributes from the
block (usually from Browser UI)
- attributes without a value raises a `QA::Resource::Base::NoValueError` error
## Creating resources in your tests
To create a resource in your tests, you can call the `.fabricate!` method on
the resource class, or use the [factory](#factories) to create it.
Note that if the resource class supports API fabrication, this uses this
fabrication by default.
Here is an example that uses the API fabrication method under the hood
since it's supported by the `Shirt` resource class:
```ruby
my_shirt = Resource::Shirt.fabricate! do |shirt|
shirt.name = 'my-shirt'
end
expect(page).to have_text(my_shirt.name) # => "my-shirt" from the resource's instance variable
expect(page).to have_text(my_shirt.brand) # => "a-brand-new-brand" from the API response
expect(page).to have_text(my_shirt.style) # => "t-shirt" from the API response
expect(page).to have_text(my_shirt.main_fabric) # => "cotton" from the API response via the block
```
If you explicitly want to use the Browser UI fabrication method, you can call
the `.fabricate_via_browser_ui!` method instead:
```ruby
my_shirt = Resource::Shirt.fabricate_via_browser_ui! do |shirt|
shirt.name = 'my-shirt'
end
expect(page).to have_text(my_shirt.name) # => "my-shirt" from the resource's instance variable
expect(page).to have_text(my_shirt.brand) # => the brand name fetched from the `Page::Shirt::Show` page via the block
expect(page).to have_text(my_shirt.style) # => QA::Resource::Base::NoValueError will be raised because no API response nor a block is provided
expect(page).to have_text(my_shirt.main_fabric) # => QA::Resource::Base::NoValueError will be raised because no API response and the block didn't provide a value (because it's also based on the API response)
```
You can also explicitly use the API fabrication method, by calling the
`.fabricate_via_api!` method:
```ruby
my_shirt = Resource::Shirt.fabricate_via_api! do |shirt|
shirt.name = 'my-shirt'
end
```
In this case, the result is similar to calling `Resource::Shirt.fabricate!`.
### Factories
You may also use [FactoryBot](https://github.com/thoughtbot/factory_bot/) invocations to create, build, and fetch resources within your tests.
```ruby
# create a project via the API to use in the test
let(:project) { create(:project) }
# create an issue belonging to a project via the API to use in the test
let(:issue) { create(:issue, project: project) }
# create a private project via the API with a specific name
let(:project) { create(:project, :private, name: 'my-project-name', add_name_uuid: false) }
# create one commit in a project that performs three actions
let(:commit) do
create(:commit, commit_message: 'my message', project: project, actions: [
{ action: 'create', file_path: 'README.md', content: '# Welcome!' },
{ action: 'update', file_path: 'README.md', content: '# Updated' },
{ action: 'delete', file_path: 'README.md' }
])
end
###
# instantiate an Issue but don't create it via API yet
let(:issue) { build(:issue) }
# instantiate a Project and perform some actions before creating
let(:project) do
build(:project) do |p|
p.name = 'Test'
p.add_name_uuid = false
end
end
# fetch an existing issue via the API with attributes
let(:existing_issue) { build(:issue, project: project, iid: issue.iid).reload! }
```
All factories are defined in [`qa/qa/factories`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/factories/) and are representative of
their respective `QA::Resource::Base` class.
For example, a factory `:issue` can be found in `qa/resource/issue.rb`. A factory `:project` can be found in `qa/resource/project.rb`.
#### Create a new Factory
Given a resource:
```ruby
# qa/resource/shirt.rb
module QA
module Resource
class Shirt < Base
attr_accessor :name
attr_reader :read_only
attribute :brand
def api_post_body
{ name: name, brand: brand }
end
end
end
end
```
Define a factory with defaults and overrides:
```ruby
# qa/factories/shirts.rb
module QA
FactoryBot.define do
factory :shirt, class: 'QA::Resource::Shirt' do
brand { 'BrandName' }
trait :with_name do
name { 'Shirt Name' }
end
end
end
end
```
In the test, create the resource via the API:
```ruby
let(:my_shirt) { create(:shirt, brand: 'AnotherBrand') } #<Resource::Shirt @brand="AnotherBrand" @name=nil>
let(:named_shirt) { create(:shirt, :with_name) } #<Resource::Shirt @brand="Brand Name" @name="Shirt Name">
let(:invalid_shirt) { create(:shirt, read_only: true) } # NoMethodError
it 'creates a shirt' do
expect(my_shirt.brand).to eq('AnotherBrand')
expect(named_shirt.name).to eq('Shirt Name')
expect(invalid_shirt).to raise_error(NoMethodError) # tries to call Resource::Shirt#read_only=
end
```
### Resources cleanup
We have a mechanism to [collect](https://gitlab.com/gitlab-org/gitlab/-/blob/44345381e89d6bbd440f7b4c680d03e8b75b86de/qa/qa/tools/test_resource_data_processor.rb#L32)
all resources created during test executions, and another to [handle](https://gitlab.com/gitlab-org/gitlab/-/blob/44345381e89d6bbd440f7b4c680d03e8b75b86de/qa/qa/tools/test_resources_handler.rb#L44)
these resources. On [dotcom environments](https://handbook.gitlab.com/handbook/engineering/infrastructure/environments/#environments), after a test suite finishes in the [QA pipelines](https://handbook.gitlab.com/handbook/engineering/infrastructure/test-platform/debugging-qa-test-failures/#qa-test-pipelines), resources from all passing test are
automatically deleted in the same pipeline run. Resources from all failed tests are reserved for investigation,
and won't be deleted until the following Saturday by a scheduled pipeline. When introducing new resources,
also make sure to add any resource that cannot be deleted to the [IGNORED_RESOURCES](https://gitlab.com/gitlab-org/gitlab/-/blob/44345381e89d6bbd440f7b4c680d03e8b75b86de/qa/qa/tools/test_resources_handler.rb#L29)
list.
## Where to ask for help?
If you need more information, ask for help on `#test-platform` channel on Slack
(internal, GitLab Team only).
If you are not a Team Member, and you still need help to contribute,
open an issue in GitLab CE issue tracker with the `~QA` label.
<!-- This redirect file can be deleted after <2025-01-03>. -->
<!-- Redirects that point to other docs in the same project expire in three months. -->
<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
+ 1
− 1
View file @ 3d5a1ea0
@@ -546,7 +546,7 @@ end
 
You can verify whether GitLab is appropriately redirecting your session to the `canary` or `non-canary` nodes with the `menu.canary?` method.
 
The above spec is verbose, written specifically this way to ensure the idea behind the implementation is clear. We recommend following the practices detailed within our [Beginner's guide to writing end-to-end tests](beginners_guide.md).
The above spec is verbose, written specifically this way to ensure the idea behind the implementation is clear. We recommend following the practices detailed within our [Beginner's guide to writing end-to-end tests](beginners_guide/index.md).
 
## Tests for GitLab as OpenID Connect (OIDC) and OAuth provider
 
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment