Add link to token expiration Rake tasks

Ideally we would remove these legacy scripts entirely or move
them into another section, but until we are confident that
this tool covers the needs of most users let's keep it for now.
We'll need to update the documentation as to what backported
GitLab versions contains this Rake task.

doc/security/token_overview.md

@@ -278,3 +278,392 @@ The following tables show the prefixes for each type of token where applicable.
    - Trigger tokens.
    - Runner registration tokens.
    - Any other sensitive secrets etc.
+
+## Expired access tokens
+
+If an existing access token is in use and reaches the `expires_at` value, the token
+expires and:
+
+- Can no longer be used for authentication.
+- Is not visible in the UI.
+
+Requests made using this token return a `401 Unauthorized` response. Too many
+unauthorized requests in a short period of time from the same IP address
+result in `403 Forbidden` responses from GitLab.com.
+
+For more information on authentication request limits, see [Git and container registry failed authentication ban](../user/gitlab_com/index.md#git-and-container-registry-failed-authentication-ban).
+
+To replace the token:
+
+1. Check where this token may have been used previously, and remove it from any
+   automation might still use the token.
+   - For personal access tokens, use the [API](../api/personal_access_tokens.md#list-personal-access-tokens)
+     to list tokens that have expired recently. For example, go to `https://gitlab.com/api/v4/personal_access_tokens`,
+     and locate tokens with a specific `expires_at` date.
+   - For project access tokens, use the
+     [project access tokens API](../api/project_access_tokens.md#list-project-access-tokens)
+     to list recently expired tokens.
+   - For group access tokens, use the
+     [group access tokens API](../api/group_access_tokens.md#list-group-access-tokens)
+     to list recently expired tokens.
+1. Create a new access token:
+   - For personal access tokens, [use the UI](../user/profile/personal_access_tokens.md#create-a-personal-access-token)
+     or [Users API](../api/users.md#create-a-personal-access-token).
+   - For a project access token, [use the UI](../user/project/settings/project_access_tokens.md#create-a-project-access-token)
+     or [project access tokens API](../api/project_access_tokens.md#create-a-project-access-token).
+   - For a group access token, [use the UI](../user/group/settings/group_access_tokens.md#create-a-group-access-token-using-ui)
+     or [group access tokens API](../api/group_access_tokens.md#create-a-group-access-token).
+1. Replace the old access token with the new access token. This process varies
+   depending on how you use the token, for example if configured as a secret or
+   embedded within an application. Requests made from this token should no longer
+   return `401` responses.
+
+## Troubleshooting
+
+### Identify personal, project, and group access tokens expiring on a certain date
+
+Access tokens that have no expiration date are valid indefinitely, which is a
+security risk if the access token is divulged.
+
+To manage this risk, when you upgrade to GitLab 16.0 and later, any
+[personal](../user/profile/personal_access_tokens.md),
+[project](../user/project/settings/project_access_tokens.md), or
+[group](../user/group/settings/group_access_tokens.md) access
+token that does not have an expiration date automatically has an expiration
+date set at one year from the date of upgrade.
+
+If you are not aware of when your tokens expire because the dates have changed,
+you might have unexpected authentication failures when trying to sign into GitLab
+on that date.
+
+To manage this issue, you can use a [tool that assists with analyzing, extending, or remove token expiration dates](../administration/raketasks/tokens/index.md).
+
+If you cannot run the tool, you can also run scripts in self-managed instances to identify
+tokens that either:
+
+- Expire on a specific date.
+- Have no expiration date.
+
+You run these scripts from your terminal window in either:
+
+- A [Rails console session](../administration/operations/rails_console.md#starting-a-rails-console-session).
+- Using the [Rails Runner](../administration/operations/rails_console.md#using-the-rails-runner).
+
+The specific scripts you run differ depending on if you have upgraded to GitLab 16.0
+and later, or not:
+
+- If you have not yet upgraded to GitLab 16.0 or later, [identify tokens that do not have an expiration date](#find-tokens-with-no-expiration-date).
+- If you have upgraded to GitLab 16.0 or later, use scripts to identify any of
+  the following:
+  - [Tokens expiring on a specific date](#find-all-tokens-expiring-on-a-specific-date).
+  - [Tokens expiring in a specific month](#find-tokens-expiring-in-a-given-month).
+  - [Dates when many tokens expire](#identify-dates-when-many-tokens-expire).
+
+After you have identified tokens affected by this issue, you can run a final script
+to [extend the lifetime of specific tokens](#extend-token-lifetime) if needed.
+
+These scripts return results in the following format:
+
+```plaintext
+Expired Group Access Token in Group ID 25, Token ID: 8, Name: Example Token, Scopes: ["read_api", "create_runner"], Last used:
+Expired Project Access Token in Project ID 2, Token ID: 9, Name: Test Token, Scopes: ["api", "read_registry", "write_registry"], Last used: 2022-02-11 13:22:14 UTC
+```
+
+For more information on this, see [incident 18003](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/18003).
+
+#### Find all tokens expiring on a specific date
+
+This script finds tokens that expire on a specific date.
+
+Prerequisites:
+
+- You must know the exact date your instance was upgraded to GitLab 16.0.
+
+To use it:
+
+::Tabs
+
+:::TabTitle Rails console session
+
+1. In your terminal window, connect to your instance.
+1. Start a Rails console session with `sudo gitlab-rails console`.
+1. Depending on your needs, copy either the entire [`expired_tokens.rb`](#expired_tokensrb)
+   or [`expired_tokens_date_range.rb`](#expired_tokens_date_rangerb) script below, and paste it into the console.
+   Change the `expires_at_date` to the date one year after your instance was upgraded to GitLab 16.0.
+1. Press <kbd>Enter</kbd>.
+
+:::TabTitle Rails Runner
+
+1. In your terminal window, connect to your instance.
+1. Depending on your needs, copy either the entire [`expired_tokens.rb`](#expired_tokensrb)
+   or [`expired_tokens_date_range.rb`](#expired_tokens_date_rangerb) script below, and save it
+   as a file on your instance:
+   - Name it `expired_tokens.rb`.
+   - Change the `expires_at_date` to the date one year after your instance was upgraded to GitLab 16.0.
+   - The file must be accessible to `git:git`.
+1. Run this command, changing the path to the _full_ path to your `expired_tokens.rb` file:
+
+   ```shell
+   sudo gitlab-rails runner /path/to/expired_tokens.rb
+   ```
+
+For more information, see the [Rails Runner troubleshooting section](../administration/operations/rails_console.md#troubleshooting).
+
+::EndTabs
+
+##### `expired_tokens.rb`
+
+This script requires you to know the exact date your GitLab instance
+was upgraded to GitLab 16.0.
+
+```ruby
+# Change this value to the date one year after your GitLab instance was upgraded.
+
+expires_at_date = "2024-05-22"
+
+# Check for expiring personal access tokens
+PersonalAccessToken.owner_is_human.where(expires_at: expires_at_date).find_each do |token|
+  puts "Expired Personal Access Token ID: #{token.id}, User Email: #{token.user.email}, Name: #{token.name}, Scopes: #{token.scopes}, Last used: #{token.last_used_at}"
+end
+
+# Check for expiring project and group access tokens
+PersonalAccessToken.project_access_token.where(expires_at: expires_at_date).find_each do |token|
+  token.user.members.each do |member|
+    type = member.is_a?(GroupMember) ? 'Group' : 'Project'
+
+    puts "Expired #{type} access token in #{type} ID #{member.source_id}, Token ID: #{token.id}, Name: #{token.name}, Scopes: #{token.scopes}, Last used: #{token.last_used_at}"
+  end
+end
+```
+
+#### Find tokens expiring in a given month
+
+This script finds tokens that expire in a particular month. You don't need to know
+the exact date your instance was upgraded to GitLab 16.0. To use it:
+
+::Tabs
+
+:::TabTitle Rails console session
+
+1. In your terminal window, start a Rails console session with `sudo gitlab-rails console`.
+1. Paste in the entire [`tokens_with_no_expiry.rb`](#tokens_with_no_expiryrb) script below.
+   If desired, change the `date_range` to a different range.
+1. Press <kbd>Enter</kbd>.
+
+:::TabTitle Rails Runner
+
+1. In your terminal window, connect to your instance.
+1. Copy this entire [`tokens_with_no_expiry.rb`](#tokens_with_no_expiryrb) script below, and save it as a file on your instance:
+   - Name it `expired_tokens_date_range.rb`.
+   - If desired, change the `date_range` to a different range.
+   - The file must be accessible to `git:git`.
+1. Run this command, changing `/path/to/expired_tokens_date_range.rb`
+   to the _full_ path to your `expired_tokens_date_range.rb` file:
+
+   ```shell
+   sudo gitlab-rails runner /path/to/expired_tokens_date_range.rb
+   ```
+
+For more information, see the [Rails Runner troubleshooting section](../administration/operations/rails_console.md#troubleshooting).
+
+::EndTabs
+
+##### `expired_tokens_date_range.rb`
+
+```ruby
+# This script enables you to search for tokens that expire within a
+# certain date range (like 1.month) from the current date. Use it if
+# you're unsure when exactly your GitLab 16.0 upgrade completed.
+
+date_range = 1.month
+
+# Check for personal access tokens
+PersonalAccessToken.owner_is_human.where(expires_at: Date.today .. Date.today + date_range).find_each do |token|
+  puts "Expired Personal Access Token ID: #{token.id}, User Email: #{token.user.email}, Name: #{token.name}, Scopes: #{token.scopes}, Last used: #{token.last_used_at}"
+end
+
+# Check for expiring project and group access tokens
+PersonalAccessToken.project_access_token.where(expires_at: Date.today .. Date.today + date_range).find_each do |token|
+  token.user.members.each do |member|
+    type = member.is_a?(GroupMember) ? 'Group' : 'Project'
+
+    puts "Expired #{type} access token in #{type} ID #{member.source_id}, Token ID: #{token.id}, Name: #{token.name}, Scopes: #{token.scopes}, Last used: #{token.last_used_at}"
+  end
+end
+```
+
+#### Identify dates when many tokens expire
+
+This script identifies dates when most of tokens expire. You can use it in combination with other scripts on this page to identify and extend large batches of tokens that may be approaching their expiration date, in case your team has not yet set up token rotation.
+
+The script returns results in this format:
+
+```plaintext
+42 Personal Access Tokens will expire at 2024-06-27
+17 Personal Access Tokens will expire at 2024-09-23
+3 Personal Access Tokens will expire at 2024-08-13
+```
+
+To use it:
+
+::Tabs
+
+:::TabTitle Rails console session
+
+1. In your terminal window, start a Rails console session with `sudo gitlab-rails console`.
+1. Paste in the entire [`dates_when_most_of_tokens_expire.rb`](#dates_when_most_of_tokens_expirerb) script.
+1. Press <kbd>Enter</kbd>.
+
+:::TabTitle Rails Runner
+
+1. In your terminal window, connect to your instance.
+1. Copy this entire [`dates_when_most_of_tokens_expire.rb`](#dates_when_most_of_tokens_expirerb)
+   script, and save it as a file on your instance:
+   - Name it `dates_when_most_of_tokens_expire.rb`.
+   - The file must be accessible to `git:git`.
+1. Run this command, changing `/path/to/dates_when_most_of_tokens_expire.rb`
+   to the _full_ path to your `dates_when_most_of_tokens_expire.rb` file:
+
+   ```shell
+   sudo gitlab-rails runner /path/to/dates_when_most_of_tokens_expire.rb
+   ```
+
+For more information, see the [Rails Runner troubleshooting section](../administration/operations/rails_console.md#troubleshooting).
+
+::EndTabs
+
+##### `dates_when_most_of_tokens_expire.rb`
+
+```ruby
+PersonalAccessToken
+  .select(:expires_at, Arel.sql('count(*)'))
+  .where('expires_at >= NOW()')
+  .group(:expires_at)
+  .order(Arel.sql('count(*) DESC'))
+  .limit(10)
+  .each do |token|
+    puts "#{token.count} Personal Access Tokens will expire at #{token.expires_at}"
+  end
+```
+
+#### Find tokens with no expiration date
+
+This script finds tokens that lack an expiration date: `expires_at` is `NULL`. For users
+who have not yet upgraded to GitLab version 16.0 or later, the token `expires_at`
+value is `NULL`, and can be used to identify tokens to add an expiration date to.
+
+You can use this script in either the [Rails console](../administration/operations/rails_console.md)
+or the [Rails Runner](../administration/operations/rails_console.md#using-the-rails-runner):
+
+::Tabs
+
+:::TabTitle Rails console session
+
+1. In your terminal window, connect to your instance.
+1. Start a Rails console session with `sudo gitlab-rails console`.
+1. Paste in the entire [`tokens_with_no_expiry.rb`](#tokens_with_no_expiryrb) script below.
+1. Press <kbd>Enter</kbd>.
+
+:::TabTitle Rails Runner
+
+1. In your terminal window, connect to your instance.
+1. Copy this entire [`tokens_with_no_expiry.rb`](#tokens_with_no_expiryrb) script below, and save it as a file on your instance:
+   - Name it `tokens_with_no_expiry.rb`.
+   - The file must be accessible to `git:git`.
+1. Run this command, changing the path to the _full_ path to your `tokens_with_no_expiry.rb` file:
+
+   ```shell
+   sudo gitlab-rails runner /path/to/tokens_with_no_expiry.rb
+   ```
+
+For more information, see the [Rails Runner troubleshooting section](../administration/operations/rails_console.md#troubleshooting).
+
+::EndTabs
+
+##### `tokens_with_no_expiry.rb`
+
+This script finds tokens without a value set for `expires_at`.
+
+   ```ruby
+   # This script finds tokens which do not have an expires_at value set.
+
+   # Check for expiring personal access tokens
+   PersonalAccessToken.owner_is_human.where(expires_at: nil).find_each do |token|
+     puts "Expires_at is nil for Personal Access Token ID: #{token.id}, User Email: #{token.user.email}, Name: #{token.name}, Scopes: #{token.scopes}, Last used: #{token.last_used_at}"
+   end
+
+   # Check for expiring project and group access tokens
+   PersonalAccessToken.project_access_token.where(expires_at: nil).find_each do |token|
+     token.user.members.each do |member|
+       type = member.is_a?(GroupMember) ? 'Group' : 'Project'
+
+       puts "Expires_at is nil for #{type} access token in #{type} ID #{member.source_id}, Token ID: #{token.id}, Name: #{token.name}, Scopes: #{token.scopes}, Last used: #{token.last_used_at}"
+     end
+   end
+   ```
+
+### Extend token lifetime
+
+Delay the expiration of certain tokens with this script.
+
+From GitLab 16.0, all access tokens have an expiration date. After you deploy at least GitLab 16.0,
+any non-expiring access tokens expire one year from the date of deployment.
+
+If this date is approaching and there are tokens that have not yet
+been rotated, you can use this script to delay expiration and give
+users more time to rotate their tokens.
+
+#### Extend lifetime for specific tokens
+
+This script extends the lifetime of all tokens which expire on a specified date, including:
+
+- Personal access tokens
+- Group access tokens
+- Project access tokens
+
+Users that have intentionally set a token to expire on the specified date will have their
+token lifetimes extended as well.
+
+To use the script:
+
+::Tabs
+
+:::TabTitle Rails console session
+
+1. In your terminal window, start a Rails console session with `sudo gitlab-rails console`.
+1. Paste in the entire [`extend_expiring_tokens.rb`](#extend_expiring_tokensrb) script below.
+   If desired, change the `expiring_date` to a different date.
+1. Press <kbd>Enter</kbd>.
+
+:::TabTitle Rails Runner
+
+1. In your terminal window, connect to your instance.
+1. Copy this entire [`extend_expiring_tokens.rb`](#extend_expiring_tokensrb) script below, and save it as a file on your instance:
+   - Name it `extend_expiring_tokens.rb`.
+   - If desired, change the `expiring_date` to a different date.
+   - The file must be accessible to `git:git`.
+1. Run this command, changing `/path/to/extend_expiring_tokens.rb`
+   to the _full_ path to your `extend_expiring_tokens.rb` file:
+
+   ```shell
+   sudo gitlab-rails runner /path/to/extend_expiring_tokens.rb
+   ```
+
+For more information, see the [Rails Runner troubleshooting section](../administration/operations/rails_console.md#troubleshooting).
+
+::EndTabs
+
+##### `extend_expiring_tokens.rb`
+
+```ruby
+expiring_date = Date.new(2024, 5, 30)
+new_expires_at = 6.months.from_now
+
+total_updated = PersonalAccessToken
+                  .not_revoked
+                  .without_impersonation
+                  .where(expires_at: expiring_date.to_date)
+                  .update_all(expires_at: new_expires_at.to_date)
+
+puts "Updated #{total_updated} tokens with new expiry date #{new_expires_at}"
+```