Skip to content
Snippets Groups Projects
Commit b79b99e2 authored by Pablo Carranza's avatar Pablo Carranza
Browse files

Add guidelines for introducing a change in production

Extracted from the old change checklist
parent a6d6abe3
No related branches found
No related tags found
1 merge request!6392Add guidelines for introducing a change in production
Pipeline #
Loading
Loading
@@ -261,6 +261,50 @@ in the invite, or get in touch in the production chat channel to ask.
Any team or individual can initiate a change to GitLab.com by following this checklist. Create an issue in the infrastructure [issue
tracker](https://gitlab.com/gitlab-com/infrastructure/issues) and select the `change_checklist` template
 
#### Guidelines for executing a change in production
If necessary, follow this guidelines:
##### Downtime
If the change introduces downtime:
* How can we reduce the impact?
* Explain what the estimation is based on.
* Can the change be tested somehow to challenge the downtime assumption?
##### People and Communications
* Always tweet before and after the change in production, if things go wrong people already knows that there's a reason why.
* Define clear roles that will be played while executing the change in production, and who will be playing each role:
* Who will execute the change
* Who will manage communications in twitter, banners, etc.
* Who will write down the timeline in the google doc.
* Any other role that is necessary.
* Consider cutting an advisory ticket to Azure to let them know that we are introducing this change, use it to handle [resources](#resources)
* Raise awareness of your change in the production engineering team.
##### Resources
* Check that we have enough resources to perform the change, check the resources in the Azure portal, for example:
* How many cores will be needed
* How many public ips
* How many disk volumes
* How many hosts.
* Consider cutting a ticket to Azure explaining the change to get their support on planning the change, they will probably know what other resources have limitations.
* Check that you have the necessary versions of software installed in all the involved hosts if applicable.
* Check that you have the necessary permissions level to perform the change.
* Check that your change is not colliding with any other change scheduled in production.
##### Executing the change
* Consider always creating a google doc to track the change, it helps a lot to write things down. And it's really easy to just C&P in the issue when the change is done.
* Consider running preparatory steps and pre-checks way ahead of time to save time.
* Write your timeline using UTC times. We are all in the same timezone in UTC.
##### When things go wrong
Perform a [blameless post mortem](#postmortems)
## Make GitLab.com settings the default
 
As said in the [production engineer job description](jobs/production-engineer/index.html)
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment