Investigate how to provide production access to team leads for troubleshooting
We are moving to an SRE world in which development will be heavily involved in running the things we build. Because of this we will need to find a way of providing troubleshooting tooling to the owners of the services, first by providing direct access, and as we move forward by helping building observable systems in which we don't need to reach into production to understand what a system is doing.
So, as a first step, we are considering the option to provide VPN and ssh access to engineering team leads. This issue is to gather feedback and ideas on how this can happen.
So far we know the following tenets:
- Don't block people, audit instead.
- Make access as specific as possible (deploy host, not the DB host or redis unless strictly required)