Add replication lag alert

This will page us when replication lag goes over 120 seconds

I checked and this would have alerted us 2 weeks ago when we had the increase in replication lag

https://prometheus.gitlab.com/graph?g0.range_input=2w&g0.expr=max(pg_replication_lag%7Bfqdn%3D~%22.%2Bcluster.gitlab.com%22%7D)+by+(fqdn)&g0.tab=0

cc/ @northrup @maratkalibek @omame @jtevnan @ahanselka @yorickpeterse

cc/ @stanhu @bjk-gitlab

merged

mentioned in issue infrastructure#1577 (closed)

mentioned in issue infrastructure#1576 (closed)