Settings for .com restored from rails console

I did:

irb(main):001:0> ApplicationSetting.find(1).dup.tap { |k| k.id=nil }.save
=> true

at 20:15 to account for settings being cleaned during the deploy.

Slack discussion start: https://gitlab.slack.com/archives/C101F3796/p1507233411000125

changed the description

Since I missed the time where the deploy was done, @gl-infra please dump your ideas on how we can improve/prevent/monitor this from happening in the future. We did this later than we could do as our attention was shifted to https://gitlab.com/gitlab-com/infrastructure/issues/2934 and https://gitlab.com/gitlab-com/infrastructure/issues/2932, so we need an automated procedure here.

I went in after @ilyaf restored the settings and verified that key things like:

• Sentry
• Akismet
• ReCaptcha
• Slack
• Repo Quota Settings

all were correctly enabled and functioning.

This restore was needed due to https://gitlab.com/gitlab-org/gitlab-ce/issues/34728

Since I missed the time where the deploy was done, @gl-infra please dump your ideas on how we can improve/prevent/monitor this from happening in the future. We did this later than we could do as our attention was shifted to https://gitlab.com/gitlab-com/infrastructure/issues/2934 and https://gitlab.com/gitlab-com/infrastructure/issues/2932, so we need an automated procedure here.

Can we implement a callback that will send a notification or emit a metric (or failing that, a log message) when the model is changed? Maybe something like https://github.com/collectiveidea/audited ?****

A big help would be when deploying to staging, making sure that the settings are not overwritten there. It should be effected the same way as prod, correct?

@jtevnan not really, this seems to be a race condition in the application.

I think that the best we can do (for now and from the infra side) is monitor for changes, which we should be doing anyway.

added app bug label

mentioned in issue #2932 (closed)

marked this issue as related to #2949 (closed)