Error serializing Prometheus metrics in beta14 deployment
We enabled this on dev and saw a number of errors:
- https://sentry.gitlap.com/gitlab/devgitlaborg/issues/44648/
- https://sentry.gitlap.com/gitlab/devgitlaborg/issues/44649/
Oj::ParseError: Array not terminated at line 1, column 108 [parse.c:894]
from prometheus/client/helper/json_parser.rb:8:in `load'
from prometheus/client/helper/json_parser.rb:8:in `load'
from prometheus/client/formats/text.rb:102:in `block (2 levels) in load_metrics'
from prometheus/client/formats/text.rb:101:in `each'
from prometheus/client/formats/text.rb:101:in `block in load_metrics'
from prometheus/client/formats/text.rb:97:in `each'
from prometheus/client/formats/text.rb:97:in `load_metrics'
from prometheus/client/formats/text.rb:41:in `marshal_multiprocess'
from app/services/metrics_service.rb:14:in `prometheus_metrics_text'
from app/services/metrics_service.rb:24:in `metrics_text'
ArgumentError: @ outside of string
from prometheus/client/mmaped_dict.rb:41:in `unpack'
from prometheus/client/mmaped_dict.rb:41:in `read_all_values'
from prometheus/client/formats/text.rb:101:in `block in load_metrics'
from prometheus/client/formats/text.rb:97:in `each'
from prometheus/client/formats/text.rb:97:in `load_metrics'
from prometheus/client/formats/text.rb:41:in `marshal_multiprocess'
from app/services/metrics_service.rb:14:in `prometheus_metrics_text'
from app/services/metrics_service.rb:24:in `metrics_text'
from app/controllers/metrics_controller.rb:9:in `index'
from action_controller/metal/implicit_render.rb:4:in `send_action'
from abstract_controller/base.rb:198:in `process_action'
from action_controller/metal/rendering.rb:10:in `process_action'
from abstract_controller/callbacks.rb:20:in `block in process_action'
/cc: @pchojnacki, @joshlambert, @bjk-gitlab