Add template containing Alerts for Cluster Auditor
Currently the opa-scorecard upstream doesn't provide any examples or recommendations for alerts within the prometheus-stack. We will need to create our own alerts within a YAML template inside chart/templates/bigbang
similar to the existing rules that are imported into Prometheus within the monitoring chart at chart/templates/prometheus/rules-1.14/
Example alerts we should configure and write verbose descriptions for are:
-
opa_scorecard_constraint_violations{violation_enforcement="deny"} > 0
This will match any violations that are picked up which don't have exceptions and match a constraint set to the deny action. -
opa_scorecard_constraint_violations{violation_enforcement="dryrun"} > 0
This will match any violations that are picked up which don't have exceptions and match a constraint set to the dryrun action. -
opa_scorecard_up{} < 1
This will throw an alert when opa_scorecard is not properly getting scraped or returning information
These should all be configured with critical
severity labels within the PrometheusRule template.