Remove BB created servicemonitor and enable upstream ones

mentioned in merge request !128 (merged)

added gitlab priority6 teamXForce labels

marked this issue as related to dsop/gitlab/gitlab/gitlab-exporter#36 (closed)

added statusblocked label

changed iteration to Big Bang Iterations Jul 26, 2022 - Aug 8, 2022

removed statusblocked label

assigned to @gmiernicki

added statusdoing label

added monitoring label and removed gitlab label

added gitlab label

created branch 131-remove-bb-created-servicemonitor-and-enable-upstream-ones to address this issue

removed iteration Big Bang Iterations Jul 26, 2022 - Aug 8, 2022

changed iteration to Big Bang Iterations Aug 9, 2022 - Aug 22, 2022

mentioned in merge request !148 (merged)

unassigned @gmiernicki

per @kevin.wilder 's advice, I've unassigned this issue. I will be OOO back on the 23rd. Someone else may work on this. However, Kevin and I already spent some time working on this, so please check with at least one of us before proceeding.

Branch and MR >> https://repo1.dso.mil/platform-one/big-bang/apps/developer-tools/gitlab/-/merge_requests/148

removed statusdoing label

The new upstream service monitor templates have

{{- if .Capabilities.APIVersions.Has "monitoring.coreos.com/v1" -}}

But it is not returning true when monitoring is deployed. Hopefully we don't have to hack all the SM templates.

From a quick check I did see the same issue as above. However... it doesn't seem like the capabilities conditional on its own is problematic. Test by deploying monitoring + any other helm chart with this template:

{{- if .Capabilities.APIVersions.Has "monitoring.coreos.com/v1" }}
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: test
  namespace: {{ $.Release.Namespace }}
spec:
  namespaceSelector:
    matchNames:
      - {{ $.Release.Namespace }}
  selector:
    matchLabels:
      app: test
  endpoints:
    - port: http-metrics
      path: '/metrics'
{{- end }}

You should find the test servicemonitor created. That leads me to thinking that the other conditional on these servicemonitors is the real problem (or at least getting the right values to enable that).

To make the other conditions return true will need to modify the top-level values.yaml. Example for gitaly

gitlab:
  gitaly:
    metrics:
      enabled: true
      serviceMonitor:
        enabled: true

assigned to @kevin.wilder

added statusdoing label

changed milestone to %1.41.0

having trouble with prometheus servicemonitor metrics for gitlab-shell
https://gitlab.com/gitlab-org/charts/gitlab/-/issues/3448
going to try gitlab-ssh
https://gitlab.com/gitlab-org/charts/gitlab/-/issues/3448
but... "gitlab-sshd is in Alpha. It is not ready for production use."
Too much trouble to get working. Needs ssh keys provided.