Compare revisions

Andrew Blanchard · Ryan Garcia · Ryan Garcia · Quentin Powell · Ryan Garcia · Ryan Garcia
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,15 @@

 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

+## [3.0.5-bb.0]
+### Added
+- Bumped grafana-enterprise-logs tag to 1.4.0
+- Bumped big-bang/base tag to 1.18.0
+
+## [3.0.4-bb.5]
+### Added
+- Added configuration overrides to the GEL values to allow port configuration on the gateway and admin-api services.
+
 ## [3.0.4-bb.4]
 ### Added
 - Added configuration for setting the `kubectl` image

--- a/README.md
+++ b/README.md
 # loki

-![Version: 3.0.4-bb.4](https://img.shields.io/badge/Version-3.0.4--bb.4-informational?style=flat-square) ![AppVersion: v2.5.0](https://img.shields.io/badge/AppVersion-v2.5.0-informational?style=flat-square)
+![Version: 3.0.5-bb.0](https://img.shields.io/badge/Version-3.0.5--bb.0-informational?style=flat-square) ![AppVersion: v2.5.0](https://img.shields.io/badge/AppVersion-v2.5.0-informational?style=flat-square)

 BigBang amalgamation of Grafana upstream charts to provide several ways of deploying Loki; like Prometheus, but for logs.

@@ -117,10 +117,10 @@ helm install loki chart/
 | gel.enabled | bool | `false` | Enable Grafana Enterprise Logs chart |
 | gel.nameOverride | string | `nil` | Overrides the chart's name |
 | gel.fullnameOverride | string | `nil` | Overrides the chart's computed fullname |
-| gel.image | object | `{"pullPolicy":"IfNotPresent","pullSecrets":[],"registry":"registry1.dso.mil","repository":"ironbank/grafana/grafana-enterprise-logs","tag":"1.3.0"}` | Definition of the Docker image for Grafana Enterprise Logs If the image block is overwritten in a custom values file, it is also required to update the values in the `loki-distributed.loki.image` block. This can be done by copying the values, or like here, by using an anchor and a pointer. |
+| gel.image | object | `{"pullPolicy":"IfNotPresent","pullSecrets":[],"registry":"registry1.dso.mil","repository":"ironbank/grafana/grafana-enterprise-logs","tag":"1.4.0"}` | Definition of the Docker image for Grafana Enterprise Logs If the image block is overwritten in a custom values file, it is also required to update the values in the `loki-distributed.loki.image` block. This can be done by copying the values, or like here, by using an anchor and a pointer. |
 | gel.image.registry | string | `"registry1.dso.mil"` | The container registry to use |
 | gel.image.repository | string | `"ironbank/grafana/grafana-enterprise-logs"` | The image repository to use |
-| gel.image.tag | string | `"1.3.0"` | The version of Grafana Enterprise Logs |
+| gel.image.tag | string | `"1.4.0"` | The version of Grafana Enterprise Logs |
 | gel.image.pullPolicy | string | `"IfNotPresent"` | Defines the policy how and when images are pulled |
 | gel.image.pullSecrets | list | `[]` | Additional image pull secrets |
 | gel.serviceAccount | object | `{"create":true}` | Definition of the ServiceAccount for containers Any additional configuration of the ServiceAccount has to be done in `loki-distributed.serviceAccount`. |
@@ -144,12 +144,12 @@ helm install loki chart/
 | gel.tokengen.image.registry | string | `"registry1.dso.mil"` | Registry for kubectl image |
 | gel.tokengen.image.repository | string | `"ironbank/opensource/kubernetes/kubectl"` | Repository for kubectl image |
 | gel.tokengen.image.tag | string | `"v1.22.2"` | Tag for kubectl image |
-| gel.adminApi | object | `{"affinity":{},"annotations":{},"env":[],"extraArgs":{},"extraContainers":[],"extraVolumeMounts":[],"extraVolumes":[],"initContainers":[],"labels":{},"livenessProbe":{"httpGet":{"path":"/ready","port":"http-metrics"},"initialDelaySeconds":45},"nodeSelector":{},"persistence":{"subPath":null},"readinessProbe":{"httpGet":{"path":"/ready","port":"http-metrics"},"initialDelaySeconds":45},"replicas":1,"resources":{},"securityContext":{"runAsGroup":10001,"runAsNonRoot":true,"runAsUser":10001},"service":{"annotations":{},"labels":{}},"strategy":{"type":"RollingUpdate"},"terminationGracePeriodSeconds":60,"tolerations":[]}` | Configuration for the `admin-api` target |
+| gel.adminApi | object | `{"affinity":{},"annotations":{},"env":[],"extraArgs":{},"extraContainers":[],"extraVolumeMounts":[],"extraVolumes":[],"initContainers":[],"labels":{},"livenessProbe":{"httpGet":{"path":"/ready","port":"http-metrics"},"initialDelaySeconds":45},"nodeSelector":{},"persistence":{"subPath":null},"readinessProbe":{"httpGet":{"path":"/ready","port":"http-metrics"},"initialDelaySeconds":45},"replicas":1,"resources":{},"securityContext":{"runAsGroup":10001,"runAsNonRoot":true,"runAsUser":10001},"service":{"annotations":{},"labels":{},"port":3100,"targetPort":3100},"strategy":{"type":"RollingUpdate"},"terminationGracePeriodSeconds":60,"tolerations":[]}` | Configuration for the `admin-api` target |
 | gel.adminApi.replicas | int | `1` | Define the amount of instances |
 | gel.adminApi.extraArgs | object | `{}` | Additional CLI arguments for the `admin-api` target |
 | gel.adminApi.labels | object | `{}` | Additional labels for the `admin-api` Deployment |
 | gel.adminApi.annotations | object | `{}` | Additional annotations for the `admin-api` Deployment |
-| gel.adminApi.service | object | `{"annotations":{},"labels":{}}` | Additional labels and annotations for the `admin-api` Service |
+| gel.adminApi.service | object | `{"annotations":{},"labels":{},"port":3100,"targetPort":3100}` | Additional labels and annotations for the `admin-api` Service |
 | gel.adminApi.securityContext | object | `{"runAsGroup":10001,"runAsNonRoot":true,"runAsUser":10001}` | Run container as user `enterprise-logs(uid=10001)` `fsGroup` must not be specified, because these security options are applied on container level not on Pod level. |
 | gel.adminApi.resources | object | `{}` | Request and limit Kubernetes resources Values are defined in small.yaml and large.yaml |
 | gel.adminApi.extraVolumes | list | `[]` | Additional volumes for Pods |
@@ -158,12 +158,12 @@ helm install loki chart/
 | gel.adminApi.nodeSelector | object | `{}` | Node selector for admin-api Pods |
 | gel.adminApi.tolerations | list | `[]` | Tolerations for admin-api Pods |
 | gel.adminApi.terminationGracePeriodSeconds | int | `60` | Grace period to allow the admin-api to shutdown before it is killed |
-| gel.gateway | object | `{"affinity":{},"annotations":{},"env":[],"extraArgs":{},"extraContainers":[],"extraVolumeMounts":[],"extraVolumes":[],"initContainers":[],"labels":{},"livenessProbe":{"httpGet":{"path":"/ready","port":"http-metrics"},"initialDelaySeconds":45},"nodeSelector":{},"persistence":{"subPath":null},"readinessProbe":{"httpGet":{"path":"/ready","port":"http-metrics"},"initialDelaySeconds":45},"replicas":1,"resources":{},"securityContext":{"fsGroup":10001,"runAsGroup":10001,"runAsNonRoot":true,"runAsUser":10001},"service":{"annotations":{},"labels":{}},"strategy":{"type":"RollingUpdate"},"terminationGracePeriodSeconds":60,"tolerations":[],"useDefaultProxyURLs":true}` | Configuration for the `gateway` target |
+| gel.gateway | object | `{"affinity":{},"annotations":{},"env":[],"extraArgs":{},"extraContainers":[],"extraVolumeMounts":[],"extraVolumes":[],"initContainers":[],"labels":{},"livenessProbe":{"httpGet":{"path":"/ready","port":"http-metrics"},"initialDelaySeconds":45},"nodeSelector":{},"persistence":{"subPath":null},"readinessProbe":{"httpGet":{"path":"/ready","port":"http-metrics"},"initialDelaySeconds":45},"replicas":1,"resources":{},"securityContext":{"fsGroup":10001,"runAsGroup":10001,"runAsNonRoot":true,"runAsUser":10001},"service":{"annotations":{},"labels":{},"port":80,"targetPort":3100},"strategy":{"type":"RollingUpdate"},"terminationGracePeriodSeconds":60,"tolerations":[],"useDefaultProxyURLs":true}` | Configuration for the `gateway` target |
 | gel.gateway.replicas | int | `1` | Define the amount of instances |
 | gel.gateway.extraArgs | object | `{}` | Additional CLI arguments for the `gateway` target |
 | gel.gateway.labels | object | `{}` | Additional labels for the `gateway` Pod |
 | gel.gateway.annotations | object | `{}` | Additional annotations for the `gateway` Pod |
-| gel.gateway.service | object | `{"annotations":{},"labels":{}}` | Additional labels and annotations for the `gateway` Service |
+| gel.gateway.service | object | `{"annotations":{},"labels":{},"port":80,"targetPort":3100}` | Additional labels and annotations for the `gateway` Service |
 | gel.gateway.securityContext | object | `{"fsGroup":10001,"runAsGroup":10001,"runAsNonRoot":true,"runAsUser":10001}` | Run container as user `enterprise-logs(uid=10001)` |
 | gel.gateway.resources | object | `{}` | Request and limit Kubernetes resources Values are defined in small.yaml and large.yaml |
 | gel.gateway.extraVolumes | list | `[]` | Additional volumes for Pods |
@@ -190,7 +190,7 @@ helm install loki chart/
 | bbtests.cypress.artifacts | bool | `true` |  |
 | bbtests.cypress.envs.cypress_check_datasource | string | `"false"` |  |
 | bbtests.cypress.envs.cypress_grafana_url | string | `"http://monitoring-grafana.monitoring.svc.cluster.local"` |  |
-| bbtests.scripts.image | string | `"registry1.dso.mil/ironbank/big-bang/base:1.17.0"` |  |
+| bbtests.scripts.image | string | `"registry1.dso.mil/ironbank/big-bang/base:1.18.0"` |  |
 | bbtests.scripts.envs.LOKI_URL | string | `"http://{{ template \"loki.fullname\" . }}.{{ .Release.Namespace }}.svc:3100"` |  |
 | bbtests.scripts.envs.LOKI_VERSION | string | `"{{ .Values.loki.image.tag }}"` |  |


--- a/chart/Chart.yaml
+++ b/chart/Chart.yaml
 apiVersion: v2
 name: loki
-version: 3.0.4-bb.4
+version: 3.0.5-bb.0
 appVersion: v2.5.0
 kubeVersion: "^1.10.0-0"
 description: "BigBang amalgamation of Grafana upstream charts to provide several ways of deploying Loki; like Prometheus, but for logs."

--- a/chart/templates/enterprise/admin-api/service-admin-api.yaml
+++ b/chart/templates/enterprise/admin-api/service-admin-api.yaml
@@ -16,9 +16,9 @@ spec:
  type: ClusterIP
  ports:
    - name: http-metrics
-      port: 3100
+      port: {{ .Values.gel.adminApi.service.port }}
      protocol: TCP
-      targetPort: http-metrics
+      targetPort: {{ .Values.gel.adminApi.service.targetPort }}
    - name: grpc
      port: 9095
      protocol: TCP

--- a/chart/templates/enterprise/gateway/deployment-gateway.yaml
+++ b/chart/templates/enterprise/gateway/deployment-gateway.yaml
@@ -60,12 +60,21 @@ spec:
            - -config.file=/etc/loki/config/loki.yaml
            - -log.level=debug
            {{- if .Values.gel.gateway.useDefaultProxyURLs }}
-            - -gateway.proxy.default.url=http://{{ template "loki.gatewayUrl" . }}.{{ .Release.Namespace }}.svc:3100
-            - -gateway.proxy.admin-api.url=http://{{ template "enterprise-logs.adminApiFullname" . }}.{{ .Release.Namespace }}.svc:3100
-            - -gateway.proxy.distributor.url=http://{{ template "loki.gatewayUrl" . }}.{{ .Release.Namespace }}.svc:3100
-            - -gateway.proxy.ingester.url=http://{{ template "loki.gatewayUrl" . }}.{{ .Release.Namespace }}.svc:3100
-            - -gateway.proxy.query-frontend.url=http://{{ template "loki.gatewayUrl" . }}.{{ .Release.Namespace }}.svc:3100
-            - -gateway.proxy.ruler.url=http://{{ template "loki.gatewayUrl" . }}.{{ .Release.Namespace }}.svc:3100
+            {{- if .Values.loki.enabled }} # monolith
+            - -gateway.proxy.default.url=http://{{ template "enterprise-logs.fullname" . }}-admin-api.{{ .Release.Namespace }}.svc:3100
+            - -gateway.proxy.admin-api.url=http://{{ template "enterprise-logs.fullname" . }}-admin-api.{{ .Release.Namespace }}.svc:3100
+            - -gateway.proxy.distributor.url=http://{{ template "enterprise-logs.fullname" . }}.{{ .Release.Namespace }}.svc:3100
+            - -gateway.proxy.ingester.url=http://{{ template "enterprise-logs.fullname" . }}.{{ .Release.Namespace }}.svc:3100
+            - -gateway.proxy.query-frontend.url=http://{{ template "enterprise-logs.fullname" . }}.{{ .Release.Namespace }}.svc:3100
+            - -gateway.proxy.ruler.url=http://{{ template "enterprise-logs.fullname" . }}.{{ .Release.Namespace }}.svc:3100
+            {{- else }} # scalable
+            - -gateway.proxy.default.url=http://{{ template "enterprise-logs.fullname" . }}-admin-api.{{ .Release.Namespace }}.svc:3100
+            - -gateway.proxy.admin-api.url=http://{{ template "enterprise-logs.fullname" . }}-admin-api.{{ .Release.Namespace }}.svc:3100
+            - -gateway.proxy.distributor.url=http://{{ template "enterprise-logs.fullname" . }}-write.{{ .Release.Namespace }}.svc:3100
+            - -gateway.proxy.ingester.url=http://{{ template "enterprise-logs.fullname" . }}-write.{{ .Release.Namespace }}.svc:3100
+            - -gateway.proxy.query-frontend.url=http://{{ template "enterprise-logs.fullname" . }}-read.{{ .Release.Namespace }}.svc:3100
+            - -gateway.proxy.ruler.url=http://{{ template "enterprise-logs.fullname" . }}-read.{{ .Release.Namespace }}.svc:3100
+            {{- end }}
            {{- end }}
            {{- range $key, $value := .Values.gel.gateway.extraArgs }}
            - "-{{ $key }}={{ $value }}"

--- a/chart/templates/enterprise/gateway/service-gateway.yaml
+++ b/chart/templates/enterprise/gateway/service-gateway.yaml
@@ -15,9 +15,9 @@ spec:
  type: ClusterIP
  ports:
    - name: http-metrics
-      port: 3100
+      port: {{ .Values.gel.gateway.service.port }}
      protocol: TCP
-      targetPort: http-metrics
+      targetPort: {{ .Values.gel.gateway.service.targetPort }}
  selector:
    {{- include "enterprise-logs.gatewaySelectorLabels" . | nindent 4 }}
 {{- end }}
--- a/chart/values.yaml
+++ b/chart/values.yaml
@@ -297,7 +297,7 @@ gel:
    # -- The image repository to use
    repository: ironbank/grafana/grafana-enterprise-logs
    # -- The version of Grafana Enterprise Logs
-    tag: 1.3.0
+    tag: 1.4.0
    # -- Defines the policy how and when images are pulled
    pullPolicy: IfNotPresent
    # -- Additional image pull secrets
@@ -379,6 +379,8 @@ gel:
    annotations: {}
    # -- Additional labels and annotations for the `admin-api` Service
    service:
+      port: 3100
+      targetPort: 3100
      labels: {}
      annotations: {}
    # -- Run container as user `enterprise-logs(uid=10001)`
@@ -442,6 +444,8 @@ gel:
    annotations: {}
    # -- Additional labels and annotations for the `gateway` Service
    service:
+      port: 80
+      targetPort: 3100
      labels: {}
      annotations: {}
    # -- Run container as user `enterprise-logs(uid=10001)`
@@ -609,7 +613,7 @@ bbtests:
      cypress_check_datasource: 'false'
      cypress_grafana_url: 'http://monitoring-grafana.monitoring.svc.cluster.local'
  scripts:
-    image: registry1.dso.mil/ironbank/big-bang/base:1.17.0
+    image: registry1.dso.mil/ironbank/big-bang/base:1.18.0
    envs:
      LOKI_URL: 'http://{{ template "loki.fullname" . }}.{{ .Release.Namespace }}.svc:3100'
      LOKI_VERSION: '{{ .Values.loki.image.tag }}'
--- a/docs/gel.md
+++ b/docs/gel.md
+# Assumptions (5/5/2022)
+* Running using BigBang branch https://repo1.dso.mil/platform-one/big-bang/bigbang/-/tree/loki-enterprise
+* Running using Loki branch https://repo1.dso.mil/platform-one/big-bang/apps/sandbox/loki/-/tree/feature/adjustable-service-ports
+
+# Setup
+
+## Installation
+* Install BigBang with [values.yaml](#values)
+
+## GEL Configuration in Grafana
+* Navigate to https://grafana.bigbang.dev/plugins/grafana-enterprise-logs-app
+* Populate fields with:
+  - Access token: Admin token from `kubectl get secret/gel-admin-token -n logging -o json | jq -r '.data.token' | base64 --decode`
+  - Grafana Enterprise Logs URL: http://logging-loki-gel-gateway.logging.svc.cluster.local
+* Click "Enable"
+* Navigate to https://grafana.bigbang.dev/a/grafana-enterprise-logs-app?path=tenants
+* Click Create Tenant, and create a new Tenant
+
+## Create Policy/Token for Promtail
+* Navigate to https://grafana.bigbang.dev/a/grafana-enterprise-logs-app?path=access-policies and create an access policy with the `logs:write` scope, ensure you've selected the tenant you just created.
+* Now create a token for that access policy by clicking 'Add Token' on the policy name. Copy the token and save it for the next step.
+
+## Promtail Configuration
+* Uncomment the promtail section in the [Values](#values)
+* Set `basic_auth.password` to the token created above and use the **tenant name** as the username.
+* Upgrade the BigBang Helm installation to enable promtail
+
+For example:
+```
+promtail:
+  enabled: true
+  values:
+    config:
+      snippets:
+        extraClientConfigs: |
+          basic_auth:
+            username: borg
+            password: cHJvbXRhaWwtcHJvbXRhaWw6ODVzfiM6KkAvOjleMjNWNjNyODRZOFxf
+          tenant_id: borg
+          external_labels:
+            environment: dev
+```
+
+
+## Create Grafana Datasource
+* Navigate to https://grafana.bigbang.dev/a/grafana-enterprise-logs-app?path=access-policies and create an access policy, ensuring to tick the box that you intend to create a data source with this polcy. It should auto populate the required scopes.
+* Click 'Add Token', and then click 'Create', and then click 'Create a datasource' which will create a new datasource pre-configured to use the token.
+
+
+# Future Considerations
+* Grafana Enterprise plugin should be configured automatically, [which is possible](https://grafana.com/docs/grafana/latest/administration/provisioning/#plugins), but enterprise plugin properites are undocumented and the admin token is not known until a `post-install` job is run.
+* Promtail configuration to set auth and tenant info should be automatic, but this depends on the addition of a job to bootstrap a tenant and create a policy plus token.
+* [Loki VirtualService for external cluster access.](https://repo1.dso.mil/platform-one/big-bang/apps/sandbox/loki/-/merge_requests/22)
+* Memberlist seems to be finicky in AWS with atypical internal subnets. [See Here](https://github.com/grafana/helm-charts/issues/157)
+
+# Values
+```
+monitoring:
+  enabled: true
+  grafana:
+    enterprise:
+      enabled: true
+      licenseContents: <GEX License>
+
+loki:
+  # -- Toggle deployment of Loki.
+  enabled: true
+
+  # -- Loki architecture.  Options are monolith and scalable
+  strategy: scalable
+
+  # Must match cluster name in GEL license
+  releaseName: <LICENSE CLUSTER NAME>
+
+  enterprise:
+    enabled: true
+    licenseContents: <GEL LICENSE>
+
+  values:
+    gel:
+      gateway:
+        service:
+          port: 3101
+
+    minio:
+      tenants:
+        buckets:
+          - name: loki-logs
+          - name: loki-admin
+
+    loki-simple-scalable:
+      read:
+        replicas: 1
+      write:
+        replicas: 1
+      gateway:
+        enabled: true
+
+    global:
+      objectStorage:
+        endpoint: minio.logging.svc.cluster.local
+        region: us-east-1
+        bucketnames: loki-logs
+        access_key_id: minio
+        secret_access_key: minio123
+        adminBucketName: loki-admin
+
+      config: |
+        auth:
+          type: enterprise
+        auth_enabled: true
+
+        license:
+          path: "/etc/enterprise-logs/license/license.jwt"
+
+        cluster_name: {{ .Release.Name }}
+
+        server:
+          http_listen_port: 3100
+          grpc_listen_port: 9095
+
+        common:
+          replication_factor: 1
+          storage:
+            filesystem: null
+            s3:
+              bucketnames: {{ .Values.global.objectStorage.bucketnames }}
+              endpoint: {{ .Values.global.objectStorage.endpoint }}
+              region: {{ .Values.global.objectStorage.region }}
+              secret_access_key: {{ .Values.global.objectStorage.secret_access_key }}
+              access_key_id: {{ .Values.global.objectStorage.access_key_id }}
+              insecure: true
+              s3forcepathstyle: true
+
+        memberlist:
+          join_members:
+            - {{ include "loki.fullname" . }}-memberlist-tcp
+
+        admin_client:
+          storage:
+            type: s3
+            s3:
+              bucket_name: {{ .Values.global.objectStorage.adminBucketName }}
+              insecure: true
+              endpoint: {{ .Values.global.objectStorage.endpoint }}
+              secret_access_key: {{ .Values.global.objectStorage.secret_access_key }}
+              access_key_id: {{ .Values.global.objectStorage.access_key_id }}
+
+        compactor:
+          shared_store: s3
+          working_directory: /var/loki/boltdb-shipper-compactor
+          compaction_interval: 30s
+
+        ingester:
+          lifecycler:
+            num_tokens: 512
+          chunk_idle_period: 30m
+          chunk_block_size: 262144
+          chunk_encoding: snappy
+          chunk_retain_period: 1m
+          wal:
+            dir: /var/loki/wal
+
+        ingester_client:
+          grpc_client_config:
+            max_recv_msg_size: 104857600
+            max_send_msg_size: 104857600
+
+        limits_config:
+          enforce_metric_name: false
+          reject_old_samples: true
+          reject_old_samples_max_age: 168h
+          max_cache_freshness_per_query: 10m
+
+        frontend:
+          log_queries_longer_than: 10s
+          compress_responses: true
+          tail_proxy_url: http://{{ include "loki.fullname" . }}:3100
+
+        querier:
+          query_ingesters_within: 12h
+
+        query_range:
+          split_queries_by_interval: 24h
+          align_queries_with_step: true
+          cache_results: true
+          results_cache:
+            cache:
+              memcached:
+                expiration: 1h
+              memcached_client:
+                timeout: 1s
+
+        schema_config:
+          configs:
+            - from: 2021-01-01
+              store: boltdb-shipper
+              object_store: aws
+              schema: v11
+              index:
+                prefix: index_
+                period: 24h
+
+        storage_config:
+          aws:
+            endpoint: {{ .Values.global.objectStorage.endpoint }}
+            bucketnames: {{ .Values.global.objectStorage.bucketnames }}
+            access_key_id: {{ .Values.global.objectStorage.access_key_id }}
+            secret_access_key: {{ .Values.global.objectStorage.secret_access_key }}
+            region: {{ .Values.global.objectStorage.region }}
+            s3forcepathstyle: true
+            insecure: true
+          boltdb_shipper:
+            active_index_directory: /var/loki/index
+            cache_location: /var/loki/cache
+            cache_ttl: 24h
+            shared_store: s3
+
+        ruler:
+          storage:
+            type: s3
+            s3:
+              bucketnames: {{ .Values.global.objectStorage.bucketnames }}
+          enable_alertmanager_discovery: false
+          enable_api: true
+          enable_sharding: true
+          rule_path: /var/loki
+
+# promtail:
+#   enabled: true
+#   values:
+#     config:
+#       snippets:
+#         extraClientConfigs: |
+#           basic_auth:
+#             username: <TENANT_NAME>
+#             password: <TOKEN>
+#           tenant_id: <TENANT_NAME>
+#           external_labels:
+#             environment: dev
+
+addons:
+  minioOperator:
+    enabled: true
+```
\ No newline at end of file
No results found