Enable Thanos Object Level Storage

Package Merge Request

Package Changes

Enable Thanos Object Storage Upload to S3

• updated bigbang/values.yaml with Object Store for S3
• updated bigbang/chart/template/thanos/values.yaml for storegateway
  • with Object Store for S3
  • customServiceEntry
• updated bigbang/chart/template/monitoring/values.yaml for thanos-sidecar with Object Store for S3
  • with Object Store for S3
  • customServiceEntry

Package MR

N/A

For Issue

Closes big-bang/product/packages/thanos#49 (closed)

Upgrade Notices

N/A

added 1 commit

• 24d21954 - add service entry for s3 object

Compare with previous version

added 1 commit

• c4cffd1e - fixed values in thanos service entry for s3

Compare with previous version

added 1 commit

• 0c2bb26e - fixed the indendation in thanos serviceentry.yaml

Compare with previous version

added 1 commit

• e3708a53 - added thanos value for storegateway-endpoint

Compare with previous version

added 1 commit

• 290d8a86 - removed the serviceEntry.yaml

Compare with previous version

added 1 commit

• 7478dc2a - added secret for thanos-sidecar

Compare with previous version

added 1 commit

• aca6fd8c - added service entry

Compare with previous version

added 1 commit

• 21328c2e - fixed the nil pointer in monitoring and thanos istio.hardened values

Compare with previous version

added 1 commit

• 8b7f6f09 - fixed dig for values.yaml monitoring

Compare with previous version

added 1 commit

• 8c3d5003 - added quotes

Compare with previous version

added 1 commit

• eb445699 - add query endpoint

Compare with previous version

added 1 commit

• 20b3b5df - fixed the nil pointer for storegateway.containerPorts

Compare with previous version

added 1 commit

• fca18913 - fixed the dualstack in thanos/values.yaml

Compare with previous version

added 1 commit

• 3ec08232 - fixed monitoring customserviceEntry

Compare with previous version

added 2 commits

• fd67513d - fix endpoints
• 574d27d0 - fix endpoints

Compare with previous version

added 1 commit

• 716c76ac - removed dualstack from monitoring serviceEntry host

Compare with previous version

added 17 commits

• 716c76ac...5482ba02 - 16 commits from branch master
• ea46a4d0 - Merge branch 'master' into thanos-sidecar

Compare with previous version

assigned to @kliu

changed the description

changed milestone to %2.30.0

added kindenhancement teamObservability labels

requested review from @jimmyungerman, @w.griffing, @blairbowden, @steven.donald, and @piotr.machaj

Confirmed S3 uploads using thanos-sidecar

Confirm Thanos query points to Thanos-storegateway endpoint with healthy upstream

values.yaml used for testing

domain: dev.bigbang.mil

flux:
  interval: 1m
  rollback:
    cleanupOnFail: false

clusterAuditor:
  enabled: false

gatekeeper:
  enabled: true

grafana:
  enabled: true

istioOperator:
  enabled: true

istio:
  enabled: true

monitoring:
  enabled: true
  istio:
    enabled: true
    hardened:
      enabled: true

addons:
  thanos:
    enabled: true
    objectStorage:
      # -- S3 compatible endpoint to use for connection information.
      # examples: "https://s3.amazonaws.com" "https://s3.us-gov-west-1.amazonaws.com" "http://minio.minio.svc.cluster.local:9000"
      endpoint: "s3.us-gov-west-1.amazonaws.com"

      # -- S3 compatible region to use for connection information.
      region: "us-gov-west-1"

      # -- Access key for connecting to object storage endpoint.
      accessKey: "ACCESS_KEY"

      # -- Secret key for connecting to object storage endpoint.
      # Unencoded string data. This should be placed in the secret values and then encrypted
      accessSecret: "ACCESS_SECRET"

      # -- Bucket Name for Thanos
      # examples: "thanos-metrics"
      bucket: "test_bucket"
    values:
      istio:
        enabled: true
        hardened:  
          enabled: true

kyverno:
  enabled: true

kyvernoPolicies:
  enabled: true
kyvernoPolicies:
  values:
    exclude:
      any:
      # Allows k3d load balancer to bypass policies.
      - resources:
          namespaces:
          - istio-system
          names:
          - svclb-*
    policies:
      restrict-host-path-mount-pv:
        parameters:
          allow:
          - /var/lib/rancher/k3s/storage/pvc-*
      

marked this merge request as ready

added statusreview label

requested review from @andrewshoell, @chris.oconnell, @ryan.thompson.44, and @michaelmartin

removed statusreview label

added statusreview label

added 5 commits

• ea46a4d0...4f9606aa - 4 commits from branch master
• 6d8b95ea - Merge branch 'master' into thanos-sidecar

Compare with previous version

changed the description

added thanos label

thanos failed in both pipes

added statusdoing label and removed statusreview label

added 3 commits

• 6d8b95ea...4995b938 - 2 commits from branch master
• 4d542844 - Merge branch 'master' into thanos-sidecar

Compare with previous version

marked this merge request as draft

added debug label

added 1 commit

• 1dc45a82 - restored values

Compare with previous version

added 7 commits

• 1dc45a82...13556b83 - 6 commits from branch master
• 2b54cad1 - rebase master

Compare with previous version

added 1 commit

• ac4b9137 - fixed istio.hardened value in monitoring chart/values.yaml

Compare with previous version

added 1 commit

• 70e6739f - fixed objectsorage if

Compare with previous version

added 1 commit

• d5a1cc2e - fixed not argument for deployment objectStorage endpoint

Compare with previous version

added 1 commit

• 8a4d083b - fixed not in thanos/values.yaml

Compare with previous version

removed statusdoing label

added statusreview label

removed statusreview label

added statusdoing label

Looking into why dashboard isn't able to get data for Kubernetes / Compute Resources / Clusters when the s3 storage is enabled

added 1 commit

• 28026f98 - remove else for thanos/prometheus

Compare with previous version

added 1 commit

• a3579d34 - reverted back to else

Compare with previous version

added 17 commits

• a3579d34...71accffe - 14 commits from branch master
• 34c47052 - appended endpoints for thanos-query to thanos-store and grafana
• 94d08b82 - Merge branch 'master' into thanos-sidecar
• 1382ffd1 - added datasources for grafana for thanos

Compare with previous version

added 1 commit

• 4dea960d - corrected svc for thanos-sidecar svc

Compare with previous version

added 1 commit

• 9b13721a - fixed the prometheus svc

Compare with previous version

added 1 commit

• c1325272 - corrected the svc for prometheus for grafana endpoint

Compare with previous version

added 1 commit

• 09a2dd5c - fixed svc of thanos-sidecar

Compare with previous version

added 1 commit

• 643c00cd - added thanos-query-frontend

Compare with previous version

added 9 commits

• 643c00cd...583600ee - 8 commits from branch master
• 4469606d - Merge branch 'master' into thanos-sidecar

Compare with previous version

added 11 commits

• 4469606d...fade28db - 8 commits from branch master
• 21b2ea83 - removed endpoint for storegateway
• 3d69f9f4 - Merge branch 'master' into thanos-sidecar
• eeb8a063 - revert to else for grafana thanos-query/prometheus

Compare with previous version

added 5 commits

• eeb8a063...d0ee48b9 - 4 commits from branch master
• e22c8985 - Merge branch 'master' into thanos-sidecar

Compare with previous version

added 5 commits

• e22c8985...6f093745 - 4 commits from branch master
• 2b291b27 - Merge branch 'master' into thanos-sidecar

Compare with previous version

added statusreview label and removed statusdoing label

marked this merge request as ready

removed debug label

added thanos label and removed thanos label

resolved all threads

approved this merge request

merged

mentioned in commit 752fd797