Decouple Logging components to allow separate Fluentbit deployment
I have a centralized Elasticsearch I want to ship logs to from other clusters. What I'd like to do in my other clusters is turn off the deployment of a full Elasticsearch deployment, but leave Fluentbit up and running and change the config.service.outputs
section of fluentbit to point to my Elasticsearch to forward logs to.
What we're noticing however is that eck and elasticsearch-kibana are enabled by an if or on whether the individual component itself is enabled, whether logging is enabled or whether clusterAuditor is enabled - https://repo1.dso.mil/platform-one/big-bang/bigbang/-/blob/master/chart/templates/logging/eck-operator/eck-operator-helmrelease.yaml#L2
Theoretically turning off both of these is also not enough to just have Fluentbit come up and be configurable because it depends on on eck being up and running - https://repo1.dso.mil/platform-one/big-bang/bigbang/-/blob/master/chart/templates/logging/fluentbit/fluentbit-helmrelease.yaml#L41
The "get this working right now" is to manually change the dependsOn
in fluent bit and decouple it from ECK but this isn't ideal because now we're modifying the upstream chart which might cause merge conflicts in the future if we move up Big Bang releases and becomes another thing that we have to manage/maintain.
Is there a way to decouple these components? I shouldn't need a full Elasticsearch cluster if I want to ship all my logs to a centralized logging location.