FluentBit CrashLoopBackOff with "too many open files" error
Big Bang Version 1.36.0
FluentBit Image: ironbank/opensource/fluent/fluent-bit:1.9.3
Pod Logs
fluent-bit Fluent Bit v1.9.3
fluent-bit * Copyright (C) 2015-2022 The Fluent Bit Authors
fluent-bit * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
fluent-bit * https://fluentbit.io
fluent-bit
fluent-bit [2022/07/05 16:20:10] [error] [parser] parser named 'apache' already exists, skip.
fluent-bit [2022/07/05 16:20:10] [ info] [fluent bit] version=1.9.3, commit=, pid=1
fluent-bit [2022/07/05 16:20:10] [ info] [storage] version=1.2.0, type=memory+filesystem, sync=normal, checksum=disabled, max_chunks_up=128
fluent-bit [2022/07/05 16:20:10] [ info] [storage] backlog input plugin: storage_backlog.2
fluent-bit [2022/07/05 16:20:10] [ info] [cmetrics] version=0.3.1
fluent-bit [2022/07/05 16:20:10] [error] [build/plugins/in_tail/CMakeFiles/flb-plugin-in_tail.dir/compiler_depend.ts:304 errno=24] Too many open files
fluent-bit [2022/07/05 16:20:10] [error] Failed initialize input tail.0
fluent-bit [2022/07/05 16:20:10] [error] [lib] backend failed
Related:
- https://github.com/fluent/fluent-bit/issues/1777
- https://discuss.newrelic.com/t/relic-solution-infrastructure-log-forwarding-too-many-open-file-issue/150475
Big Bang acceptance criteria:
- Add a new
TROUBLESHOOTING.md
to the fluentbit package docs - Include details about the
Too many open files
issue - links to upstream issues, workarounds with node/host fs limits, etc
Edited by Micah Nagel