FluentBit CrashLoopBackOff with "too many open files" error

Big Bang Version 1.36.0
FluentBit Image: ironbank/opensource/fluent/fluent-bit:1.9.3

Pod Logs

fluent-bit Fluent Bit v1.9.3            
fluent-bit * Copyright (C) 2015-2022 The Fluent Bit Authors
fluent-bit * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd 
fluent-bit * https://fluentbit.io
fluent-bit 
fluent-bit [2022/07/05 16:20:10] [error] [parser] parser named 'apache' already exists, skip.
fluent-bit [2022/07/05 16:20:10] [ info] [fluent bit] version=1.9.3, commit=, pid=1
fluent-bit [2022/07/05 16:20:10] [ info] [storage] version=1.2.0, type=memory+filesystem, sync=normal, checksum=disabled, max_chunks_up=128
fluent-bit [2022/07/05 16:20:10] [ info] [storage] backlog input plugin: storage_backlog.2
fluent-bit [2022/07/05 16:20:10] [ info] [cmetrics] version=0.3.1 
fluent-bit [2022/07/05 16:20:10] [error] [build/plugins/in_tail/CMakeFiles/flb-plugin-in_tail.dir/compiler_depend.ts:304 errno=24] Too many open files
fluent-bit [2022/07/05 16:20:10] [error] Failed initialize input tail.0
fluent-bit [2022/07/05 16:20:10] [error] [lib] backend failed    

Related:

• https://github.com/fluent/fluent-bit/issues/1777
• https://discuss.newrelic.com/t/relic-solution-infrastructure-log-forwarding-too-many-open-file-issue/150475

Big Bang acceptance criteria:

• Add a new TROUBLESHOOTING.md to the fluentbit package docs
• Include details about the Too many open files issue - links to upstream issues, workarounds with node/host fs limits, etc