Petabyte Scale Logging with Fluentd and Fluent Bit - A Use Case From Intuit

Explore petabyte-scale logging solutions using Fluentd and Fluent Bit in this conference talk from KubeCon + CloudNativeCon North America 2021. Discover how Intuit tackled challenges in scaling up logging for their Kubernetes-deployed applications, including decreased throughput, higher resource consumption, and unexpected backpressure. Learn about leveraging CNCF projects to configure reliable buffering for network outages, implement flexible configurations for backpressure management, and avoid common mistakes. Gain insights into Intuit's journey to achieve petabyte-scale logging, covering topics such as Fluentd basics, architecture patterns, log transport, offloading, cost breakdown, and test results. This presentation offers valuable knowledge for developers and end-users working with large-scale observability systems in financial and software companies.