Challenges of Logging on the Edge: Building an Observability Pipeline with FluentBit

Explore the challenges and solutions of implementing logging systems on edge devices in this conference talk from Mickey Pashov of Medtronic Digital Surgery. Discover the journey from arbitrary log collection to building an observability pipeline using FluentBit, including the rationale behind choosing this agent over alternatives. Learn about on-premises deployment hurdles such as firewall configurations, cloud authentication, offline mode, and unfamiliar toolchain packaging. Delve into the complexities of deploying an observability pipeline on Kubernetes with FluentD, covering topics like Helm charts, liveness probes, TLS termination, and autoscaling. Gain insights into advanced concepts such as exponential backoff, memory and file buffering, synthetic transactions, and the use of OpenSearch Prometheus Exporter. Understand the implementation of forwarder and aggregator setups, and explore future work involving S3 archives and MQTT output.