Using Serverless Functions for Real-Time Observability

Explore a conference talk from Copenhagen Developers Festival that delves into extending a sub-second latency query engine beyond RAM-only operations to handle extensive data queries using cloud storage and serverless functions. Learn about the evolution of a Scuba-inspired system, from managing local SSD data to processing months of information through cloud infrastructure. Discover critical insights into large-scale lambda management, addressing challenges like impatience and maximum concurrency limits, while understanding the complexities of runtime configuration and architecture experimentation. Gain valuable knowledge about the cost-effectiveness and performance implications of deploying tens of thousands of parallel workers across x86 and ARM ecosystems for real-time observability solutions.