How WebAssembly Simplifies Streaming Data Pipelines

Explore how WebAssembly (Wasm) simplifies streaming data pipelines in this 45-minute Linux Foundation webinar. Dive into the challenges of integrating a Wasm runtime into Redpanda, a high-performance streaming storage engine. Learn about the unique constraints of the Seastar framework's execution environment and how to support arbitrary user functions while maintaining strict memory consumption and latency bounds. Discover the benefits of eliminating data ping-pong between systems, reducing latency and resource usage. Gain insights into Redpanda's architecture, including async thread-per-core design, reactor loops, and shared-nothing allocator. Understand the implementation of Wasm transforms, preallocated memory, and asynchronous host functions. Explore related technologies such as Envoy proxy and Open Policy Agent.