Hyper Speed - When Big Data Blooms

Explore the challenges of processing massive data volumes in this NDC Security 2022 conference talk. Dive into the infrastructure of Report URI, capable of handling nearly 65,000 JSON payloads per second from global clients. Discover the power of Bloom Filters, an underutilized data structure that enables processing incredible amounts of data with remarkable speed and space efficiency. Learn about cross-site scripting, cryptojacking, Magecart, and JavaScript security issues. Understand deduplication techniques, probabilistic data structures, and advanced concepts like counting Bloom Filters, TopK, and HyperLogLog. Gain insights into handling one million reports daily, implementing safe browsing features, and managing per-user data efficiently.