Building a Global Decentralized Storage Network - Lessons Learned from S3 Alternative Platform

Explore a 38-minute technical conference talk from SDC2022 that delves into the challenges and solutions of building a global, decentralized storage network. Learn how Storj developed an architecture to deliver S3 workload demands while ensuring durability across 13,500+ independent node operators. Discover the intricacies of managing encrypted files for video streaming, handling large-scale network disruptions, and maintaining fault tolerance through erasure coding, encryption, and repair mechanisms. Gain insights into overcoming unique challenges like managing data audits, reputation systems, and performance optimization in a distributed environment. Through practical examples and real-world scenarios, understand how the network maintained file integrity during blackouts and internet access restrictions. Master key concepts including erasure codes implementation, streaming 4K video from encrypted content across thousands of nodes, and strategies for eliminating long-tail effects in a decentralized storage system.