From Laptop to Lambda - Outsourcing Everyday Jobs to Thousands of Transient Functional Containers

Explore a groundbreaking framework called gg in this 21-minute conference talk from USENIX ATC '19. Discover how gg enables the execution of everyday applications such as software compilation, unit tests, video encoding, and object recognition using thousands of parallel threads on cloud-functions services. Learn about the potential future of computing where users can spawn 10,000 parallel cloud functions to complete large jobs in seconds. Understand how gg expresses jobs as compositions of lightweight OS containers that are transient and functional, and how it manages container instantiation, dependency loading, data movement, and failure handling. Examine the performance comparisons between gg-based applications and conventional tools, including a distributed compiler that outperformed icecc by 2-5 times without requiring a continuously running warm cluster.