Explore the principles and applications of distributed system design in data engineering through this informative 34-minute talk. Gain insights into key concepts such as data partitioning, replication, fault tolerance, scalability, and data consistency. Examine real-world examples to understand best practices and trade-offs in building robust, large-scale data systems. Learn about different replication models, synchronous vs. asynchronous replication, and the concept of quorums. Delivered by Touseef Liaqat, a seasoned software engineer and entrepreneur, this talk provides a comprehensive overview of distributed systems and their crucial role in modern data engineering challenges.
Distributed System Design for Data Engineering
Overview
Syllabus
– Introduction
– What is a Distributed System
– Key concepts in distributed systems
– Fault Tolerance
– Replication
– Synchronous VS Asynchronous Replication
– Replication Models
– Quorums
Taught by
Data Science Dojo
