Completed
Spark Data preprocessing
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Realtime Voting System - End-to-End Data Engineering Project
Automatically move to the next video in the Classroom when playback concludes
- 1 Introduction
- 2 The Election Flow
- 3 The System Architecture
- 4 Setting up the Project
- 5 The docker compose for the system architecture
- 6 Candidate, Party and Voters Accredditation
- 7 Voting System
- 8 Streaming votes from/to Kafka
- 9 Reading Voting Data Streams from Kafka with Pyspark
- 10 Spark Data preprocessing
- 11 Spark Aggregations
- 12 Writing aggregated data back to kafka
- 13 Realtime Votes Dashboard with Streamlit
- 14 Results and Live commentaries
- 15 Winner of the Election
- 16 31 Outro
