High-Performance Data Engineering with Kafka and Spark - Processing 1.2 Billion Records Per Hour

High-Performance Data Engineering with Kafka and Spark - Processing 1.2 Billion Records Per Hour

CodeWithYu via YouTube Direct link

Introduction

1 of 13

1 of 13

Introduction

Class Central Classrooms beta

YouTube videos curated by Class Central.

Classroom Contents

High-Performance Data Engineering with Kafka and Spark - Processing 1.2 Billion Records Per Hour

Automatically move to the next video in the Classroom when playback concludes

  1. 1 Introduction
  2. 2 High Level Architecture Whiteboard
  3. 3 Data Storage Estimation with workings!
  4. 4 Clean Architecture
  5. 5 System Architecture
  6. 6 System Architecture Setup and Coding
  7. 7 Python Producer 😩
  8. 8 Java Producer yay! 😁
  9. 9 300,000 records per second!
  10. 10 Apache Spark Consumer
  11. 11 Spark Job Optimisation and Statistics
  12. 12 Cluster Health issues
  13. 13 Part 1 Outro

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.