Smart City End-to-End Realtime Data Engineering Project - AWS Data Engineer

Smart City End-to-End Realtime Data Engineering Project - AWS Data Engineer

CodeWithYu via YouTube Direct link

Introduction

1 of 22

1 of 22

Introduction

Class Central Classrooms beta

YouTube videos curated by Class Central.

Classroom Contents

Smart City End-to-End Realtime Data Engineering Project - AWS Data Engineer

Automatically move to the next video in the Classroom when playback concludes

  1. 1 Introduction
  2. 2 System Architecture
  3. 3 Project Setup
  4. 4 Docker containers setup and coding
  5. 5 IOT services producer
  6. 6 Vehicle information Generator
  7. 7 GPS Information Generator
  8. 8 Traffic information Generator
  9. 9 Weather information Generator
  10. 10 Emergency Incident Generator
  11. 11 Producing IOT Data to Kafka
  12. 12 AWS S3 setup with policies
  13. 13 AWS IAM Roles and Credentials Management
  14. 14 Apache Spark Realtime Streaming from Kafka
  15. 15 Fixing Schema Issues in Apache Spark Structured Streaming
  16. 16 AWS Glue Crawlers
  17. 17 Working with AWS Athena
  18. 18 Loading Data into Redshift from AWS Glue Data Catalog
  19. 19 Connecting and Querying Redshift DW with DBeaver
  20. 20 Connecting Redshift to AWS Glue Catalog
  21. 21 Fixing IAM Permission issues with Redshift
  22. 22 Outro

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.