Elixir for Data Engineering - Batch and Stream Processing

Elixir for Data Engineering - Batch and Stream Processing

Databricks via YouTube Direct link

Scheduled Job framework written in El • Handles coordination of jobs across the Erlang nodies in a service rerunning failed jobs and persisting of status logs

3 of 10

3 of 10

Scheduled Job framework written in El • Handles coordination of jobs across the Erlang nodies in a service rerunning failed jobs and persisting of status logs

Class Central Classrooms beta

YouTube videos curated by Class Central.

Classroom Contents

Elixir for Data Engineering - Batch and Stream Processing

Automatically move to the next video in the Classroom when playback concludes

  1. 1 Intro
  2. 2 Services architecture • Services configured as Erlang clusters with nodes. • Nodes deployed on containers • Nodes running the service will spawn Erlang processes
  3. 3 Scheduled Job framework written in El • Handles coordination of jobs across the Erlang nodies in a service rerunning failed jobs and persisting of status logs
  4. 4 Steps of a scheduled job workflow Airflow
  5. 5 Scheduled jobs in Application start()
  6. 6 Case study: Notification view analytic
  7. 7 Case study: Notification analysis
  8. 8 Setting up Kafka Ex 1. Add mix dependency to build
  9. 9 Supervisor module to listen on consur
  10. 10 GenServer consumer

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.