Faster Data Integration Pipeline Execution Using Spark-Jobserver

Faster Data Integration Pipeline Execution Using Spark-Jobserver

Databricks via YouTube Direct link

HTTPS/SSL Enabled Server

22 of 30

22 of 30

HTTPS/SSL Enabled Server

Class Central Classrooms beta

YouTube videos curated by Class Central.

Classroom Contents

Faster Data Integration Pipeline Execution Using Spark-Jobserver

Automatically move to the next video in the Classroom when playback concludes

  1. 1 Intro
  2. 2 Informatica ETL Pipeline
  3. 3 Dealing with buggy pipelines
  4. 4 Data Preview - Feature Requirements
  5. 5 What spark-submit based data preview achieved?
  6. 6 Execution Profiling Results - Spark-submit
  7. 7 Compare Spark-submit with Spark Job Server
  8. 8 Spark-submit based Architecture
  9. 9 SJS based Architecture
  10. 10 Execution Flow
  11. 11 Spark Job Server vs Spark-submit
  12. 12 Setup Details
  13. 13 Getting started
  14. 14 Environment Variables (local.sh. template)
  15. 15 Application Code Migration
  16. 16 WordCount Example
  17. 17 Running Jobs
  18. 18 Handling Job Dependencies
  19. 19 Multiple Spark Job Servers
  20. 20 Concurrency
  21. 21 Support for Kerberos
  22. 22 HTTPS/SSL Enabled Server
  23. 23 Logging
  24. 24 Key Takeaways
  25. 25 Timeouts (in local.conf. template)
  26. 26 Complex Data Representation in Informatica Developer Tool
  27. 27 Monitoring: Binaries
  28. 28 Monitoring: Spark Context
  29. 29 Monitoring: Jobs
  30. 30 Monitoring: Yarn Job

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.