3000+ Courses from California Community Colleges

Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Google

Introduction to Data Engineering on Google Cloud

Google via Google Cloud Skills Boost

Go to class Write review

Overview

In this course, you learn about data engineering on Google Cloud, the roles and responsibilities of data engineers, and how those map to offerings provided by Google Cloud. You also learn about ways to address data engineering challenges.

Syllabus

  • Course Introduction
    • Course Introduction
  • Data Engineering Tasks and Components
    • Module Introduction
    • The Role of a Data Engineer
    • Data Sources Versus Data Sinks
    • Data Formats
    • Storage Solution Options on Google Cloud
    • Metadata Management Options on Google Cloud
    • Sharing Datasets using Analytics Hub
    • Lab Intro: Loading Data into BigQuery
    • Loading data into BigQuery
    • Quiz
  • Data Replication and Migration
    • Module Introduction
    • Replication and Migration Architecture
    • The gcloud Command Line Tool
    • Moving Datasets
    • Datastream
    • Lab Intro: Datastream: PostgreSQL Replication to BigQuery
    • Datastream: PostgreSQL Replication to BigQuery
    • Quiz
  • The Extract and Load Data Pipeline Pattern
    • Module Introduction
    • Extract and Load Architecture
    • The bq Command Line Tool
    • BigQuery Data Transfer Service
    • BigLake
    • Lab Intro: BigLake: Qwik Start
    • BigLake: Qwik Start
    • Quiz
  • The Extract, Load, and Transform Data Pipeline Pattern
    • Module Introduction
    • Extract, Load, and Transform (ELT) Architecture
    • SQL Scripting and Scheduling with BigQuery
    • Dataform
    • Lab Intro: Create and Execute a SQL Workflow in Dataform
    • Create and execute a SQL workflow in Dataform
    • Quiz
  • The Extract, Transform, and Load Data Pipeline Pattern
    • Module Introduction
    • Extract, Transform, and Load (ETL) Architecture
    • Google Cloud GUI Tools for ETL Data Pipelines
    • Batch Data Processing Using Dataproc
    • Lab Intro: Use Dataproc Serverless for Spark to Load BigQuery
    • Use Dataproc Serverless for Spark to Load BigQuery
    • Streaming Data Processing Options
    • Bigtable and Data Pipelines
    • Lab Intro: Creating a Streaming Data Pipeline for a Real-Time Dashboard with Dataflow
    • Creating a Streaming Data Pipeline for a Real-Time Dashboard with Dataflow
    • Quiz
  • Automation Techniques
    • Module Introduction
    • Automation Patterns and Options for Pipelines
    • Cloud Scheduler and Workflows
    • Cloud Composer
    • Cloud Run Functions
    • Eventarc
    • Lab Intro: Use Cloud Run Functions to Load BigQuery
    • Use Cloud Run Functions to Load BigQuery
    • Quiz
  • Course Summary
    • Course Summary
    • Course Resources
  • Your Next Steps
    • Course Badge

Tags

Reviews

Start your review of Introduction to Data Engineering on Google Cloud

Go to class

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free
Someone learning on their laptop while sitting on the floor.