Cloud Data Engineering

Overview

Welcome to the third course in the Building Cloud Computing Solutions at Scale Specialization! In this course, you will learn how to apply Data Engineering to real-world projects using the Cloud computing concepts introduced in the first two courses of this series. By the end of this course, you will be able to develop Data Engineering applications and use software development best practices to create data engineering applications. These will include continuous deployment, code quality tools, logging, instrumentation and monitoring. Finally, you will use Cloud-native technologies to tackle complex data engineering solutions. This course is ideal for beginners as well as intermediate students interested in applying Cloud computing to data science, machine learning and data engineering. Students should have beginner level Linux and intermediate level Python skills. For your project in this course, you will build a serverless data engineering pipeline in a Cloud platform: Amazon Web Services (AWS), Azure or Google Cloud Platform (GCP).

Syllabus

Getting Started with Cloud Data Engineering

This week, you will learn about the methodologies involved in Data Engineering. You will also learn to evaluate best practices for dealing with the end of Moore’s Law, develop distributed systems that apply software engineering best practices and evaluate best practices for implementing solutions with Big Data. You will apply these practices to build a GPU programming project using Numba and the CUDA SDK.

Examining Principles of Data Engineering

This week, you will learn what Data Engineering is and how to use software engineering best practices in Data Engineering. You will then apply this knowledge by building a command-line data processing tool.

Building Data Engineering Pipelines

This week, you will learn serverless data engineering techniques and data governance best practices. You will then apply this knowledge by building a serverless Data Engineering system.

Applying Key Data Engineering Tasks

This week, you will learn about key Data Engineering tasks including ETL, Cloud Databases and Cloud Storage. You will then apply this knowledge by building a serverless AWS lambda function that labels an image using the AWS Rekognition API.