Learn how to design and implement a data warehouse from scratch in this comprehensive end-to-end data engineering tutorial. Explore the process of creating a logical data warehouse architecture, setting up an AWS Redshift cluster, converting logical models to physical implementations, loading data, and implementing the medallion architecture. Gain hands-on experience with AWS services, including VPC, Redshift, Glue Data Catalog, and S3, while also learning to use tools like DBeaver and dbt for data modeling and transformation. Follow along as the instructor guides you through each step, from designing dimension and fact tables to creating silver and gold models in a real-world business scenario.
Overview
Syllabus
Introduction
System Prerequisites
Steps Involved in Designing a Data Warehouse
The Business Usecase
Designing the Logical Architecture
Creating a VPC on AWS
Creating Redshift Data Warehouse Cluster
Creating Subnet Group on AWS
Creating Security Group and allowing external connections on AWS
Connecting to Redshift Cluster with DBeaver
Connecting to Redshift Cluster with Redshift Query Editor
Creating Dimensions and Fact data
Loading data into Data Warehouse
Creating AWS Data Catalog DB and Tables
Connecting to Redshift to AWS Glue Data Catalog
Creating DBT project
Configuration connections to Redshift from DBT
DBT Project configuration with Variables and Schema
Creating Silver Dimension models
Creating Silver Fact models
Creating Gold Dimension and Fact Models
Other course information
Taught by
CodeWithYu
Related Courses
-
Build a Data Warehouse in AWS
-
Data Engineering on AWS - A Data Warehouse Solution
-
Data Engineering, Serverless ETL & BI on Amazon Cloud
-
Reddit Data Pipeline Engineering with AWS - End-to-End Data Engineering
-
Smart City End-to-End Realtime Data Engineering Project - AWS Data Engineer
-
Building Realtime Data Warehouses from Scratch - End-to-End Data Engineering Project
