Overview

Class Central Tips

This course will cover various topics in Data Engineering in support of decision support systems, data analytics, data mining, machine learning, and artificial intelligence. You will study on-premises data warehouse architecture, and dimensional modeling of data warehouses.

Syllabus

Database Concepts 1

This module introduces data warehousing and business intelligence, emphasizing their role in enhancing organizational decision-making. Data warehouses transform raw data into actionable insights using processes like ETL (Extract, Transform, Load), supported by tools such as OLAP for querying and data mining. While operational databases (OLTP) are suited for daily transactions, OLAP databases are optimized for complex analytics. To effectively implement data warehousing solutions, it is essential to understand the underlying database design principles. Therefore, the module reviews key concepts related to operational databases, focusing on conceptual database design. We examine Entity Relationship Diagrams (ERD) as a vital tool for conceptual representation, identifying crucial aspects of the database design process that convert business requirements into a conceptual model. In the subsequent module, we will build on this foundation by reviewing logical modeling and the implementation of databases, equipping students with a comprehensive understanding of both the database design process and OLAP systems. This knowledge will serve as a stepping stone as we explore the complexities of data warehouses.

Database Concepts 2

This module builds on the foundations of database design from the previous module focussing on relational database modeling, normalization, and SQL. The readings will guide you in translating a conceptual EER diagram into a relational model, ensuring adherence to normalization principles, particularly aiming for the 3rd Normal Form. We’ll also emphasize understanding primary keys and foreign keys for maintaining data integrity and establishing table relationships. Additionally, you will have the opportunity to create and critique relational models. We’ll then explore SQL basics, covering syntax (SELECT, INSERT, UPDATE, DELETE), querying techniques (WHERE, ORDER BY, JOIN), and operations involving functions and aggregates (COUNT, SUM, AVG, MIN, MAX), which are fundamental in database querying and management. By the end of this module, we expect students to be comfortable with database design, which is essential for implementing an OLTP system.

Data Warehouse Concepts 1

This module provides an introduction to Data Warehouse Concepts. Data warehouses are based on a multidimensional model. We will look closely into the multidimensional model and its representation as data cubes (also known as hypercubes). We’ll examine how different aspects of data are categorized into facts, measures, and dimensions. Dimensions like Product, Time, and Customer are organized hierarchically within a cube, allowing data to be analyzed at various levels of detail.

Data Warehouse Concepts 2

This module continues an introduction to Data Warehouse Concepts. We’ll examine how different aspects of data are categorized into facts, measures, and dimensions. Dimensions like Product, Time, and Customer are organized hierarchically within a cube, allowing data to be analyzed at various levels of detail. Measures such as Quantity and Sales Amount are stored within these cubes, and analysts can navigate through different levels of detail using "rolling up" and "drilling down" techniques. Key concepts like granularity, dimension schema, and member hierarchies are essential in understanding how data is structured and analyzed in multidimensional models. Additionally, principles like disjointness, completeness, and correctness ensure data accuracy and integrity when aggregating information in data cubes, collectively known as summarizability.