Data Modeling, Storage, and Pipelines

This unit develops foundational skills in designing and optimizing data structures and in building reliable data pipelines. Students learn relational modeling (normalization and denormalization), storage architectures (databases, warehouses, marts, lakes, lakehouses), pipeline patterns (batch, streaming, Lambda), orchestration with Apache Airflow, and methods to detect and remediate bad data and ensure data quality.

Learning Objectives

• Analyze and compare database and analytical storage architectures (databases, data warehouses, data marts, data lakes, and lakehouses) to recommend appropriate solutions for given use cases
• Apply data modeling techniques, including normalization and denormalization, and implement schema and indexing strategies to optimize database performance
• Demonstrate the design and deployment of batch and stream data pipelines, explain Lambda architecture patterns, and build and schedule an ETL workflow using Apache Airflow in a lab
• Evaluate data quality dimensions, identify sources of bad data, and apply validation and cleansing strategies to improve pipeline reliability and trustworthiness