Cloud and Distributed Data Systems

This capstone unit integrates distributed systems, cloud computing, and practical data engineering skills to prepare students for entry-level roles. Students will apply distributed processing, file format selection, pipeline design, SQL/Spark operations, cloud cost optimization, and system design principles while reflecting on career pathways and role expectations.

Learning Objectives

• Analyze distributed systems and processing frameworks (including Apache Spark) to recommend architectures that meet specified scalability, fault tolerance, and latency requirements
• Apply knowledge of big-data file types and SQL to design and implement efficient data pipelines that optimize query performance and storage footprint
• Demonstrate cloud computing concepts, service and management models, and cost-optimization strategies by provisioning and estimating resources for a sample data workload
• Evaluate types of data engineering roles and produce a targeted plan (resume, portfolio artifact, and interview strategy) to pursue an entry-level data engineering position