AI Evaluation, Readings, and ROI

This capstone unit synthesizes prior Data Science and Behavioral AI learning by focusing on experimental evaluation, return-on-investment (ROI) measurement, and critical engagement with advanced literature on classical machine learning, deep learning, generative AI, and biologically inspired models. Students consolidate notes from unit readings, design experiments to measure impact, and prepare actionable deployment and evaluation plans that address performance, cost, and ethical considerations.

Learning Objectives

• Analyze experimental designs (including A/B and controlled trials) to define hypotheses, select appropriate evaluation metrics, calculate expected lift and ROI, and interpret statistical significance for AI/ML interventions
• Evaluate and compare applications of classical machine learning, modern machine learning, deep learning, generative AI, and biologically inspired models with respect to performance trade-offs, data and compute requirements, interpretability, and ethical/privacy concerns
• Apply synthesis and academic-note practices to produce an organized portfolio summarizing key chapter notes and additional readings, including annotated bibliographies and concise recommendations for model selection
• Demonstrate the ability to produce a capstone deployment and evaluation plan that recommends models, outlines monitoring and experiment strategies, estimates costs and ROI, and identifies risk-mitigation and bias-reduction measures