Efficient Inference and Fine-Tuning

This unit covers techniques to produce concise model outputs, scale inference, and optimize models for deployment. Students will learn summarization and batch prompting strategies, parameter-efficient fine-tuning and general optimization methods, and how to weigh cost versus performance when selecting models for Python-based solutions.

Learning Objectives

• Analyze summarization strategies and prompt designs to produce accurate, concise outputs and compare results using quantitative metrics (e.g., length, fidelity, relevance).
• Apply batch prompting techniques in Python to increase inference throughput and measure improvements in latency and resource utilization.
• Demonstrate parameter-efficient fine-tuning and model optimization methods (e.g., adapters, LoRA, quantization) using Python toolchains to reduce compute and storage while maintaining task performance.
• Evaluate cost-performance trade-offs across optimization and model-selection approaches and justify a recommended model and configuration for a given deployment constraint using quantitative evidence.