Efficient LLM Customization and Inference

This unit introduces methods for adapting large language models with limited compute through parameter-efficient fine-tuning, explores cost/performance trade-offs, and covers practical inference strategies. Students will learn prompt engineering, caching with vector stores, and chaining approaches for long documents to prepare models for efficient, real-world Python-based applications.

Learning Objectives

• Analyze trade-offs between full fine-tuning and parameter-efficient fine-tuning (PEFT) methods by comparing parameter counts, training time, and inference latency in example experiments
• Apply at least one PEFT method (for example LoRA or adapters) in Python to adapt a pre-trained model and report measured changes in performance and resource usage
• Demonstrate effective inference and prompt-engineering techniques, including temperature/settings adjustments and chain-of-thought or document-chaining strategies, to improve output relevance on multi-page inputs
• Evaluate caching and vector-store retrieval strategies by designing and testing a retrieval/caching workflow that reduces redundant calls and improves average response time