NLP Deception and Defense

This unit examines how rule-based, statistical, and emergent language models produce convincing or deceptive outputs, explores philosophical and ethical questions about machine ‘sentience’ and empathy, and investigates practical risks such as social engineering and CAPTCHA evasion. It builds on foundational concepts from Unit 1 and prepares students for Unit 3’s focus on defensive controls, mitigation strategies, and policy responses.

Learning Objectives

• Analyze architectural and operational differences between rule-based and statistical language models (including emergent behaviors and RLHF enhancements) and determine how those differences affect predictability, attack surfaces, and exploitability.
• Evaluate real-world scenarios in which conversational systems produce persuasive but misleading outputs—such as simulated empathy, false claims of sentience, or social-engineering vectors—and assess the implications for user trust and cybersecurity risk.
• Demonstrate the ability to detect and characterize automated agents through controlled Turing-style probes, behavioral heuristics, and defensive CAPTCHA assessments, and document findings with recommended mitigation strategies.
• Formulate ethical, legal, and policy recommendations addressing deployment, transparency, and accountability of conversational AI in networked environments to guide safe and responsible use.