Model Stealing for Low-Rank Language Models Through Reverse Engineering

Learn about groundbreaking research in AI cybersecurity and data privacy through a 28-minute technical video exploring methods for reverse-engineering Large Language Models (LLMs). Delve into MIT's innovative approach for efficiently learning and sampling from low-rank distributions over sequences, featuring detailed explanations of Hidden Markov Models, barycentric spanners, and convex optimization techniques. Master the mathematical foundations behind a novel method that uses conditional queries and dimensionality reduction to reconstruct transition models and generate sequences mimicking LLM behavior. Follow along as MIT researchers demonstrate how to capture essential features of complex language models without requiring access to their parameters or training data. Progress through key concepts including KL divergence, low-rank distributions, and the mathematical theorems underpinning this breakthrough in AI model analysis.