Data Mining: Frequent Directions and Random Projections - Spring 2023

Learn about dimensionality reduction techniques in this university lecture that explores random projections, PCA/SVD, and frequent directions algorithms. Begin with a thorough recap of random projection motivation and algorithmic implementation, including methods for choosing random unit vectors. Compare and contrast random projection with PCA/SVD approaches, before diving into an in-depth discussion of frequent directions. Understand the Misra-Gries algorithm for frequent items and its relationship to dimensionality reduction. Conclude with a practical example examining Linformer, which demonstrates the real-world application of SVD and random projection techniques in modern machine learning architectures.