Mathematics of Machine Learning: An Introduction - Lecture 1

Overview

Explore the foundations of machine learning in this comprehensive lecture by Sanjeev Arora from Princeton University. Delve into the mathematical formulations of various learning types, including supervised, unsupervised, and interactive learning. Gain insights into the philosophical and scientific issues surrounding machine learning. Examine key concepts such as curve fitting, gradient descent, and deep learning models. Discover applications in natural language processing, game playing, and brain fMRI decoding. Investigate the challenges of optimization and overfitting in deep learning. Conclude with a discussion on the future of machine learning and its potential impact in India. Engage in a thought-provoking Q&A session to further enhance your understanding of this rapidly evolving field.

Syllabus

Date & Time: Tuesday, 12 February,
Date & Time: Tuesday, 12 February,
Date & Time: Wednesday, 13 February,
ICTS-TIFR: An Overview
ICTS and its Mandate
The ICTS Campus - Imagined 2012
The ICTS Campus - Realised 2017
ICTS Research - Structure
ICTS Programs
What ICTS is Not
ICTS Programs - Format
ICTS Programs - Duration
ICTS Programs - Organisation
ICTS Programs - Directions
ICTS Programs - Numbers
ICTS Programs - A Sampling
ICTS Outreach - Initiatives
ICTS Outreach-Kaapi with Kuriosity
Thank You See You Again at ICTS
What is machine learning and deep learning?
Machine learning ML: A new kind of science
Talk overview
Part 1 - Mathematical formalization of Machine Learning ML
Old Idea: Curve fitting Legendre, Gauss, c. 1800
Example: Learning to score reviews
Example: Learning to rate reviews contd
ML ~ finding suitable function "model" given examples of desired input/output behavior
Formal framework
Training via Gradient Descent "natural algorithm"
Subcase: deep learning* deep models = "multilayered"
Summary so far:
Unsupervised learning no human-supplied labels
A Language model baby "word2ver" [Mikolov et al'1 3]
Properties of semantic word vectors
Sequential decision-making framework
Game-playing via Deep Learning crude account of Alpha-Go Zero
Part 3 - Toward mathematical understanding of Deep Learning
Special case: deep learning deep = "multilayered"
Some key questions
Analysis of optimization
Black box analysis sketch
More about optimization in next talk, including recent works using trajectory analysis for gradient descent
Why no overfitting?
Part 4 - Taking stock, wrapping up
1. Imitation approach has not worked well in the past: airplanes, chess/go etc.
Sample Task: "Decoding" Brain fMRI [Vodrahalli et al, Neurolmage'17]
Brain regions useful for decoding
Can Machine Learning thrive in India?
Concluding thoughts on ML
Q&A