Description
The course assumes students are comfortable with analysis, probability, statistics, and basic programming. This course will cover core concepts in machine learning and statistical inference. The ML concepts covered are spectral methods (matrices and tensors), non-convex optimization, probabilistic models, neural networks, representation theory, and generalization. In statistical inference, the topics covered are detection and estimation, sufficient statistics, Cramer-Rao bounds, Rao-Blackwell theory, variational inference, and multiple testing. In addition to covering the core concepts, the course encourages students to ask critical questions such as: How relevant is theory in the age of deep learning? What are the outstanding open problems? Assignments will include exploring failure modes of popular algorithms, in addition to traditional problem-solving type questions.
General Information
Outline
Lecture 1: Introduction, Probability
Lecture 2: Sufficient statistics
Lecture 3: Bayesian
Lecture 4: Neyman Pearson
Lecture 5: Sequential detection
Lecture 6: Estimation and UMVU
Lecture 7: Cramer Rao
Lecture 8: Midterm exam
Lecture 9: Spectral Methods: PCA/CCA, HMM
Lecture 10: Spectral Methods: Tensor methods, method of moments
Lecture 11: Optimization: Non-convex
Lecture 12: Optimization in deep learning: Adam, CGD, MAdam
Lecture 13: generalization theory
Lecture 14: generalization theory
Lecture 15: approximation theory
Lecture 16: operator learning
Lecture 17: operator learning
Final presentation: Saturday 9am-1pm, March 18th.
Lecture 2: Sufficient statistics
Lecture 3: Bayesian
Lecture 4: Neyman Pearson
Lecture 5: Sequential detection
Lecture 6: Estimation and UMVU
Lecture 7: Cramer Rao
Lecture 8: Midterm exam
Lecture 9: Spectral Methods: PCA/CCA, HMM
Lecture 10: Spectral Methods: Tensor methods, method of moments
Lecture 11: Optimization: Non-convex
Lecture 12: Optimization in deep learning: Adam, CGD, MAdam
Lecture 13: generalization theory
Lecture 14: generalization theory
Lecture 15: approximation theory
Lecture 16: operator learning
Lecture 17: operator learning
Final presentation: Saturday 9am-1pm, March 18th.
Location
ANB 213
Time
Class: Tuesday, Thursday 13:00-14:25
Office hour: TBD
Office hour: TBD
Grading
Homework Assignments 30%
Project 50%
Quiz 20% (1 in-class quiz)
Project 50%
Quiz 20% (1 in-class quiz)
Recitation schedule
Week 1: software tool
Week 2: probability
Week 3: statistics
Week 4: linear algebra
Week 5: tensor
Week 2: probability
Week 3: statistics
Week 4: linear algebra
Week 5: tensor
Name | Office Hours | |
---|---|---|
Anima Anandkumar | When? Where? | |
Jiawei Zhao | When? Where? | |
Zongyi | When? Where? | |
Kaiyu Yang | When? Where? | |
Peter Wang | When? Where? | |
Julius | When? Where? | |
Bahareh Tolooshams | When? Where? | |
Rafal | When? Where? |