06-16-2026, 12:03 AM
597G Theoretical Foundations of Deep Learning
Sanjeev Arora
Summary
The Princeton course COS 597G (Fall 2018), “Theoretical Foundations of Deep Learning,” is a graduate-level class focused on understanding the mathematical and algorithmic principles behind deep learning. It studies why modern neural networks train effectively despite being highly nonconvex and overparameterized, and explores questions such as when gradient-based optimization succeeds, how many samples are needed for learning, and what theoretical limits exist for different architectures.
The course combines lectures, paper readings, and student presentations, covering topics like optimization methods (gradient descent, adaptive learning rates), generalization theory (Rademacher complexity, PAC-Bayes, compression bounds), representation learning, generative models (VAEs and GANs), and the behavior of deep networks in overparameterized regimes. It also connects theory to practice in language models and neural architectures, and includes discussion of adversarial examples and robustness. Overall, the course aims to bridge modern deep learning practice with rigorous theoretical understanding through research-focused study and participation.
COURSE PAGE
Sanjeev Arora
Summary
The Princeton course COS 597G (Fall 2018), “Theoretical Foundations of Deep Learning,” is a graduate-level class focused on understanding the mathematical and algorithmic principles behind deep learning. It studies why modern neural networks train effectively despite being highly nonconvex and overparameterized, and explores questions such as when gradient-based optimization succeeds, how many samples are needed for learning, and what theoretical limits exist for different architectures.
The course combines lectures, paper readings, and student presentations, covering topics like optimization methods (gradient descent, adaptive learning rates), generalization theory (Rademacher complexity, PAC-Bayes, compression bounds), representation learning, generative models (VAEs and GANs), and the behavior of deep networks in overparameterized regimes. It also connects theory to practice in language models and neural architectures, and includes discussion of adversarial examples and robustness. Overall, the course aims to bridge modern deep learning practice with rigorous theoretical understanding through research-focused study and participation.
COURSE PAGE
┌────────────────────────────────┐
│ KONSTANTINOS MICHAILIDIS │
└────────────────────────────────┘
│ KONSTANTINOS MICHAILIDIS │
└────────────────────────────────┘