ePOWERAI
CoursesOptimization

Optimization

Optimization

First-order methods and convex toolkit for training models — from gradient descent to Adam, with enough convexity to know what your optimizer can and cannot guarantee.

5 modules · 14 lessons · runs in your browser · no account

Your progress00 / 14 · 0%
1 lesson here is on the Guided Foundations Path· Step 10

Course contents

Foundations

  1. W01Week 1: Optimization for Learning
  2. W02Week 2: Convex Sets and Functions
  3. W03Week 3: Gradient Descent Theory

Model problems

  1. W04Week 4: Least Squares and Conditioning
  2. W05Week 5: Classification Losses

First-order stack

  1. W06Week 6: Stochastic Gradient Descent
  2. W07Week 7: Momentum and Acceleration
  3. W08Week 8: Adaptive First-Order Methods
  4. W09Week 9: Practical Training Dynamics

Structure

  1. W10Week 10: Nonsmooth and Composite Objectives
  2. W11Week 11: Constraints: Projections and Penalties
  3. W12Week 12: Duality for Practitioners

Nonconvex practice

  1. W13Week 13: Nonconvex Deep Learning
  2. W14Week 14: Capstone — Theory Meets Training