Optimization Algorithms: Theory and Software Implementation

Overview

ABOUT THE COURSE:The course will introduce various iterative algorithms used to numerically solve the unconstrained and constrained optimization problems. Each algorithm will be introduced with examples and a python code that implements the algorithm. The focus will be on introducing the theory behind each algorithm, and implementing it using python.Programming assignments will consist of applications of these algorithms in fields such as machine learning, econometrics, and game theory.INTENDED AUDIENCE: PG/ PhD students from the department of mathematics/ statistics/economic sciences who wish to learn various optimization algorithms along with its software implementation.PREREQUISITES: A basic knowledge on optimization theory is expectedINDUSTRY SUPPORT: Many firms value the knowledge of python coding. Knowledge on implementing optimization algorithms using python would be an added advantage

Syllabus

Week 1: Introduction to optimization. Need for iterative algorithms.Week 2:Line Search Algorithms. Implementation of exact and backtracking line search.Week 3:Descent Algorithms. Implementation of steepest descent algorithm.Week 4:Need for conjugate gradient algorithm. Implementation. Week 5:Newton’s method. Advantages. Damped Newton method. Implementation.Week 6:Quasi-Newton methods. Rank-one correction, DFP, BFGS methods. Implementation.Week 7:Optimization with constraints. Linear program. Simplex method. Implementation.Week 8:Interior point methods. Karmakar’s algorithm. Implementation.Week 9:Nonlinear optimization. Projected Gradient Descent. Implementation.Week 10:Penalty methods. Barrier methods. Implementation.Week 11:Augmented Lagrangian Method. Implementation.Week 12:Applications of optimization algorithms in machine learning, econometrics, game theory.