This course is a continuation of Stat 309 and Stat 310. We will discuss topics that we did not have a chance to cover in these two earlier courses. However it is not necessary to have taken Stat 309 or Stat 310 before taking this course.
In Stat 310, we focused almost exclusively on optimization theory. The first part of this course will be on optimization algorithms. These will be for convex optimization as well as for nonlinear optimization. The algorithms will include those for unconstrained optimization (line search methods, trust region methods, quasi-Newton methods, nonlinear conjugate gradient methods) and those for constrained optimization (augmented Lagrangian method, penalty methods, interior point methods). Time permitting, we will also discuss first-order methods, stochastic gradient descent methods, and derivative-free methods.
We will see that the most expensive step in these optimization algorithms is invariably the solution of large systems of linear equations Ax = b where A is often a sparse matrix. This brings us to the subject matter of the second part of this course.In Stat 309, we focused almost exclusively on direct methods. The second part of this course will be on iterative methods in matrix computation. We shall discuss iterative methods for solving linear systems, least squares, eigenvalue, and singular value problems. We will discuss stationary methods (Jacobi, Gauss-Seidel, SOR), semi-iterative methods (Richardson, steepest descent Chebyshev, conjugate gradient), and Krylov subspace methods (MINRES, SYMMLQ, LSQR, GMRES, QMR, BiCG). We will cover some basic ideas for preconditioning and stopping conditions.
Location: Eckhart Hall, Room 117
Times: 10:30–11:50am on Tue/Thu.
Office: Eckhart 122
Tel: (773) 702-4263
Office hours: Wed, 3:30–4:30pm.
Course Assistant: Stefano
Office: Eckhart 8
Office hours: Wed, 2:30–3:30pm.
Problem set will be assigned weekly and will be due the following week. Collaborations are permitted but you will need to write up your own solutions.
Bug report on the problem sets or the solutions: lekheng(at)galton.uchicago.edu
Grade composition: No in-class examination. Grade based entirely on six take-home problem sets.
Boyd & Vandenberghe is freely available online. Nocedal & Wright is freely accessible with your CNetID.