STAT 28000. Optimization
Department of Statistics
University of Chicago
Spring 2017
This is an introductory course on optimization that will cover the
rudiments of unconstrained and constrained optimization of a real-valued
multivariate function. The focus is on the settings where this function
is, respectively, linear, quadratic, convex, or differentiable. Time
permitting, topics such as nonsmooth, integer, vector, and dynamic
optimization may be briefly addressed. Materials will include basic
duality theory, optimality conditions, and intractability results, as well
as algorithms and applications.
Announcements
- 05/26/17: Reminder: Quiz II from 3:00–4:20pm on Tue, May 30, in
Eckhart 133.
- 05/26/17: Lecture 18 notes posted.
- 05/23/17: Office hours from 3:00–5:00pm on Wed, May 24, in Jones
122B. Greg has moved his office hours this week to 1:00–3:00pm
on Fri, May 26, in Gates-Blake 401.
- 05/23/17: Lecture 17 notes posted.
- 05/18/17: Lecture 16 notes posted.
- 05/17/17: Lecture 15 notes posted.
- 05/12/17: Lecture 14 notes and Homework 4 posted.
- 05/09/17: Office hours from 1:00–3:00pm on Mon, May 10, in Jones
122B. Greg will also hold his regular office hours from
4:00–5:00pm in Jones 226.
- 05/09/17: Lecture 13 notes posted.
- 05/05/17: Lecture 12 notes posted.
- 05/02/17: Lecture 11 notes and Homework 3 posted.
- 04/28/17: Lecture 10 notes posted. Greg will hold office hours from
2:00–4:00pm on Fri, Apr 28, in Cobb 102.
- 04/27/17: Office hours from 1:00–3:00pm on Mon, May 1, in Jones
122B.
- 04/25/17: Lecture 9 notes posted.
- 04/18/17: Lecture 8 notes and Homework 2 posted.
- 04/18/17: Reminder: Quiz I from 3:00–4:20pm on Thu, Apr 20, in Eckhart 133.
- 04/14/17: Lecture 7 notes posted.
- 04/14/17: Office hours from 2:00–4:00pm on Mon, Apr 17, in Jones
122B.
- 04/11/17: Lecture 6 notes posted. Greg will hold office hours from
3:00–5:00pm on Fri, Apr 14, in Cobb 101.
- 04/10/17: Lecture 5 notes posted.
- 04/06/17: Homework 1 and Lecture 4 notes posted.
- 04/04/17: Lecture 3 notes posted.
- 03/31/17: Lecture 2 notes posted.
- 03/29/17: Lecture 1 notes posted.
- 03/28/17: Make-up lecture Friday, Apr 7, 5:00–8:00pm in Eckhart
133.
- 03/27/17: Quiz I on Thu, Apr 20. Quiz II on Tue, May 30.
- 03/27/17: Check back regularly for announcements.
Lectures
Location: Eckhart
Hall, Room 133.
Times: Tue & Thu, 3:00–4:20pm
Course staff
Instructor: Lek-Heng
Lim
Office: Jones 122B
lekheng(at)galton.uchicago.edu
Tel: (773) 702-4263
Office hours: Two-hour session before problem set is due, Jones
122B
Course Assistant: Greg
Naisat
Office: Jones 226
gregn(at)galton.uchicago.edu
Office hours: Wed, 4:00–5:00pm
Grader I: Dongyue
Xie
dyxie(at)galton.uchicago.edu
Grader II: Yiling
You
ylyou(at)galton.uchicago.edu
Syllabus
- Optimization of a univariate real-valued function
- Multivariate real-valued functions
- Higher-order derivatives of multivariate functions
- Multivariate Taylor expansion with remainder in mean-value form and
integral form
- Unconstrained optimization
- Newton, quasi-Newton, conjugate gradient, and steepest descent
methods
- Constrained optimization
- Implicit function theorem and Lagrange multipliers
- KKT conditions
- Nonlinear optimization
- Convex sets and convex functions
- Convex optimization
- Newton and steepest descent methods revisited
- Linear programming
- Basic duality theory
- Brief overview of nonsmooth, integer, vector, and dynamic
optimization
Collaborations are permitted but you will need to write up your own
solutions and declare your collaborators. The problem sets are designed to
get progressively more difficult. You will get at least six days for each
problem set.
You are required to implement your own programs for problems that
require some amount of simple coding (using Matlab, Mathematica, R,
or SciPy).
Bug report on the problem sets:
lekheng(at)galton.uchicago.edu
Supplementary materials
Grades and quizzes
Grade composition: 60% Problem Sets, 40% Quizzes.
Midterm exams: Quiz I on Thu, Apr 20, 3:00–4:20pm,
Eckhart 133. Quiz II on Tue, May 30, 3:00–4:20pm, Eckhart 133.
Closed book, closed notes, no cheat sheet.
References
We will not use any specific textbook but will use selected material
from the following references, all of which would be accessible to
undergraduates.
You may download all these books online from an UChicago IP address or
via ProxyIt! if you are off-campus.