STAT 28000. Optimization
Department of Statistics
University of Chicago
Spring 2016
This is an introductory course on optimization that will cover the
rudiments of unconstrained and constrained optimization of a real-valued
multivariate function. The focus is on the settings where this function
is, respectively, linear, quadratic, convex, or differentiable. Time
permitting, topics such as nonsmooth, integer, vector, and dynamic
optimization may be briefly addressed. Materials will include basic
duality theory, optimality conditions, and intractability results, as well
as algorithms and applications.
Announcements
- 06/08/16: Office hours 2:00–3:30pm on Thu, Jun 09.
- 05/30/16: Vivak will hold office hours 3:00–3:50pm today in
the Regenstein cafe.
- 05/25/16: Office hours 1:00–2:30pm on Fri, May 27.
- 05/17/16: Homework 4 posted, due May 31 in class.
- 05/17/16: Reminder: Midterm Exam this Thursday, May 19,
3:00–4:20pm, Eckhart 133. Closed book, closed notes, no cheat
sheet.
- 05/12/16: Office hours 2:30–4:00pm on Fri, May 13 and
1:00–2:30pm on Mon, May 16.
- 05/05/16: Homework 3 posted, due May 17 in class.
- 05/05/16: Make-up lecture on Mon, May 9, 5:30–7:30pm
in E133.
- 04/26/16: Office hours 1:00–3:00pm on Mon, May 2 in Jones 122B.
- 04/25/16: Reminder: Midterm Exam this Thursday, Apr 28,
3:00–4:20pm, Eckhart 133. Closed book, closed notes, no cheat
sheet.
- 04/24/16: Handout 1 providing some linear algebra background posted.
- 04/19/16: Homework 2 posted, due May 3 in class.
- 04/14/16: Office hours 3:30–5:00pm on Fri, Apr 15 and
2:00–3:30pm on Mon, Apr 18.
- 04/06/16: Homework 1 posted, due Apr 19 in class.
- 04/06/16: Class will meet as usual on Thu, Apr 7.
- 04/01/16: Exam dates: Midterm I on Apr 28, Midterm II on May 19,
3:00–4:20pm, in E133.
- 03/30/16: Make-up lectures on Apr 4 and Apr 11, 5:30–7:30pm in
E133.
- 03/30/16: Make-up lecture originally scheduled for today (Mar 30) has
been rescheduled.
- 03/29/16: Class will not meet on Thu, Apr 21.
- 03/29/16: Check back regularly for announcements.
Lectures
Location: Eckhart
Hall, Room 133.
Times: Tue & Thu, 3:00–4:20pm
Course staff
Instructor: Lek-Heng
Lim
Office: Jones 122B
lekheng(at)galton.uchicago.edu
Tel: (773) 702-4263
Office hours: Two 1.5-hour sessions before problem set is due, Jones
122B
Course Assistant: Vivak
Patel
Office: Ryerson N375
vpatel(at)galton.uchicago.edu
Office hours: 3:00–3:50pm, Harper Memorial 150
Syllabus
- Optimization of a univariate real-valued function
- Multivariate real-valued functions
- Higher-order derivatives of multivariate functions
- Multivariate Taylor expansion with remainder in mean-value form and
integral form
- Unconstrained optimization
- Newton, quasi-Newton, conjugate gradient, and steepest descent
methods
- Constrained optimization
- Implicit function theorem and Lagrange multipliers
- KKT conditions
- Nonlinear optimization
- Convex sets and convex functions
- Convex optimization
- Newton and steepest descent methods revisited
- Linear programming
- Basic duality theory
- Brief overview of nonsmooth, integer, vector, and dynamic
optimization
Collaborations are permitted but you will need to write up your own
solutions and declare your collaborators. The problem sets are designed to
get progressively more difficult. You will get at least six days for each
problem set.
You are required to implement your own programs for problems that
require some amount of simple coding (using Matlab, Mathematica, R,
or SciPy).
Bug report on the problem sets:
lekheng(at)galton.uchicago.edu
Supplementary materials
Grades and Midterms
Grade composition: 60% Problem Sets, 40% Midterms.
Midterm exams: Midterm I on Apr 28, 3:00–4:20pm, Eckhart
133. Midterm II on May 19, 3:00–4:20pm, Eckhart 133. Closed book,
closed notes, no cheat sheet.
References
We will not use any specific textbook but will use selected material
from the following references, all of which would be accessible to
undergraduates.
You may download all these books online from an UChicago IP address or
via ProxyIt! if you are off-campus.