University of Chicago

Spring 2019

This is an introductory course on optimization that will cover the rudiments of unconstrained and constrained optimization of a real-valued multivariate function. The focus is on the settings where this function is, respectively, linear, quadratic, convex, or differentiable. Time permitting, topics such as nonsmooth, integer, vector, and dynamic optimization may be briefly addressed. Materials will include basic duality theory, optimality conditions, and intractability results, as well as algorithms and applications.

- 05/30/19: Lecture Notes 16 posted.

- 05/28/19: Office hours Thu, May 30, 2:00–3:00pm

- 05/24/19: Lecture Notes 15 posted.

- 05/21/19: No lecture on Tue, May 28, but TA will be in Eckhart 133 from 3:30–3:45pm to pick up Homework 4.

- 05/21/19: Lecture Notes 14 posted.

- 05/18/19: Office hours Fri, May 24, 2:00–4:00pm

- 05/16/19: Lecture Notes 13 posted.

- 05/14/19: Homework 4 and Lecture Notes 12 posted.

- 05/09/19: Lecture Notes 11 posted.

- 05/08/19: Office hours Thu, May 9, 2:00–3:00pm and Mon, May 13, 2:00–4:00pm.

- 05/07/19: Lecture Notes 10 posted.

- 05/02/19: Lecture Notes 9 posted.

- 04/30/19: Homework 3 posted.

- 04/30/19: Lecture Notes 7 and 8 posted.

- 04/26/19: Office hours Mon, Apr 29, 1:00–2:30pm.

- 04/20/19: Office hours Tue, Apr 23, 2:00–3:00pm.

- 04/19/19: Lecture Notes 5 and 6 posted.

- 04/18/19: Homework 2 posted.

- 04/17/19: Lecture Notes 4 posted. Video of tonight's make-up lecture available (same password).

- 04/16/19: Office hours Wed, Apr 17, 3:00–5:00pm; reminder: make-up lecture 6:00–8:00pm.

- 04/11/19: Lecture Notes 3 posted. Video of yesterday's make-up lecture available.

- 04/10/19: Lecture Notes 2 posted.

- 04/09/19: Make-up lectures 6:00–8:00pm on Weds, Apr 10 and 17 in Eckhart 133.

- 04/09/19: Homework 1 posted.

- 04/09/19: Lecture Notes 0 and 1 posted.

- 03/29/19: No lectures in the first week, i.e., class will not meet Apr 2 and 4.

- 03/29/19: Check back regularly for announcements.

**Location:** Eckhart
Hall, Room 133.

**Times:** Tue & Thu, 3:30–4:50pm

**Instructor:** Lek-Heng
Lim

Office: Jones 122C

`lekheng(at)galton.uchicago.edu`

Tel: (773) 702-4263

Office hours: Two hours in weeks when a problem set is due, one hour in
other weeks. Venue: Jones 122C. Times would be announced by email.

**Course Assistant I:** Zhipeng
Lou

`zplou(at)galton.uchicago.edu`

Office hours: 5:30–7:00pm Mon, Jones 226

**Course Assistant II:** Zhisheng
Xiao

`zxiao(at)uchicago.edu`

Office hours: 10:00–11:20am, Tue, Searle 206

**Course Assistant III:** Lijia
Zhou

`zlj(at)galton.uchicago.edu`

Office hours: 10:30–11:50am, Wed, Jones 304

- Optimization of a univariate real-valued function
- Multivariate real-valued functions
- Higher-order derivatives of multivariate functions
- Multivariate Taylor expansion with remainder in mean-value form and integral form
- Unconstrained optimization
- Newton, quasi-Newton, conjugate gradient, and steepest descent methods
- Constrained optimization
- Implicit function theorem and Lagrange multipliers
- KKT conditions
- Nonlinear optimization
- Convex sets and convex functions
- Convex optimization
- Methods of penalty function, augmented Lagrangian, and barrier function
- Basic duality theory
- Brief overview of nonsmooth, integer, vector, and dynamic optimization

Collaborations are permitted but you will need to write up your own solutions and declare your collaborators. The problem sets are designed to get progressively more difficult. You will get at least six days for each problem set.

You are required to implement your own programs for problems that require some amount of simple coding (using Matlab, Mathematica, R, or SciPy).

- Problem Set 4 (posted: May 14, due: May 28)

- Problem Set 3 (posted: Apr 30, due: May 14)

- Problem Set 2 (posted: Apr 18, due: Apr 30)

- Problem Set 1 (posted: Apr 9, due: Apr 18)

**Bug report** on the problem sets:
`lekheng(at)galton.uchicago.edu`

- Course homepages from Spring 2018, Spring 2017, Spring 2016, Spring 2015.

- These are slightly more advanced versions of this course, intended primarily for graduate students: Stat 31015 Winter 2015, Stat 31020 Winter 2009–2012.

**Grade composition:** 60% Problem Sets, 40% Quizzes.

**Quizzes:** Quiz I on Thu, Apr 25, 3:30–4:50pm,
Eckhart 133. Quiz II on Tue, Jun 4, 3:30–4:50pm,
Eckhart 133.
Closed book, closed notes, no cheat sheet.

We will not use any specific textbook but will use selected material from the following references, all of which would be accessible to undergraduates.

You may download all these books online from an UChicago IP address or via ProxyIt! if you are off-campus.

- E. Çinlar and R. J. Vanderbei, Real and Convex Analysis, Springer, 2013.

- J. E. Dennis and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, SIAM, 1996.

- G. Hurlbert, Linear Optimization: The Simplex Workbook, Springer 2010.

- J. Nocedal and S. J. Wright, Numerical Optimization, 2nd Ed, Springer, 2006.

- P. Pedregal, Introduction to Optimization, Springer, 2004.