University of Chicago

Fall 2011

This is an introductory course on numerical linear algebra. The course will present a global overview of a number of topics, from classical to modern to state-of-the-art. The fundamental principles and techniques will be covered in depth but towards the end of the course we will also discuss some exciting recent developments.

Numerical linear algebra is quite different from linear algebra. We will be much less interested in algebraic results that follow from the axiomatic definitions of fields and vector spaces but much more interested in analytic results that hold only over the real and complex fields. The main objects of interest are real- or complex-valued matrices, which may come from differential operators, integral transforms, bilinear and quadratic forms, boundary and coboundary maps, Markov chains, graphs, metrics, correlations, hyperlink structures, cell phone signals, DNA microarray measurements, movie ratings by viewers, friendship relations in social networks, etc. Numerical linear algebra provides the mathematical and algorithmic tools for matrix problems that arise in engineering, scientific, and statistical applications.

- 12/04/11: Problem Set 5 posted. Due 4pm, Tue, Dec 13.

- 11/17/11: Office hours Fri, Nov 18, 2:00–3:30.

- 11/14/11: Problem Set 4 posted.

- 11/11/11: Extra lecture on Fri, Nov 11, 1:30–2:50 in Reyerson 358.

- 11/03/11: Office hours Fri, Nov 04, 1:30–3:30.

- 10/29/11: Problem Set 3 posted.

- 10/26/11: Office hours Thu, Oct 27, 2:00–4:00.

- 10/25/11: Check out the SVD song.

- 10/25/11: Extra lecture on Fri, Oct 28, 1:30–2:50 in Reyerson 358.

- 10/20/11: Problem Set 2 posted.

- 10/10/11: Make-up lecture on Fri, Oct 14, 1:30–2:50 in Reyerson 358.

- 10/04/11: Problem Set 1 posted.

- 10/04/11: Lecture notes posted on Chalk.

- 09/30/11: Matlab Tutorial II 6:00–7:00pm, Wed, Oct 6, in Eckhart 133.

- 09/28/11: Matlab Tutorial I 6:00–7:00pm, Thu, Sep 29, in Eckhart 133.

- 09/21/11: Class will meet in Ryerson 358 instead of Eckhart 117.

- 09/20/11: Due to a clash with Stat 343, the meeting time for this class has been changed to MW 1:30-2:50.

**Location:** Ryerson
Physical Laboratory, Room 358.

**Times:** 1:30–2:50pm on Mon/Wed.

**Instructor:** Lek-Heng
Lim

Office: Eckhart 122

`lekheng(at)galton.uchicago.edu`

Tel: (773) 702-4263

**Office hours:** 2:00–4:00pm, Tue

**Course Assistant:** Somak
Dutta

Office: Eckhart 117

`sdutta(at)galton.uchicago.edu`

**Office hours:** 6:30–7:30pm, Mon

The last three topics we would only touch upon briefly (no discussion of actual algorithms); they would be treated in greater detail in a second course.

- Linear algebra over
**R**or**C**: How this course differs from your undergraduate linear algebra course.

- Three basic matrix decompositions: LU, QR, SVD.

- Gaussian elimination revisited: LU and LDU decompositions.

- Backward error analysis: Guaranteeing correctness in approximate computations.

- Gram-Schmidt orthogonalization revisited: QR and complete orthogonal decompositions.

- Solving system of linear equations in the exact and the approximate sense: Linear systems, least squares, data least squares, total least squares.

- Low rank matrix approximations and matrix completion.

- Iterative methods: Stationary methods and Krylov subspace methods.

- Eigenvalue and singular value problems.

- Sparse linear algebra: Sparse matrices and sparse solutions.

- Lecture 5:
- Notes (requires Chalk access)

- Lecture 4:
- Notes (requires Chalk access)
- All norms are equivalent on finite-dimensional spaces

- Lecture 3:
- Notes (requires Chalk access)

- Lecture 2:
- Notes (requires Chalk access)
- Top 10 Algorithms of the 20th Century

- Lecture 1:
- Notes (requires Chalk access)
- Top 500 Supercomputers
- Communication Avoiding Numerical Linear Algebra

**Bug report** on lecture notes:
`lekheng(at)galton.uchicago.edu`

Except for the take-home final, collaborations are permitted but you will need to write up your own solutions. The problem sets are designed to get progressively more difficult. You will get at least seven days for each problem set.

- Problem Set 1 (posted: Oct 03; due: Oct 12)

- Problem Set 2 (posted: Oct 20; due: Oct 28)

- Problem Set 3 (posted: Oct 29; due: Nov 07)

- Problem Set 4 (posted: Nov 14; due: Nov 21)

- Problem Set 5 (posted: Dec 04; due: Dec 13)

**Bug report** on the problem sets or the solutions:
`lekheng(at)galton.uchicago.edu`

- Course homepage from Fall 2009 (courtesy of Yali Amit) and Fall 2010. Related course homepages from Fall 2005 and Spring 2006.

**Grade composition:** 75% Problem Sets, 25% Final

Any one of the following books, listed in order of increasing sophistication, would be acceptable.

- L.N. Trefethen, D. Bau, Numerical Linear Algebra, SIAM, 1997.

- D. Watkins, Fundamentals of Matrix Computations, 3rd Ed., Wiley, 2010.

- G. Golub, C. Van Loan, Matrix Computations, 3rd Ed., John Hopkins, 1996.

- J. Demmel, Applied Numerical Linear Algebra, SIAM, 1997.

We will use the following references when discussing applications to statistics and economics:

- R.B. Bapat, Linear Algebra and Linear Models, 2nd Ed., Springer, 2000.

- H. Theil, "Linear algebra and matrix methods in econometrics,"
pp. 3–65, Z. Griliches and M.D. Intriligator (Eds.),
*Handbook of Econometrics*,**I**, North Holland, 1983.

- D.S. Bernstein, Matrix Mathematics, 2nd Ed., Princeton, 2009.

- G. Golub, G. Meurant, Matrices, Moments and Quadrature with Applications, Princeton, 2010.

- N.J. Higham, Accuracy and Stability of Numerical Algorithms, 2nd Ed., SIAM, 2002.

- M. Overton, Numerical Computing with IEEE Floating Point Arithmetic, SIAM, 2001.

- R. Thisted, Elements of Statistical Computing: Numerical Computation, CRC, 1988.