University of Chicago

Fall 2019

This is an introductory course on numerical linear algebra. The course will present a global overview of a number of topics, from classical to modern to state-of-the-art. The fundamental principles and techniques will be covered in depth but towards the end of the course we will also discuss some exciting recent developments.

Numerical linear algebra is quite different from linear algebra. We will be much less interested in algebraic results that follow from the axiomatic definitions of fields and vector spaces but much more interested in analytic results that hold only over the real and complex fields. The main objects of interest are real- or complex-valued matrices, which may come from differential operators, integral transforms, bilinear and quadratic forms, boundary and coboundary maps, Markov chains, graphs, metrics, correlations, hyperlink structures, cell phone signals, DNA microarray measurements, movie ratings by viewers, friendship relations in social networks, etc. Numerical linear algebra provides the mathematical and algorithmic tools for matrix problems that arise in engineering, scientific, and statistical applications.

- 11/20/19: Homework 5 and Lecture Notes 16 posted.

- 11/20/19: Class will not meet during Thanksgiving week. Zehua will hold office hours on Wed, Nov 27, 10:00am–12:00n in Jones 304.

- 11/18/19: Lecture Notes 15 posted.

- 11/17/19: Quiz II will be held Wed, Dec 4, 3:00–4:20pm in Eckhart 133.

- 11/13/19: Homework 4 and Lecture Notes 14 posted.

- 11/11/19: Lecture Notes 13 posted.

- 11/06/19: Lecture Notes 12 posted.

- 11/04/19: Homework 3 and Lecture Notes 11 posted.

- 10/28/19: Lecture Notes 10 posted.

- 10/23/19: Lecture Notes 9 posted.

- 10/23/19: Matlab tutorial 10am–12pm, R tutorial 1–3pm in Room 112 Math-Stat Bldg (Stevanovich Center) on Fri, Nov 1.

- 10/21/19: Homework 2 and Lecture Notes 8 posted.

- 10/17/19: Lecture Notes 7 posted. Dates for Quiz I and Quiz II fixed (see below).

- 10/14/19: Lecture Notes 6 posted.

- 10/12/19: Homework 1 and Lecture Notes 5 posted.

- 10/09/19: Lecture Notes 4 posted.

- 10/08/19: Lecture Notes 3 posted.

- 10/07/19: Zhen's office hours on Tue, Oct 8, will be from 10:30–12:00 in Jones 304.

- 10/05/19: Lecture Notes 2 posted.

- 10/02/19: Homework 0 posted. Lecture Notes 0 & 1 posted.

- 10/02/19: Make-up lectures 3:00–4:20pm on Fridays, Oct 4 and Oct 11 in Eckhart 133.

- 09/29/19: Check back regularly for announcements.

**Location:** Room 133, Eckhart
Hall.

**Times:** 3:00–4:20pm on Mon and Wed.

**Instructor:** Lek-Heng
Lim

Office: Jones 122C

`lekheng(at)galton.uchicago.edu`

Tel: (773) 702-4263

**Office hours:** Wed, 1:00–2:30pm, Jones 122C.

**Course Assistant I:** Zhen Dai

Office: Jones 307

`zhen9(at)uchicago.edu`

**Office hours:** Tue, 10:30am–12:00pm, Jones 304.

**Course Assistant II:** Zehua Lai

Office: Jones 307

`laizehua(at)uchicago.edu`

**Office hours:** Thu, 2:00–3:30pm, Jones 226.

The last two topics we would only touch upon briefly (no discussion of actual algorithms); they would be treated in greater detail in a second course.

- Linear algebra over
**R**or**C**: How this course differs from your undergraduate linear algebra course.

- Three basic matrix decompositions: LU, QR, SVD.

- Gaussian elimination revisited: LU and LDU decompositions.

- Backward error analysis: Guaranteeing correctness in approximate computations.

- Gram–Schmidt orthogonalization revisited: QR and complete orthogonal decompositions.

- Solving system of linear equations in the exact and the approximate sense: Linear systems, least squares, data least squares, total least squares.

- Low rank matrix approximations and matrix completion.

- Iterative methods: Stationary methods and Krylov subspace methods.

- Eigenvalue and singular value problems.

- Sparse linear algebra: Sparse matrices and sparse solutions.

Collaborations are permitted but you will need to write up your own solutions and declare your collaborators. The problem sets are designed to get progressively more difficult. You will get about 10 days for each problem set.

- Problem Set 5 (posted: Nov 20, due: Dec 6)

- Problem Set 4 (posted: Nov 13, due: Nov 25)

- Problem Set 3 (posted: Nov 4, due: Nov 13)

- Problem Set 2 (posted: Oct 21, due: Nov 4)

- Problem Set 1 (posted: Oct 11, due: Oct 21)

- Problem Set 0 (posted: Oct 2, due: Oct 11)

**Bug report** on the problem sets:
`lekheng(at)galton.uchicago.edu`

- Course homepage from Fall 2009 (courtesy of Yali Amit), Fall 2010, Fall 2011, Fall 2012, Fall 2013, Fall 2014, Fall 2015, Fall 2016, Fall 2017, Fall 2018. Related course homepages from Fall 2005 and Spring 2006.

**Grade composition:** 50% Problem Sets (six altogether, lowest
grade would be dropped), 50% Quizzes (two altogether, in-class, closed
book)

**Exam dates:** Quiz I on Wed, Oct 30, 3:00–4:20pm. Quiz II on
Wed, Dec 4, 3:00–4:20pm.

We will use the 4th edition of Golub–Van Loan.

- D.S. Bernstein, Matrix Mathematics, 2nd Ed., Princeton, 2009.

- J. Demmel, Applied Numerical Linear Algebra, SIAM, 1997.

- G. Golub, G. Meurant, Matrices, Moments and Quadrature with Applications, Princeton, 2010.

- G. Golub, C. Van Loan, Matrix Computations, 4th Ed., John Hopkins, 2013.

- N.J. Higham, Accuracy and Stability of Numerical Algorithms, 2nd Ed., SIAM, 2002.

- M. Overton, Numerical Computing with IEEE Floating Point Arithmetic, SIAM, 2001.

- R. Thisted, Elements of Statistical Computing: Numerical Computation, CRC, 1988.

- L.N. Trefethen, D. Bau, Numerical Linear Algebra, SIAM, 1997.