STAT 30900/CMSC 37810. Mathematical Computation I — Matrix Computation

Department of Statistics
University of Chicago
Fall 2014

This is an introductory course on numerical linear algebra. The course will present a global overview of a number of topics, from classical to modern to state-of-the-art. The fundamental principles and techniques will be covered in depth but towards the end of the course we will also discuss some exciting recent developments.

Numerical linear algebra is quite different from linear algebra. We will be much less interested in algebraic results that follow from the axiomatic definitions of fields and vector spaces but much more interested in analytic results that hold only over the real and complex fields. The main objects of interest are real- or complex-valued matrices, which may come from differential operators, integral transforms, bilinear and quadratic forms, boundary and coboundary maps, Markov chains, graphs, metrics, correlations, hyperlink structures, cell phone signals, DNA microarray measurements, movie ratings by viewers, friendship relations in social networks, etc. Numerical linear algebra provides the mathematical and algorithmic tools for matrix problems that arise in engineering, scientific, and statistical applications.

Announcements

11/23/14: Please turn in Homework 5 to Somak in Eckhart 131 during his office hours from 3–5pm on Thu, Dec 4.
11/23/14: Lecture notes 10 and Homework 5 posted.
11/14/14: Midterm exam 1:30–4:20pm in Eckhart 133.
11/09/14: Lecture notes 9 posted.
11/06/14: Remarks regarding flop counts.
11/05/14: Lecture notes 8 posted.
11/03/14: Third make-up lecture tomorrow (Nov 4), 6:00–8:00pm in Eckhart 133.
11/01/14: Lecture notes 7 and Homework 4 posted.
10/24/14: Lecture notes 6 and Homework 3 posted.
10/21/14: Lecture notes 5 posted.
10/20/14: Second make-up lecture tomorrow (Oct 21), 6:00–8:20pm in Eckhart 133.
10/18/14: Lecture notes 4 and Homework 2 posted.
10/10/14: Lecture notes 3 and Homework 1 posted.
10/08/14: Lecture notes 2 posted.
10/06/14: First make-up lecture tomorrow (Oct 7), 6:00–8:00pm in Eckhart 133.
10/05/14: Some of the things I mentioned in class — how numerical solutions of PDEs and integral equations, Fast Fourier Transforms, Fast Multipole Methods, etc, can be reduced to matrix computations — have been superbly described in this set of notes.
10/04/14: Lecture notes 1 and Homework 0 posted.
10/03/14: We will miss lectures on Nov 14 (midterm exam) and Nov 28 (Thanksgiving). Make-up lectures will be held on Tues Oct 7, Oct 21, Nov 4, 6–8pm in Eckhart 133.
10/03/14: Check back regularly for announcements.

Lectures

Location: Room 133, Eckhart Hall.

Times: 1:30–4:20pm on Fri.

Course staff

Instructor: Lek-Heng Lim
Office: Eckhart 122
lekheng(at)galton.uchicago.edu
Tel: (773) 702-4263
Office hours: 2:00–4:00pm on Oct 9 (Thu), Oct 23 (Thu), Nov 6 (Thu), Nov 25 (Tue)

Course Assistant: Somak Dutta
Office: Eckhart 8
somakd(at)uchicago.edu
Office hours: 3:00–5:00pm on Oct 16 (Thu), Oct 30 (Thu), Nov 13 (Thu), Dec 04 (Thu)

Syllabus

The last two topics we would only touch upon briefly (no discussion of actual algorithms); they would be treated in greater detail in a second course.

Linear algebra over R or C: How this course differs from your undergraduate linear algebra course.

Three basic matrix decompositions: LU, QR, SVD.

Gaussian elimination revisited: LU and LDU decompositions.

Backward error analysis: Guaranteeing correctness in approximate computations.

Gram–Schmidt orthogonalization revisited: QR and complete orthogonal decompositions.

Solving system of linear equations in the exact and the approximate sense: Linear systems, least squares, data least squares, total least squares.

Low rank matrix approximations and matrix completion.

Iterative methods: Stationary methods and Krylov subspace methods.

Eigenvalue and singular value problems.

Sparse linear algebra: Sparse matrices and sparse solutions.

Problem Sets

Collaborations are permitted but you will need to write up your own solutions and declare your collaborators. The problem sets are designed to get progressively more difficult. You will get at least six days for each problem set.

Problem Set 5 (posted: Nov 23, due: Dec 04)

Problem Set 4 (posted: Nov 01, due: Nov 14)

Problem Set 3 (posted: Oct 24, due: Oct 31)

Problem Set 2 (posted: Oct 18, due: Oct 24)

Problem Set 1 (posted: Oct 10, due: Oct 17)

Problem Set 0 (posted: Oct 04, due: Oct 10)

Bug report on the problem sets: lekheng(at)galton.uchicago.edu

Supplementary materials

Course homepage from Fall 2009 (courtesy of Yali Amit), Fall 2010, Fall 2011, Fall 2012, Fall 2013. Related course homepages from Fall 2005 and Spring 2006.

Grades

Grade composition: 50% Problem Sets (six altogether, lowest grade would be dropped), 50% Midterm Exam (Nov 14, 1:30–4:20pm, in-class, closed book)