Basic pattern recognition. The Bayes classifier. Generative vs.
Discriminative approaches. Nearest Neighbors. Linear discriminant
analysis vs. logistic regression. Examples of pattern recognition problems.
Reading: Chapter 1.5.1-1.5.4, 4.2, 4.3
Basic multivariate distributions. Multivariate Normal model,
conditional distributions,discrete
distributions: multinomial, product distributions. The exponential family.
Maximum likelihood and Bayesian estimation.
Reading: Section 2.2, 2.3.1-6, 2.4.1-2
A
summary of properties of the multivariate normal
2
Derivations of MLE for covariance matrix of the multivariate normal
Mixture models and the EM algorithm. Mixture models. Estimation with
missing observations. Some elementary information theory. The EM
algorithm and its variants.
Reading: 9.1, 9.2, 9.3.1-3
Some notes on
the EM algorithm
Graphical models. Directed graphs, conditional independence,
efficient computation (peeling). Undirected graphs, Cliques, Markov
Random Fields, decomposable graphs.
Reading: Chapters 8.1, 8.2, 8.3.1-2, 8.4.1-2.
Sequential Data. Hidden Markov models: estimation and
computation.
Reading: 13.1-2
A review paper on HMM's and their use in speech recognition.
Discriminative classification methods. Decision trees. Cross
validation. Boosting. Example: Handwritten
digit recognition.
Reading: Chapter 14.3,4.
Generative classification methods. Hierarchical models. Mixture models. Example: Handwritten digit recognition.
Markov Chain Monte-Carlo. Motivation: hierarchical models. Basic Markov chain theory. Irreducibility, ergodicity, stationary distributions.
MCMC continued. Importance sampling, Metropolis Hastings. The
Multivariate Normal case.
Reading: Chapter
11.1-3