Statistical Models and Computing Methods

Description and Objectives

This course develops computational methods for statistical estimation and inference: random-variable generation, Monte Carlo integration, importance sampling, likelihood optimization, EM, data augmentation, MCMC, Kalman and particle filtering, resampling, robust statistics and regression, causal graphical algorithms, and causal effect estimation.

Students will learn to derive and implement representative algorithms, state their assumptions, diagnose inappropriate applications, and interpret results. Causal computation covers both algorithms for a given DAG and the limitations of learning a graph from data.

Homework (60%)

Problem Set 1. Due on Oct. 12

Problem Set 2. Due on Oct. 12

Final Project (40%)

Suggested team size: 1–4 students.
Proposal (5%): one page stating the problem, methods and example; due November 16. 参考主题和要求
Presentations (5%): December 21 and 23. Proposed format: 15 minutes plus 5 minutes of questions, for up to eight teams; adjusted if enrollment requires it.
Report (30%): 6–8 pages of main text, with code and a contribution statement; final deadline TBD.

Schedule

Class	Week	Date	Topic	Milestone
1	1	Mon 09/07	Random-variable generation I
2	2	Mon 09/14	Random-variable generation II
3	2	Wed 09/16	Monte Carlo integration and methods
4	3	Mon 09/21	Importance sampling	PS1 out
5	4	Mon 09/28	Varaince reduction
6	4	Wed 09/30	No class
–	5	Mon 10/05	No class: National Day holiday
7	6	Mon 10/12	Bootstrap	PS1 due
8	6	Wed 10/14	Gibbs sampling and data augmentation
9	7	Mon 10/19	MCMC implementation
10	8	Mon 10/26	Codes illustration	Illustration codes for MCMC
11	8	Wed 10/28	Permutation test	PS2 out
12	9	Mon 11/02	Optimization, EM algorithm
13	10	Mon 11/09	Data Augmentation	PS2 due

October 5 is a holiday.

Textbook and References

GH — Primary text: Givens, G. H. and Hoeting, J. A. Computational Statistics, 2nd ed., Wiley. First published 2012; copyright 2013.
LD — Chinese companion: 李东风，《统计计算》，2022-08-29 讲义版。
RZ: Rizzo, M. L. (2008). Statistical Computing with R, 1st ed., Chapman & Hall/CRC.
TA: Tanner, M. A. (1993). Tools for Statistical Inference, 2nd ed., Springer.
HB: Gentle, J. E., Härdle, W. K. and Mori, Y., eds. (2012). Handbook of Computational Statistics: Concepts and Methods, 2nd revised and updated ed., Springer. The Davies–Gather chapter supports robust statistics.

Chapter and page references below use these editions. Kalman filtering and causal methods use the supplementary readings listed at the end. Readings are alternatives and selected passages, not a requirement to read all five books for each class.

Supplementary Readings

Särkkä, S. and Svensson, L. (2023). Bayesian Filtering and Smoothing, 2nd ed., Cambridge University Press. Ch. 6, pp.91–107; Ch. 11 for particle filtering (selections).
Kalman, R. E. (1960). A New Approach to Linear Filtering and Prediction Problems. Journal of Basic Engineering, 82(1), 35–45. Optional original paper.
Vehtari, A. et al. (2021). Rank-normalization, folding, and localization: An improved R-hat for assessing convergence of MCMC (with discussion). Bayesian Analysis, 16, 667–718.
Shachter, R. D. (1998). Bayes-Ball: The Rational Pastime. UAI, 480–487; §3. The linked mirror was uploaded in 2013.
Perković, E. et al. (2018). Complete Graphical Characterization and Construction of Adjustment Sets in Markov Equivalence Classes of Ancestral Graphs. JMLR, 18(220), 1–62; §§4.3, 5, DAG selections.
Hernán, M. A. and Robins, J. M. (2020). Causal Inference: What If. Chs. 12–13 and identification background.
Chernozhukov, V. et al. (2018). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21, C1–C68; §5.1.
Kalisch, M. et al. (2012). Causal Inference Using Graphical Models with the R Package pcalg. JSS, 47(11), 1–26; §§2.1, 3.2.
Colombo, D. and Maathuis, M. H. (2014). Order-Independent Constraint-Based Causal Structure Learning. JMLR, 15, 3921–3962; §§3, 4.1.