This is the course webpage for MTH 511: Statistical Simulation and Data Analysis from 2020 Semester-I.

The course will be run completely online through mooKIT and live tutorial sessions. Due to bandwidth issues, the videos will also be on Youtube.

Information:

Course Outline

Wk Lectures Topics Resources
1 Lec 1 Introduction to Monte Carlo Notes, R code, Video
  Lec 2 Pseudo-random number generator Notes, R code, Video
2 Lec 3 Inverse transform method (discrete) Notes, Video 1, Video 2
  Lec 4 Accept-rejection method (discrete) Notes, Code, Video
  Lec 5 Composition methods and Bernoulli factories Notes, Video
3 Lec 6 Inverse transform method (continuous) Notes, Video
  Lec 7 Accept-reject method (continuous) Notes, Code, Video
  Lec 8 Box-Muller and Ratio-of-Uniforms method Notes 1, Notes 2, Code 1, Code 2, Video
4 Lec 9 Accept-Reject: more examples Notes, Video
  Lec 10 Miscellaneous topics in sampling Notes, Code, Video
  Lec 11 Simple Importance Sampling Notes, Code, Video
5 Lec 12 Simple importance sampling Notes, Code, Video
  Lec 13 Weighted importance sampling Notes, Video
6 Lec 14 MLE and (ridge) regression Notes, Video
  Lec 15 Newton Raphson’s method Notes, Code, Video
  Lec 16 Gradient Ascent Notes, Code, Video
7 – Midsem Exams  
8 Lec 17 MM algorithm Notes, Code, Video
  Lec 18 EM algorithm and mixture of Gaussians Notes, Code, Video
  Lec 19 EM and Monte Carlo EM Notes, Code, Video
9 Lec 20 Loss functions Notes, Code, Video
  Lec 21 Cross-validation Notes, Code, Video
  Lec 22 Bootstrapping Notes, Code, Video
10 Lec 23 Stochastic Gradient Descent Notes, Code, Video
  Lec 24 Simulated Annealing Notes, Code, Video
  Lec 25 A Recap of Data Analysis Methods Notes, Video
11 – Mini-Project Week  
12 Lec 26 Introduction to Bayesian Models Notes, Code, Video
  Lec 27 Accept-Reject for Bayes Notes, Code, Video
  Lec 28 Bayesian Linear Regression Notes, Code, Video
13 Lec 29 Introduction to MCMC Notes, Code, Video
  Lec 30 Metropolis-Hastings Examples Notes, Code, Video
  Lec 31 Bayesian Logistic Regression Notes, Code, Video

References:

  • Sampling from Distributions:
    • “Simulation” by Sheldon M. Ross (Academic Press, Fourth Edition), 2006, Chaps. 1-5.
    • “Non-Uniform Random Variable Generation” by Luc Devroye. Online book
    • Bernoulli Factories: “Designing perfect simulation algorithms using local correctness” arxiv
    • Ratio-of-Uniforms method Paper, Resource 1, Resource 2
  • Maximum Likelihood Estimation
    • “Statistical Inference” by Casella and Berger.
    • MLE estimation notes at UChicago link
  • Regression
    • “Applied Linear Regression” by Sanford Weisberg
    • Elements of Statistical Learning by Hastie, Tibshirani, and Friedman Link
  • Optimization
    • “Convex Optimization” by Boyd and Vandenberghe Link
    • MM Algorithm notes by Kenneth Lange Link
    • MM Algorithm notes Link
  • EM Algorithm
  • Cross-validation
    • Slides Link
    • Note Link
    • Elements of Statistical Learning by Hastie, Tibshirani, and Friedman - Chapter 7 Link
  • Bootstrap
  • MCMC
  • Linchpin Accept-Reject
  • Importance Sampling