Probability and Random Processes


Spring 2020
Kannan Ramchandran
TuTh 12:30-2 PM, Lewis 100

OH: Tuesday 2-3 Cory 212

Announcements

Lecture Schedule

Lectures are not recorded. Subject to some changes.

Date Topics Reading
1/21 Introduction, Probability Spaces, Conditional Probability, Law of Total Probability B-T 1
1/23 Independence, Bayes Rule, Discrete Random Variables B-T 1, 2
1/28 Expectation, Uniform, Geometric, Binomial and Poisson Distributions B-T 2
1/30 (Co)variance, Correlation, Conditional / Iterated Expectation, Law of Total Variance B-T 2
2/4 Continuous Probability, Uniform, Exponential Distributions B-T 3
2/6 Gaussian Distribution, Derived Distributions, Continuous Bayes B-T 3, 4.1-4.2
2/11 Order Statistics, Convolution, Moment Generating Functions B-T 4.3-4.6
2/13 MGFs, Bounds/Concentration Inequalities (Markov, Chebyshev, Chernoff) B-T 5.1
2/18 Convergence, Weak and Strong Law of Large Numbers, Central Limit Theorem B-T 5.2-5.6, W 2.1-2.3
Convergence
2/20 No Lecture (Midterm 2/21)  
2/25 Information Theory Information Theory
2/27 Binary Erasure Channel Capacity W 1, 13.3, B-T 7.1-7.4
Capacity of BEC
3/3 Information Theory Wrapup W 1, 2.4, 2.6, 13.3, B-T 7.1-7.4
Markov Chains
3/5 Discrete Time Markov Chains, Stationary Distribution, Hitting Time, First Step Equations W 1, 2.4, 2.6, 13.3, B-T 7.1-7.4
3/10 DTMCs: Reversibility, Infinite States, Classification, Big Theorem Reversibility
3/12 Poisson Processes: Counting Process, Memorylessness, Merging, Splitting B-T 6.1-6.3, W 13.4
3/17 PP: Erlang Distribution, Random Incidence B-T 6.1-6.3, W 13.4
3/19 Continuous Time Markov Chains: Rate Matrix and Stationary Distribution B-T 7.5, W 13.5
3/31 CTMCs: Big Theorem, First Step Equations and Jump Chain B-T 7.5, W 13.5
4/2 No Lecture (Midterm 4/3)  
4/7 Erdos-Renyi Random Graphs Random Graphs
4/9 Maximum Likelihood Estimation, Maximum a Posteriori Estimation B-T 8.1-8.2, 9.1, W 5.1
4/14 Statistical Hypothesis Testing, Neyman-Pearson Lemma Hypothesis Testing
4/16 Minimum Mean Square Error Estimation, Vector Space of Random Variables Hilbert Space
4/21 Linear Least Square Estimate  
4/23 Jointly Gaussian Random Variables  
4/28 Orthogonal Updates and Kalman Filter Geometric Derivation
of Scalar Kalman Filter
4/30 Fisher Information and Cramer Rao Bound