Probability and Random Processes


Spring 2022
Kannan Ramchandran
Lecture: TuTh 3:30-5 PM (v)
Office Hours: Tu 5-6 PM (v)

Announcements

Lecture Schedule

Schedule is subject to some changes.

Date Topics Readings
01/18 Introduction, Probability Spaces, Conditional Probability, Law of Total Probability B-T 1
01/20 Independence, Bayes Rule, Discrete Random Variables B-T 1, 2
01/25 Expectation, Uniform, Geometric, Binomial and Poisson Distributions B-T 2
01/27 (Co)variance, Correlation, Conditional / Iterated Expectation, Law of Total Variance B-T 2
02/01 Continuous Probability, Uniform, Exponential Distributions B-T 3
02/03 Gaussian Distribution, Derived Distributions, Continuous Bayes B-T 3, 4.1-4.2
02/08 Order Statistics, Convolution, Moment Generating Functions B-T 4.3-4.6
02/10 MGFs, Bounds/Concentration Inequalities (Markov, Chebyshev, Chernoff) B-T 5.1
02/15 Convergence, Weak and Strong Law of Large Numbers, Central Limit Theorem B-T 5.2-5.6, W 2.1-2.3
Convergence
02/17 Information Theory Information Theory
02/22 No Lecture (Midterm)  
02/24 Binary Erasure Channel Capacity W 1, 13.3, B-T 7.1-7.4
Capacity
03/01 Information Theory Wrapup  
03/03 Discrete Time Markov Chains, Stationary Distribution, Hitting Time, First Step Equations W 1, 2.4, 2.6, 13.3, B-T 7.1-7.4
Markov Chains
03/08 DTMCs: Reversibility, Infinite States, Classification, Big Theorem Reversibility
03/10 Poisson Processes: Counting Process, Memorylessness, Merging, Splitting B-T 6.1-6.3, W 13.4
03/15 PP: Erlang Distribution, Random Incidence B-T 6.1-6.3, W 13.4
03/17 Continuous Time Markov Chains: Rate Matrix and Stationary Distribution B-T 7.5, W 13.5
03/29 CTMCs: Big Theorem, First Step Equations and Jump Chain B-T 7.5, W 13.5
CTMCS
03/31 Erdos-Renyi Random Graphs Random Graphs
04/05 No Lecture (Midterm)  
04/07 Maximum Likelihood Estimation, Maximum a Posteriori Estimation B-T 8.1-8.2, 9.1, W 5.1
04/12 Statistical Hypothesis Testing, Neyman-Pearson Lemma Hypothesis Testing
B-T 9.3-9.4, W 5.5-5.6, 6.5
04/14 Minimum Mean Square Error Estimation, Vector Space of Random Variables Hilbert space of RVs
B-T 8.3-8.5, W 7.1-7.5
04/19 Linear Least Square Estimate W 7.1-7.5, W 8.1
04/21 Jointly Gaussian Random Variables W 6.3-6.4, 7.6, 8.1-8.3
04/26 Jointly Gaussian Random Variables Wrapup W 6.3-6.4, 7.6, 8.1-8.3
04/28 Orthogonal Updates and Kalman Filter Kalman Filter
W 7.6, 8.1-8.3