Probability and Random Processes


Fall 2022
Kannan Ramchandran
Lecture: TuTh 12:30-2 PM (Birge 50)
Office Hours: Tu 2-3 PM (Cory 212)

Announcements

Lecture Schedule

Schedule is subject to some changes.

Date Topics Readings
08/25 Introduction, Probability Spaces, Conditional Probability, Law of Total Probability B-T 1
08/30 Independence, Bayes Rule, Discrete Random Variables B-T 1, 2
Random Variables
09/01 Expectation, Uniform, Geometric, Binomial and Poisson Distributions B-T 2
09/06 (Co)variance, Correlation, Conditional / Iterated Expectation, Law of Total Variance B-T 2
09/08 Continuous Probability, Uniform, Exponential Distributions B-T 3
09/13 Gaussian Distribution, Derived Distributions, Continuous Bayes B-T 3, 4.1-4.2
09/15 Order Statistics, Convolution, Moment Generating Functions B-T 4.3-4.6
09/20 MGFs, Bounds/Concentration Inequalities (Markov, Chebyshev, Chernoff) B-T 5.1
09/22 Convergence, Weak and Strong Law of Large Numbers, Central Limit Theorem B-T 5.2-5.6, W 2.1-2.3
Convergence
09/27 Information Theory Information Theory
09/29 No Lecture (Midterm)  
10/04 Binary Erasure Channel Capacity W 1, 13.3, B-T 7.1-7.4
Capacity
10/06 Information Theory Wrapup  
10/11 Discrete Time Markov Chains, Stationary Distribution, Hitting Time, First Step Equations W 1, 2.4, 2.6, 13.3, B-T 7.1-7.4
Markov Chains
10/13 DTMCs: Reversibility, Infinite States, Classification, Big Theorem Reversibility
10/18 DTMC Wrapup B-T 6.1-6.3, W 13.4
10/20 Poisson Processes: Counting Process, Memorylessness, Merging, Splitting B-T 6.1-6.3, W 13.4
Poisson Process
10/25 PP: Erlang Distribution, Random Incidence B-T 6.1-6.3, W 13.4
10/27 Continuous Time Markov Chains: Rate Matrix and Stationary Distribution B-T 7.5, W 13.5
11/01 CTMCs: Big Theorem, First Step Equations and Jump Chain B-T 7.5, W 13.5
CTMCS
11/03 No Lecture (Midterm)  
11/08 Erdos-Renyi Random Graphs Random Graphs
11/10 Maximum Likelihood Estimation, Maximum a Posteriori Estimation B-T 8.1-8.2, 9.1, W 5.1
11/15 Statistical Hypothesis Testing, Neyman-Pearson Lemma Hypothesis Testing
B-T 9.3-9.4, W 5.5-5.6, 6.5
11/17 Minimum Mean Square Error Estimation, Vector Space of Random Variables Hilbert space of RVs
B-T 8.3-8.5, W 7.1-7.5
11/20 Linear Least Square Estimate W 7.1-7.5, W 8.1
11/29 Jointly Gaussian Random Variables W 6.3-6.4, 7.6, 8.1-8.3
Jointly Gaussian RVs
12/1 Orthogonal Updates and Kalman Filter W 7.6, 8.1-8.3
Kalman Filter (1)
Kalman Filter (2)