Probability and Random Processes
Spring 2021
Thomas Courtade
TuTh 2-3:30 PM, Internet/Online
Office Hours: Wednesday 12-1 PM
Announcements
- Welcome to EECS 126! Please read the course info, join Piazza, and join Gradescope (code RWKVJ7).
Lecture Schedule
Readings refer to Walrand’s “Probability in Electrical Engineering and Computer Science”. Online notes only serve as optional supplemental readings, and will not directly correspond to the lectures or textbook (see content). The B&T textbook may also be useful but is not the primary textbook.
Schedule is subject to some changes.
| Date | Topics | Readings |
|---|---|---|
| 1/19 | Introduction/Logistics, Probability Basics | Appendix A, B&T 1 |
| 1/21 | Bayes Rule, Independence, Discrete Random Variables | Appendix A, B&T 1,2 |
| 1/26 | Expectation (Linearity, Tail Sum), Discrete Distributions | Appendix B, B&T 2 |
| 1/28 | Sum of Independent Binomials, Variance, Covariance, Correlation Coefficient, Conditional Expectation and Iterated Expectation, Entropy | Appendix B, B&T 2 |
| 2/2 | Entropy, Continuous Probability (Sample Space, Events, PDFs, CDFs), Continuous Distributions | Appendix B, B&T 3 |
| 2/4 | Gaussian Distribution, Derived Distributions, Continuous Bayes | Appendix B, B&T 3, 4.1-4.2 |
| 2/9 | Order Statistics, Convolution, Moment Generating Functions | Sections 3-4, B&T 4.3-4.6 |
| 2/11 | MGFs, Bounds/Concentration Inequalities (Markov, Chebyshev, Chernoff) | Sections 3-4, B&T 5.1 |
| 2/16 | Convergence, Weak and Strong Law of Large Numbers, Central Limit Theorem | Sections 3-4, B&T 5.2-5.6 Convergence |
| 2/18 | No Lecture (Midterm 1) | |
| 2/23 | Information Theory and Digital Communication, Capacity of the Binary Erasure Channel (BEC) | Section 7 Capacity of a BEC |
| 2/25 | Achievability of BEC Capacity, Markov Chains Introduction | Section 7, B&T 7.1-7.4 Information Theory |
| 3/2 | Discrete Time Markov Chains: Irreducibility, Aperiodicity, Invariant Distribution and Balance Equations | Sections 1-2, B&T 7.1-7.4 Markov Chains |
| 3/4 | DTMCs: Hitting Time and First Step Equations (FSEs), Infinite State Space, Classification of States, Big Theorem | Sections 1-2, B&T 7.1-7.4 |
| 3/9 | DTMCs: Classification, Reversibility, Poisson Processes: Construction | Sections 1-2, B&T 6.1-6.3 Reversibility |
| 3/11 | Poisson Processes: Counting Process, Memorylessness, Merging, Splitting | Section 5, B&T 7.5 |
| 3/16 | Poisson Processes: Erlang Distribution, Random Incidence, Continuous Time Markov Chains Introduction, Rate Matrix | Section 5, B&T 7.5 |
| 3/18 | CTMCs: Balance Equations, Big Theorem, FSEs | Section 6, B&T 7.5 CTMCS |
| 3/23 | No Lecture (Spring Break) | |
| 3/25 | No Lecture (Spring Break) | |
| 3/30 | CTMCs: Simulated DTMC, Erdos-Renyi Random Graphs | Section 6 Random Graphs |
| 4/1 | Maximum Likelihood Estimation, Maximum A Posteriori Estimation | B&T 8.1-8.2, 9.1 |
| 4/6 | No Lecture (Midterm 2) | |
| 4/8 | MLE/MAP, Neyman Pearson Hypothesis Testing | Sections 7-8, B&T 8.1-8.2, 9.3-9.4 Hypothesis Testing |
| 4/13 | Neyman Pearson Hypothesis Testing, Vector Space of Random Variables and Least Squares Estimation | Sections 8-9, B&T 9.3-9.4, 8.3-8.5 Hilbert Space of RVs |
| 4/15 | Linear Least Squares Estimation, Minimum Mean Square Error (MMSE) Estimation | Section 9, B&T 8.3-8.5 |
| 4/20 | MMSE, Gram Schmidt Process | Sections 9-10 |
| 4/22 | Jointly Gaussian Random Variables, Kalman Filter | Sections 8, 10 Kalman Filter |
| 4/27 | Kalman Filter | Section 10 |
| 4/29 | Hidden Markov Models | Section 11 Hidden Markov Models |