Probability and Random Processes
Spring 2023
Thomas Courtade
Lecture: Tue & Thu 12:30 pm - 2:00 pm, Lewis 100
Office Hour: Mon 1:00 pm - 2:00 pm, Cory 265
Announcements
- Welcome to EECS 126! Please read the course info for logistics. We will be syncing Ed with the course roster periodically. If you are newly enrolled in the course but not added to Ed after a few days, please email eecs126-inst (at) berkeley.edu.
- Time conflicts are allowed only if you choose to attend EECS 126 lectures. The lectures are not recorded and midterms will be held during lecture times. There will not an alternate final exam time. Please make sure that your exam times don't conflict with other classes you're taking!
Lecture Schedule
Schedule is subject to some changes.
| Date | Topics | Readings |
|---|---|---|
| 1/17 | Introduction/Logistics, Probability Basics | B&T 1, W Appendix A |
| 1/19 | Bayes Rule, Independence, Discrete Random Variables | B&T 1,2, W Appendix A Random Variables |
| 1/24 | Expectation (Linearity, Tail Sum), Discrete Distributions | B&T 2, W Appendix B |
| 1/26 | Sum of Independent Binomials, Variance, Covariance, Correlation Coefficient, Conditional Expectation and Iterated Expectation, Entropy | B&T 2, W Appendix B |
| 1/31 | Entropy, Continuous Probability (Sample Space, Events, PDFs, CDFs), Continuous Distributions | B&T 3, W Appendix B |
| 2/2 | Gaussian Distribution, Derived Distributions, Continuous Bayes | B&T 3, 4.1-4.2, W Appendix B |
| 2/7 | Order Statistics, Convolution, Moment Generating Functions | B&T 4.3-4.6, W Sections 3-4 |
| 2/9 | MGFs, Bounds/Concentration Inequalities (Markov, Chebyshev, Chernoff) | B&T 5.1, W Sections 3-4 Concentration Inequalities |
| 2/14 | Convergence, Weak and Strong Law of Large Numbers, Central Limit Theorem | B&T 5.2-5.6, W Sections 3-4 Convergence |
| 2/16 | No Lecture (Midterm 1) | |
| 2/21 | Information Theory and Digital Communication, Capacity of the Binary Erasure Channel (BEC) | W Section 7 Information Theory |
| 2/23 | Achievability of BEC Capacity, Markov Chains Introduction | B&T 7.1-7.4l, W Section 7 Capacity of BEC |
| 2/28 | Discrete Time Markov Chains: Irreducibility, Aperiodicity, Invariant Distribution and Balance Equations | B&T 7.1-7.4, W Sections 1-2 Discrete Time Markov Chains |
| 3/2 | DTMCs: Hitting Time and First Step Equations (FSEs), Infinite State Space, Classification of States, Big Theorem | B&T 7.1-7.4, W Sections 1-2 |
| 3/7 | DTMCs: Classification, Reversibility, Poisson Processes: Construction | B&T 6.1-6.3, W Sections 1-2 Reversibility |
| 3/9 | Poisson Processes: Counting Process, Memorylessness, Merging, Splitting | B&T 7.5, W Section 5 Poisson Processes |
| 3/14 | Poisson Processes: Erlang Distribution, Random Incidence, Continuous Time Markov Chains Introduction, Rate Matrix | B&T 7.5, W Section 5 |
| 3/16 | CTMCs: Balance Equations, Big Theorem, FSEs | B&T 7.5, W Section 6 Continuous Time Markov Chains |
| 3/21 | CTMCs: Simulated DTMC, Erdos-Renyi Random Graphs | W Section 6 Random Graphs |
| 3/23 | No Lecture (Midterm 2) | |
| 3/28 | No Lecture (Spring Break) | |
| 3/30 | No Lecture (Spring Break) | |
| 4/4 | Maximum Likelihood Estimation, Maximum A Posteriori Estimation | B&T 8.1-8.2, 9.1 |
| 4/6 | MLE/MAP, Neyman Pearson Hypothesis Testing | B&T 8.1-8.2, 9.3-9.4, W Sections 7-8 Hypothesis Testing |
| 4/11 | Neyman Pearson Hypothesis Testing, Vector Space of Random Variables and Least Squares Estimation | B&T 9.3-9.4, 8.3-8.5, W Sections 8-9 Hilbert Space of RVs |
| 4/13 | Linear Least Squares Estimation, Minimum Mean Square Error (MMSE) Estimation | B&T 8.3-8.5, W Section 9 |
| 4/18 | MMSE, Gram Schmidt Process | W Sections 9-10 |
| 4/20 | Jointly Gaussian Random Variables, Kalman Filter | W Sections 8, 10 Jointly Gaussian RVs |
| 4/25 | Kalman Filter | W Section 10 Kalman Filter (1) Kalman Filter (2) |
| 4/27 | Hidden Markov Models | W Section 11 Hidden Markov Models |