# Probability and Random Processes

Spring 2020

Kannan Ramchandran

TuTh 12:30-2 PM, Lewis 100

OH: Tuesday 2-3 Cory 212

## Announcements

- Lab 6 is released. It is due by Friday night (4/10) at 11:59PM.
- Homework 9 solutions are released. Self grades are due by Wednesday night (4/8) at 11:59PM.
- Homework 10 is released. It is due by Wednesday night (4/8) at 11:59PM.
- Welcome to EECS 126! Please read the course info, join Piazza, and join Gradescope (code 9XJ64Z).

## Lecture Schedule

Lectures are not recorded. Subject to some changes.

Date | Topics | Reading |
---|---|---|

1/21 | Introduction, Probability Spaces, Conditional Probability, Law of Total Probability | B-T 1 |

1/23 | Independence, Bayes Rule, Discrete Random Variables | B-T 1, 2 |

1/28 | Expectation, Uniform, Geometric, Binomial and Poisson Distributions | B-T 2 |

1/30 | (Co)variance, Correlation, Conditional / Iterated Expectation, Law of Total Variance | B-T 2 |

2/4 | Continuous Probability, Uniform, Exponential Distributions | B-T 3 |

2/6 | Gaussian Distribution, Derived Distributions, Continuous Bayes | B-T 3, 4.1-4.2 |

2/11 | Order Statistics, Convolution, Moment Generating Functions | B-T 4.3-4.6 |

2/13 | MGFs, Bounds/Concentration Inequalities (Markov, Chebyshev, Chernoff) | B-T 5.1 |

2/18 | Convergence, Weak and Strong Law of Large Numbers, Central Limit Theorem | B-T 5.2-5.6, W 2.1-2.3 Convergence |

2/20 | No Lecture (Midterm 2/21) | |

2/25 | Information Theory | Information Theory |

2/27 | Binary Erasure Channel Capacity | W 1, 13.3, B-T 7.1-7.4 Capacity of BEC |

3/3 | Information Theory Wrapup | W 1, 2.4, 2.6, 13.3, B-T 7.1-7.4 Markov Chains |

3/5 | Discrete Time Markov Chains, Stationary Distribution, Hitting Time, First Step Equations | W 1, 2.4, 2.6, 13.3, B-T 7.1-7.4 |

3/10 | DTMCs: Reversibility, Infinite States, Classification, Big Theorem | Reversibility |

3/12 | Poisson Processes: Counting Process, Memorylessness, Merging, Splitting | B-T 6.1-6.3, W 13.4 |

3/17 | PP: Erlang Distribution, Random Incidence | B-T 6.1-6.3, W 13.4 |

3/19 | Continuous Time Markov Chains: Rate Matrix and Stationary Distribution | B-T 7.5, W 13.5 |

3/31 | CTMCs: Big Theorem, First Step Equations and Jump Chain | B-T 7.5, W 13.5 |

4/2 | No Lecture (Midterm 4/3) | |

4/7 | Erdos-Renyi Random Graphs | Random Graphs |

4/9 | Maximum Likelihood Estimation, Maximum a Posteriori Estimation | B-T 8.1-8.2, 9.1, W 5.1 |

4/14 | Statistical Hypothesis Testing, Neyman-Pearson Lemma | Hypothesis Testing |

4/16 | Minimum Mean Square Error Estimation, Vector Space of Random Variables | Hilbert Space |

4/21 | Linear Least Square Estimate | |

4/23 | Jointly Gaussian Random Variables | |

4/28 | Orthogonal Updates and Kalman Filter | Geometric Derivation of Scalar Kalman Filter |

4/30 | Fisher Information and Cramer Rao Bound |