Probability and Random Processes
Spring 2019
Kannan Ramchandran
Lectures: 12.30-2pm, 105 Stanley
Announcements
Lecture Schedule
(*Lectures are not recorded)Date | Topics | Reading | Assignments |
---|---|---|---|
01/22 | Introduction/Logistics, Probability Spaces/Axioms, Conditional Probability, Multiplication Rule, Law of Total Probability, Bayes Rule | B-T 1 | Discussion 1 (Solutions) |
1/24 | Bayes Rule, Independence, Discrete Random Variables | B-T 1,2 | Homework 1 (Solutions) Lab 1 (Solutions) |
1/29 | Expectation (Linearity, Tail Sum), Uniform, Geometric, Binomial and Poisson Distributions | B-T 2 | Discussion 2 (Solutions) |
1/31 | Sum of Independent Binomials, Variance, Covariance, Correlation Coefficient, Conditional Expectation and Iterated Expectation, Entropy | B-T 2 | Homework 2 (Solutions) Lab 2 (Solutions) |
2/5 | Entropy, Continuous Probability (Sample Space, Events, PDFs, CDFs), Uniform, Exponential Distributions | B-T 3 | Discussion 3 (Solutions) |
2/7 | Gaussian Distribution, Derived Distributions, Continuous Bayes | B-T 3, B-T 4.1-4.2 | Homework 3 (Solutions) Lab 3 (Solutions) |
2/12 | Order Statistics, Convolution, Moment Generating Functions | B-T 4.3-4.6 | Discussion 4 (Solutions) |
2/14 | MGFs, Bounds/Concentration Inequalities (Markov, Chebyshev, Chernoff) | B-T 5.1, W 13.7 | Homework 4 (Solutions) |
2/19 | No Lecture (Midterm) | Discussion 5 (Solutions) | |
2/21 | Convergence, Weak and Strong Law of Large Numbers, Central Limit Theorem | B-T 5.2-5.6, W 2.1-2.3 | Homework 5 (Solutions) |
2/26 | Central Limit Theorem Proof Sketch, Information Theory and Digital Communication, Capacity of the Binary Erasure Channel (BEC) | Capacity of BEC | Discussion 6 (Solutions) |
2/28 | Achievability of BEC Capacity, Markov Chains Introduction | W 1, 13.3, B-T 7.1-7.4 | Homework 6 (Solutions) Lab 4 (Solutions) |
3/5 | Discrete Time Markov Chains: Irreducibility, Aperiodicity, Invariant Distribution and Balance Equations | W 1, 2.4, 2.6, 13.3, B-T 7.1-7.4 | Discussion 7 (Solutions) |
3/7 | DTMCs: Hitting Time and First Step Equations (FSEs), Infinite State Space, Classification of States, Big Theorem | W 1, 2.4, 2.6, 13.3, B-T 7.1-7.4, Markov Chains Note | Homework 7 (Solutions) Lab 5 (Solutions) |
3/12 | DTMCs: Classification, Reversibility, Poisson Processes: Construction | B-T 6.1-6.3, W 13.4, Reversibility Note | Discussion 8 (Solutions) |
3/14 | Poisson Processes: Counting Process, Memorylessness, Merging, Splitting | B-T 6.1-6.3, W 13.4 | Homework 8 (Solutions) |
3/19 | Poisson Processes: Erlang Distribution, Random Incidence, Continuous Time Markov Chains Introduction, Rate Matrix | B-T 7.5, W 13.5 | Discussion 9 (Solutions) |
3/21 | CTMCs: Balance Equations, Big Theorem, FSEs | B-T 7.5, W 13.5 | Homework 9 (Solutions) |
3/26 | Spring Break | ||
3/28 | Spring Break | ||
4/2 | CTMCs: Simulated DTMC, Erdos-Renyi Random Graphs | Random Graphs | Discussion 10 (Solutions) |
4/4 | Maximum Likelihood Estimation, Maximum A Posteriori Estimation | W 5.1, B-T 8.1-8.2, 9.1 | Lab 6 (Solutions) |
4/9 | No Lecture (Midterm) | Discussion 11 (Solutions) | |
4/11 | MLE/MAP, Neyman Pearson Hypothesis Testing | W 5.1, B-T 8.1-8.2, 9.1/ W 5.5-5.6, 6.5, B-T 9.3-9.4, Hypothesis Testing | Homework 10 (Solutions) |
4/16 | Neyman Pearson Hypothesis Testing, Vector Space of Random Variables and Least Squares Estimation | W 5.5-5.6, 6.5, B-T 9.3-9.4/ W 7.1-7.5, B-T 8.3-8.5 | Discussion 12 (Solutions) |
4/18 | Linear Least Squares Estimation, Minimum Mean Square Error (MMSE) Estimation | W 7.1-7.5, B-T 8.3-8.5, Hilbert Space of Random Variables | Homework 11 (Solutions) |
4/23 | MMSE, Gram Schmidt Process | W 7.1-7.5, W 8.1 | Discussion 13(Solutions) |
4/25 | Jointly Gaussian Random Variables, Kalman Filter | W 6.3-6.4, 7.6, 8.1-8.3 Geometric Derivation of Scalar Kalman Filter | Homework 12(Solutions) Lab 7(Solutions) |
4/30 | Kalman Filter | W 7.6, 8.1-8.3 | Discussion 14 (Solutions) |
5/2 | Hidden Markov Models | W 9.2,9.4, HMMs and the Viterbi Algorithm | Homework 13(Solutions ) |
Discussions
Discussion worksheets will be posted here.
- Discussion 1 (Solutions)
- Discussion 2 (Solutions)
- Discussion 3 (Solutions)
- Discussion 4 (Solutions)
- Discussion 5 (Solutions)
- Discussion 6 (Solutions)
- Discussion 7 (Solutions)
- Discussion 8 (Solutions)
- Discussion 9 (Solutions)
- Discussion 10 (Solutions)
- Discussion 11 (Solutions)
- Discussion 12 (Solutions)
- Discussion 13 (Solutions)
- Discussion 14 (Solutions)
Homework
Homework will be posted here.
- Homework 1 (Solutions)
- Homework 2 (Solutions)
- Homework 3 (Solutions)
- Homework 4 (Solutions)
- Homework 5 (Solutions)
- Homework 6 (Solutions)
- Homework 7 (Solutions)
- Homework 8 (Solutions)
- Homework 9 (Solutions)
- Homework 10 (Solutions)
- Homework 11 (Solutions)
- Homework 12 (Solutions)
Labs
Labs will be posted here.