University of California at Berkeley
Dept of Electrical Engineering & Computer Sciences

CS 294-40
Learning for robotics and control

(3 units)


Problem sets


Tuesdays and Thursdays 11:00-12:30, 405 Soda


Pieter Abbeel
Office: Soda 721
Office hours: Tuesdays 12:30 - 2:30, and by appointment

Course description

This is an advanced course in learning for robotics and control. The goal of this course is to help the audience with their research in learning for robotics and control or related topics. A tentative list of topics includes:


Familiarity with mathematical proofs, machine learning, artificial intelligence, optimization, probability, algorithms, linear algebra; ability to implement algorithmic ideas in code (C/C++ and matlab).

Graduate students only (consent of instructor required for undergraduate students, please talk to me after first lecture and hand me summary of relevant classes/experience so I can decide whether to make an exception).


Final project

The final projects are open-ended projects. Students are encouraged to make appointments to discuss possible projects and to start this discussion right away. Students are also encouraged to closely interact with us on their projects throughout the semester.

Homework policy

Syllabus (subject to substantial change!) and scribed notes

Latex style file, latex macros file.
Lecture Topic Notes Scriber
Th Aug 28 Class outline. MDPs. pdf   tex Pieter Abbeel
Tu Sep 2 Dynamic programming, value iteration, contractions. pdf   tex   png Anand Kulkarni
Th Sep 4 Contractions, asynchronous value iteration pdf   tex   Yan Zhang
Tu Sep 9 Policy iteration, function approximation pdf   tex   figures Fernando Garcia Bermudez
Th Sep 11 Function approximation pdf   tex   Nimbus Goehausen
Tu Sep 16 LQR pdf   tex   Ankur Mehta
Th Sep 18 DDP pdf   tex   Brandon Basso
Tu Sep 23 Quadruped locomotion zip J. Zico Kolter
Th Sep 25 POMDP pdf   tex   figures Martin Moler Sorensen
Tu Sep 30 POMDP pdf   tex   png David Nachum
Th Oct 2 Bandits pdf   tex   png David Nachum
Tu Oct 7 Separation Principle, Dynamics Modeling pdf   tex   Pål From
Th Oct 9 Dynamics Modeling, Kalman Filtering pdf   tex   Andrew Wan
Tu Oct 14 Kalman Filtering pdf   tex   Jared Wood
Th Oct 16 Policy Gradient pdf   tex   zip Fernando Garcia Bermudez
Tu Oct 21 Policy Gradient pdf   tex   zip Jan Biermayer
Th Oct 23 TD, Sarsa, Q-learning pdf   tex   Yan Zhang
Tu Oct 28 TD, Sarsa, Q-learning, TD-Gammon pdf   tex   figure Anand Kulkarni
Th Oct 30 Reward Shaping pdf   tex   Pål From
Tu Nov 4 Exploration/Exploitation pdf   tex   Brandon Basso
Th Nov 6 No lecture
Tu Nov 11 Academic and Administrative Holiday
Th Nov 13 LP approach pdf   tex   Nimbus Goehausen
Tu Nov 18 Inverse reinforcement learning pdf   tex   Ankur Mehta
Th Nov 20 MPC, SLAM, Linearly solvable MDP's pdf   tex   Andrew Wan
Tu Nov 25 Learning to walk pdf   tex   figures Jared Wood
Th Nov 27 Happy Thanksgiving!
Tu Dec 2 Project presentations Martin Moler Sorensen: logistics czar
Th Dec 4 Project presentations Jan, Jared, David, Brandon Jan Biermayer: logistics czar

Further topics to be covered: POMDPs, certainty equivalence, Kalman filters, SLAM, linear systems, stability, Lyapunov theory, exploration, bandits, case studies of robotic systems, apprenticeship learning, linear programming approach, ...

Related materials