CS 182/282A | Deep Neural Networks

Fall 2022

Lectures: Tu/Th 9:30–11:00 am, Soda 306

Neural Networks

Description

Deep Networks have revolutionized computer vision, language technology, robotics and control. They have a growing impact in many other areas of science and engineering, and increasingly, on commerce and society. They do not however, follow any currently known compact set of theoretical principles. In Yann Lecun's words they require "an interplay between intuitive insights, theoretical modeling, practical implementations, empirical studies, and scientific analyses." This is a fancy way of saying “we don’t understand this stuff nearly well enough, but we have no choice but to muddle through anyway.” This course attempts to cover that ground and show you how to muddle through even as we aspire to do more.

Lecture reference materials

Lectures are webcast by the department and recordings will be posted to this youtube playlist. You must be logged into your @berkeley.edu account to access the videos. Lectures will have a substantial amount (if not all) of the content covered on the whiteboard (not on presentation slides)

Because Deep Learning is rapidly evolving field, the material covered in this course can change substantially from semester to semester. If you are interested in materials from previous iterations of this course, please see here: [Sp21] [Sp22]

Syllabus

W Date Lecture Topic Resources Section Homework
0 Aug 25 Introduction Recording
1 Aug 30 Basic Principles Recording Scribe Notes Worksheet Code Solutions HW0 - Basics
Written Coding Solutions Code Sol Self-Grades
Sep 1 Basic Principles Recording Scribe Notes
2 Sep 6 Basic Principles Recording Scribe Notes Worksheet Solutions HW1 - Math Review Written Coding Solutions Code Sol Self-Grades
Sep 8 Survey of Problems and Architectures Recording Scribe Notes
3 Sep 13 Survey of Problems and Architectures Recording Scribe Notes Worksheet Code Solutions HW2 - Optimizers and Initalization
Written Coding Solutions Code Sol Self-Grades
Sep 15 ConvNets and Computer Vision Recording Scribe Notes
4 Sep 20 ConvNets and Computer Vision Recording Scribe Notes Worksheet Code Solutions HW3 - GD with Momentum, BN and CNN
Written Coding Solutions Code Sol Self-Grades
Sep 22 Computer Vision Recording Scribe Notes
5 Sep 27 Graph Neural Networks, Dropout Recording Scribe Notes Worksheet Solutions HW4 - GNNs and RNNs Written Coding Solutions Code Sol Self-Grades
Sep 29 GNNs, Recurrent Neural Networks Recording Scribe Notes
6 Oct 4 Guest Lecture: CV in Practice Recording Slides Scribe Notes Worksheet Solutions HW5 - Dropout and Inductive Bias Written Coding Solutions Code Sol Self-Grades
Oct 6 RNNs, LSTMs Recording Scribe Notes
7 Oct 11 Seq-to-Seq Models, Attention Recording Scribe Notes Worksheet Solutions HW6 - Seq-to-Seq, Attention and Self-supervision
Written Coding Solutions Code Sol Self-Grades
Oct 13 Self-supervision, Autoencoders Recording Scribe Notes
8 Oct 18 Transformers Recording Scribe Notes Worksheet Solutions
Oct 20 Transformers Recording Scribe Notes
9 Oct 25 Transformers Recording Scribe Notes Review Sessions
Basics Worksheet Solutions
CNNs/GNNs Worksheet Solutions
Seq2Seq Worksheet Solution 		Midterm Exam Solutions
Oct 27 Meta-learning, fine-tuning, transfer Recording Scribe Notes
10 Nov 1 Meta-learning, fine-tuning, transfer Recording Scribe Notes Worksheet Notes Solutions HW7 - Redo Midterm + Transformers
Written Coding Solutions Code Sol Self-Grades
Nov 3 Meta-learning, fine-tuning, transfer Recording Scribe Notes
11 Nov 8 Meta-learning, fine-tuning, transfer Recording Scribe Notes Worksheet Solutions HW8 Transformers
Written Coding Solutions Code Sol Self-Grades
Nov 10 Meta-learning, fine-tuning, transfer Recording Scribe Notes
12 Nov 15 Generative Models Recording Scribe Notes Worksheet Solutions HW9 Autoencoders, Meta-Learning
Written Coding Solutions Code Sol Self-Grades
Nov 17 Generative Models Recording Scribe Notes
13 Nov 22 Generative Models Recording Scribe Notes HW10
Prompt Engineering, Generative Models, Continual Learning
Written Coding Solutions Code Sol Self-Grades
Nov 24 No lecture - Thanksgiving
14 Nov 29 Generative Models Recording Scribe Notes Worksheet Solutions
Dec 1 No Lecture
15 Dec 6 Generative Models Recording Transformer/Finetuning Review Transformer/Finetuning Solutions Transformer Review Slides Generative Models
Dec 8 RRR Week
16 Dec 13 Final Exam: Tue, Dec 13, 3pm - 6pm