Description: [SCS dragon logo]

CS194-26/294-26: Image Manipulation, Computer Vision and Computational Photography
Computer Science Division
University of California Berkeley

INSTRUCTOR: Alexei (Alyosha) Efros (Office hours: after lecture)
GSI: Ashish Kumar (Office hours: 5-6pm Wed at Soda Alcove-341B), Violet Fu (Office hours: 5pm-6pm Fri at Soda Alcove-341B), and Shivam Parikh (Office hours: 11am-1pm Mon at Cory 531).
UNIVERSITY UNITS: 4
SEMESTER: Spring 2020
WEB PAGE: http://inst.eecs.berkeley.edu/~cs194-26/fa20/
Q&A: Piazza Course Website
LOCATION: Hearst Field Annex A1
TIME
: TueThu 5:00 PM-6:30 PM
MIDTERM: April 16th, Thurs, during the class.

PREREQUISITES:
This is a heavily project-oriented class, therefore good programming proficiency (at least CS61B) is absolutely essential. Moreover, familiarity with linear algebra (MATH 54 or EE16A/B or Gilbert Strang's online class) and calculus are vital. Experience with neural networks (e.g. CS189) is a plus. For these taking CS294-26, consent of instructor is required to register (please sign up on the waitlist first).

COURSE DESCRIPTION:
The aim of this advanced undergraduate course is to introduce students to computing with visual data (images and video). We will cover acquisition, representation, and manipulation of visual information from digital photographs (image processing), image analysis and visual understanding (computer vision), and image synthesis (computational photography). Key algorithms will be presented, ranging from classical (e.g. Gaussian and Laplacian Pyramids) to contemporary (e.g. ConvNets, GANs), with an emphasis on using these techniques to build practical systems. This hands-on emphasis will be reflected in the programming assignments, in which students will have the opportunity to acquire their own images and develop, largely from scratch, the image analysis and synthesis tools for solving applications.

PROGRAMMING ASSIGNMENTS:       

Project 1: Images of the Russian Empire -- colorizing the Prokudin-Gorskii photo collection
Description: http://www.cs.cmu.edu/afs/andrew/scs/cs/15-463/pub/www/images/3-8086-left.jpg

See student submissions here

Class Choice Awards: Scott Shao

 

Project 2: Fun with Filters and Frequencies

orple

See student submissions here

 

Project 3: Face Morphing and Modelling a Photo Collection

morph

See student submissions here
Also, see the class morph video.

 

Project 4: Classification and Segmentation

morph

See student submissions here

 

Project 5: (Auto)stitching and photo mosaics

stitching

See student submissions here

 

Project 6: 3D geometry

TEXT:
There is a textbook that covers most (if not all) of the topics related to Computational Photography.  This will be the primary reference for the course:

            Computer Vision: Algorithms and Applications, Richard Szeliski, 2010

There is a number of other fine texts that you can use for general reference:

Computer Vision: A Modern Approach (2nd edition), Forsyth and Ponce (classic computer vision text)
Vision Science: Photons to Phenomenology, Stephen Palmer (great book on human visual perception)
Digital Image Processing, 2nd edition, Gonzalez and Woods (a good general image processing text)
Linear Algebra and its Applications, Gilbert Strang (a truly wonderful book on linear algebra)

CLASS NOTES
The instructor is extremely grateful to a large number of researchers for making their slides available for use in this course.  Steve Seitz and Rick Szeliski have been particularly kind in letting me use their wonderful lecture notes.  In addition, I would like to thank Paul Debevec, Stephen Palmer, Paul Heckbert, David Forsyth, Steve Marschner and others, as noted in the slides.  The instructor gladly gives permission to use and modify any of the slides for academic and research purposes. However, please do also acknowledge the original sources where appropriate.

   

TENTATIVE CLASS SCHEDULE:

CLASS DATE

TOPICS

Material

Tues Jan 21

Introduction

Thurs
Jan 23

Capturing Light... in man and machine

Tues
Jan 28

The Camera
Pinhole Camera

 

Tues/Thurs
Jan 30

Sampling and Reconstruction

Tues
Feb 4

Sampling and Reconstruction

  • Point Processing Slides: pdf pptx
  • Continue Szeliski Ch 3

Thurs
Feb 6

Derivative and Template Filters

  • Derivative and Template Filters Slides: pdf pptx

Tues
Feb 11

Image Blending and Compositing

Thurs
Feb 13

Compression and Gradient Domain

Tues
Feb 18

Edge Detection and Image Warping

Thurs
Feb 20

Image Morphing

  • Slides: pdf ppt
  • Continue Szeliski Ch 3

Feb 25

Data-driven Methods: Faces

Feb 27, Mar 3

Data-driven Methods: Video Textures

Thurs
Mar 5

Feature Learning with Neural Networks

Tues, Thurs
Mar 10, Mar 12

Convolutional Neural Networks

Mar 17, 19

Modeling Light

Mar 31, Apr 7

Homographies and Mosaics

CAMERAS:
Although it is not required, students are highly encouraged to obtain a digital camera for use in the course.

METHOD OF EVALUATION:
Grading will be based on a set of programming and written assignments (60%), a midterm exam + several in-class pop quizzes (20%) and a final project (20%).  For the programming assignments, students will be allowed a total of 5 (five) late days per semester; each additional late day will incur a 10% penalty.

Students taking CS294-26 will also be required to submit a conference-style paper describing their final project.

PROGRAMMING RESOURCES:
Students will be encouraged to use either MATLAB (with the Image Processing Toolkit) or Python (with either scikit-image or opencv) as their primary computing platform.  Specific libraries in both languages offer tons of build-in image processing functions.  Here is a link to some useful MATLAB and Python resources compiled for this class.

PREVIOUS OFFERINGS OF THIS COURSE:
Previous offerings of this course can be found here.

SIMILAR COURSES IN OTHER UNIVERSITIES:

 

Page design courtesy of Doug James