Description: [SCS dragon logo]

CS194-26: Image Manipulation and Computational Photography
Computer Science Division
University of California Berkeley

INSTRUCTOR: Alexei (Alyosha) Efros (Office hours: Wed 11am-12pm at 724 Sutardja Dai Hall)
GSI: Taesung Park (Office hours: 4-5pm Thurs at Soda-Alcove-341A) and Shiry Ginosar (Office hours: 10am-11am Tuesdays at one of the long tables at Sutardja Dai Hall 2nd Floor Yali's).
UNIVERSITY UNITS: 4
SEMESTER: Fall 2018
WEB PAGE: http://inst.eecs.berkeley.edu/~cs194-26/fa18/
Q&A: Piazza Course Website
LOCATION: Valley Life Sciences Building (VLSB) 2060
TIME
: TueThu 5:00 PM-6:30 PM
MIDTERM: Nov 13th Tue in-class.

PREREQUISITES:
Programming experience (CS61B) and familiarity with linear algebra (MATH 54 or EE16A/B or Strang's online class) and calculus is assumed.  Some background in computer graphics, computer vision, or image processing is helpful.  This class does not significantly overlap with cs184 (Computer Graphics) and can be taken concurrently.
Note: if the system doesn't let you sign up, or puts you on the waitlist, do talk to me.

COURSE OVERVIEW:
Computational Photography is an emerging new field created by the convergence of computer graphics, computer vision and photography. Its role is to overcome the limitations of the traditional camera by using computational techniques to produce a richer, more vivid, perhaps more perceptually meaningful representation of our visual world.

The aim of this advanced undergraduate course is to study ways in which samples from the real world (images and video) can be used to generate compelling computer graphics imagery. We will learn how to acquire, represent, and render scenes from digitized photographs. Several popular image-based algorithms will be presented, with an emphasis on using these techniques to build practical systems. This hands-on emphasis will be reflected in the programming assignments, in which students will have the opportunity to acquire their own images of indoor and outdoor scenes and develop the image analysis and synthesis tools needed to render and view the scenes on the computer.

TOPICS TO BE COVERED:

  • Cameras, Image Formation
  • Visual Perception
  • Image and Video Processing (filtering, anti-aliasing, pyramids)
  • Image Manipulation (warping, morphing, mosaicing, matting, compositing)
  • Modeling and Synthesis with Visual Big Data
  • High Dynamic Range Imaging and Tone Mapping
  • Image-Based Lighting
  • Image-Based Rendering
  • Non-photorealistic Rendering


PROGRAMMING ASSIGNMENTS:       

Project 1: Images of the Russian Empire -- colorizing the Prokudin-Gorskii photo collection
Description: http://www.cs.cmu.edu/afs/andrew/scs/cs/15-463/pub/www/images/3-8086-left.jpg

See student submissions here

Class Choice Awards: Zeyana Musthafa

 

Project 2: Building a Pinhole Camera

camera obscura


See student submissions here

Class Choice Awards: Andrew Campbell

 

Project 3: Fun with Frequencies and Gradients

orple swimming

See student submissions here

Class Choice Awards: Andrew Campbell

 

Project 4: Face Morphing and Modelling a Photo Collection

morph

See student submissions here
Class Choice Awards: Vivian Liu
Also, see the class morph video.

Project 5: Depth Refocusing and Aperture Adjustment with Lightfield data

morphmorph


See student submissions here
Class Choice Awards: Eli Lipsitz

 

Project 6: (Auto)stitching and photo mosaics

stitching

See student submissions here
Class Choice Awards: Andrew Campbell

 

Final Project

multifredo

See student submissions (pre-canned projects) here
See student submissions (own proposed projects) here

TEXT:
There is a textbook that covers most (if not all) of the topics related to Computational Photography.  This will be the primary reference for the course:

            Computer Vision: Algorithms and Applications, Richard Szeliski, 2010

There is a number of other fine texts that you can use for general reference:

Photography (8th edition), London and Upton, (a great general guide to taking pictures)
Vision Science: Photons to Phenomenology, Stephen Palmer (great book on human visual perception)
Digital Image Processing, 2nd edition, Gonzalez and Woods (a good general image processing text)
The Art and Science of Digital Compositing, Ron Brinkmann (everything about compositing)
Multiple View Geometry in Computer Vision, Hartley & Zisserman (a bible on recovering 3D geometry)
The Computer Image, Watt and Policarpo (a nice "vision for graphics" text, somewhat dated)
3D Computer Graphics (3rd Edition), Watt (a good general graphics text)
Fundamentals of Computer Graphics, Peter Shirley (another good general graphics text)
Linear Algebra and its Applications, Gilbert Strang (a truly wonderful book on linear algebra)

CLASS NOTES
The instructor is extremely grateful to a large number of researchers for making their slides available for use in this course.  Steve Seitz and Rick Szeliski have been particularly kind in letting me use their wonderful lecture notes.  In addition, I would like to thank Paul Debevec, Stephen Palmer, Paul Heckbert, David Forsyth, Steve Marschner and others, as noted in the slides.  The instructor gladly gives permission to use and modify any of the slides for academic and research purposes. However, please do also acknowledge the original sources where appropriate.

   

TENTATIVE CLASS SCHEDULE:

CLASS DATE

TOPICS

Material

Thurs Aug 23

Introduction

Tues/Thurs
Aug 28

Capturing Light... in man and machine

Tues/Thurs
Aug 30, Sept 4

The Camera
Pinhole Camera

 

Tues/Thurs
Sept 4

Sampling and Reconstruction

Thurs
Sept 6

Sampling and Reconstruction

  • Slides : pdf ppt
  • Point Processing Slides: pdf pptx
  • Continue Szeliski Ch 3

Tues, Thurs
Sept 11, 13

Image Blending and Compositing

Tues, Thurs
Sept 18, 20

Point Processing and Image Warping

  • Point Processing Slides: pdf pptx
  • Image Warping Slides: pdf ppt
  • Continue Szeliski Ch 3, 2.1.2

Thurs, Tues
Sept 22, 25

Image Morphing

  • Slides: pdf ppt
  • Continue Szeliski Ch 3

Sept 27, Oct 2, 4

Data-driven Methods: Faces

Oct 9

Data-driven Methods: Video Textures

Oct 11

Data-driven Methods: Visual Data on the Internet

Oct 16

Deep Learning in Computational Photography

Oct 18, 23

Modeling Light

Oct 25

Homographies and Mosaics

  • Szeliski Ch 9
  • Slides: pdf ppt

Oct 30, Nov 1

More Mosaic Madness

Nov 6, 8

Automatic Alignment
auto

Nov 8

Multi-perspective Panoramas
multi-perspective

Tues Nov 13

2/3rds-term exam!

Nov 28

Image-based Lighting
1 2 3

 

What makes a great picture?
birds

CAMERAS:
Although it is not required, students are highly encouraged to obtain a digital camera for use in the course.

METHOD OF EVALUATION:
Grading will be based on a set of programming and written assignments (60%), a midterm exam (20%) and a final project (20%).  For the programming assignments, students will be allowed a total of 5 (five) late days per semester; each additional late day will incur a 10% penalty.

Students taking CS294-26 will also be required to submit a conference-style paper describing their final project.

PROGRAMMING RESOURCES:
Students will be encouraged to use either MATLAB (with the Image Processing Toolkit) or Python (with either scikit-image or opencv) as their primary computing platform.  Specific libraries in both languages offer tons of build-in image processing functions.  Here is a link to some useful MATLAB and Python resources compiled for this class.

PREVIOUS OFFERINGS OF THIS COURSE:
Previous offerings of this course can be found here.

SIMILAR COURSES IN OTHER UNIVERSITIES:

 

Page design courtesy of Doug James