University of California, Berkeley
Electrical Engineering and Computer Sciences Department
Course Details | Announcements | Lecture Notes | Literature Reading | Homework | Course Handouts | Useful Links |
 

EE225B, Spring 2019
Digital Image Processing

Wed. 1:00 - 4:00 pm
540 Cory

Prerequisite:   EE120

Required Text:

  1. R. C. Gonzalez and R. E. Woods, Digital Image Processing, Prentice Hall, 4th Edition.

Video lectures:
EE225B, Spring 2006

Course Details:

Lecturer:
Professor Avideh Zakhor
avz@eecs.berkeley.edu
507 Cory Hall
Phone: (510) 643-6777

Office Hours:
Wed. 4:00pm - 5:00pm in 507 Cory

TA:
Luya Zhang
luyazhang@berkeley.edu

Office Hours:
Tue. 12:00 - 1:00 pm in Cory 504;
Fri. 4:30 - 5:30 pm in Cory 504.


 

Recommended Books:

  1. Bovik, Handbook of Image and Video Processing, Academic Press 2000.
  2. N. Netravali and Barry G. Haskell, Digital Pictures, Plenum Press, 1988.
  3. W.K.Pratt, Digital Image Processing, John Wiley and Sons, 1992.
  4. A.M. Tekalp, Digital Video Processing, Prentice Hall, 1995.
  5. Szeliski, Richard. Computer vision: algorithms and applications. Springer Science & Business Media, 2010. PDF
  6. Ponce, Jean, et al. Computer vision: a modern approach. Computer 16.11 (2011). PDF

Other useful resources online: Coursera course

  1. Fundamentals of Digital Image and Video Processing
  2. Deep Learning in Computer Vision
  3. Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital

Outline of Topics:

  1. Image sensing and acquisition, sampling, quantization
  2. Spatial transformations, filtering in space domain and frequency domain.
  3. Image restoration, enhancement, reconstruction; computed tomography
  4. Wavelets and multi-resolution processing
  5. Compressed sensing; Sparse signal representation
  6. Image and video compression and communication; watermarking
  7. Morphological Image processing
  8. Color processing
  9. Edge detection; feature extraction; SIFT, MSER
  10. Image segmentation
  11. Neural networks and deep learning

Homework:

Homework will be issued approximately once a week. They will either consist of written assignments, Matlab assignments or C programming assignments. Homework will be graded, and will contribute 55% to the final grade, where 10% is for reading assignments and 45 percent for actual homework. Homework handed in late will not be accepted unless consent is obtained from the Professor prior to the due date.
There will be a project that will constitute 35% of your grade. The project can be individual or in a group. You are to submit a proposal to the instructor by the end of March. More details on the project will be provided later, and a list of suggested topics will be provided. In addition, 10% of your grade will be for in class participation. The participation will be determined by 15 minute presentation of teams of 2 students on review of two or three research papers assigned by the instructor.


Announcements:

  • Welcome to EE225B!

  • 01/18/19: Instructions on how to get matlab are here.

  • Summer Internship Opportunity
    Fyusion is looking for talented interns during Spring, Summer and Fall of 2019 in San Francisco, CA. We are looking for motivated MS/PhD students (or talented senior undergraduates) who are excited about pushing the state of the art in areas such as computer vision (segmentation, tracking, 3D reconstruction), computational photography (HDR, denoising, light fields), image-based rendering, deep learning (GANs) and translating research into real world products.
    You will collaborate with one or more researchers (computer vision and robotics PhDs from Stanford, Berkeley, Georgia Tech, TUM, Willow Garage) with access to a world-class engineering team as well as large scale 3D datasets. Internships can be geared towards publishing at major conferences (CVPR, ICCV, ECCV, SIGGRAPH, NIPS) - our previous internships have resulted in shipped products and conference submissions (reach out to know more!).
    To apply, please send an email to research-internships@fyusion.com with your CV, a list of your research interests and anything else you would like us to know! We look forward to hearing from you!
    About us:
    Fyusion's vision is to combine the power of computer vision and machine learning to make images as intelligent as they are beautiful. Our technology gives deep super powers to images, making 3D photography more valuable than 2D imaging and our goal is to make it similarly pervasive. We are a VC funded computer vision company co-founded by a team of leading experts in computer vision, machine learning and robotics. Our core technology allows anyone to create immersive, interactive 3D images called 'fyuses' by moving any camera around a person, object or scene. You can read more about us here and get a glimpse of some our tech here.

  • Interesting New York Times Article on the Use of Infrared Sattelite Imagery
    I thought you would find this interesting;
    'Businesses Will Not Be Able to Hide': Spy Satellites May Give Edge From Above

  • Possible Class Project
    In case you are looking for a class project for EE225B this semester, here is a possibility. Last year there was an image compression challenge Google ran at CVPR. This year, they are doing a follow on workshop at CVPR , this time with two challenge tracks: the same low-bit-rate one as last year plus another "transparent compression" track. Here is a link in case you want to find out more.

  • Class Project on Activity Recognition
    I-ARPA has a grand challenge going on in activity recognition with a deadline of Feb. 28th.I realize it is a short fuse, but in case you wanted to do a project in this area, (even though you decide not to submit to the grand challenge), here is the link to more information.

  • Interesting Talk
    Title: Data-Driven Datasets: Deep Active Learning for Autonomous Vehicles and Beyond
    Speaker: Adam Lesnikowski
    Affiliation: NVIDIA
    Date and location: Friday, February 8, 12:30 - 1:30 pm; Wozniak Lounge (430 Soda Hall)
    Abstract: Data is the source code of the software 2.0 paradigm. So why has there been a tremendous amount of focus on neural network architectures and relatively little on dataset construction in the development of modern machine learning? The speaker believes that this focus is misplaced, with the largest future gains in data-driven machine learning systems for computer vision and other applications coming from improved data set building strategies rather than architecture improvements. In particular, employing feedback from trained models allows us to iteratively build datasets and models that in many cases leads to substantial improvements in performance with less labelled examples. The speaker will present several cases studies related to autonomous vehicle at NVIDIA where this paradigm has been exploited, and speculate on current and future areas of challenges and opportunities.

Back to top


Lecture Notes:

Back to top


Textbook Reading Schedule:

Back to top


Literature Reading:

Image Super-resolution papers:

  1. Dong, C., Loy, C. C., He, K., & Tang, X. (2016). Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2), 295-307. Link
  2. Kim, J., Kwon Lee, J., & Mu Lee, K. (2016). Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1646-1654). Link
  3. Wang, Z., Liu, D., Yang, J., Han, W., & Huang, T. (2015). Deep networks for image super-resolution with sparse prior. In Proceedings of the IEEE International Conference on Computer Vision (pp. 370-378). Link
  4. Tai, Y., Yang, J., & Liu, X. (2017, July). Image super-resolution via deep recursive residual network. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2790-2798). IEEE. Link

Image Denosing papers:

  1. Xie, J., Xu, L., & Chen, E. (2012). Image denoising and inpainting with deep neural networks. In Advances in neural information processing systems (pp. 341-349). Link
  2. Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017). Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Transactions on Image Processing, 26(7), 3142-3155. Link

Image Enhancement papers:

  1. Lore, K. G., Akintayo, A., & Sarkar, S. (2017). LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognition, 61, 650-662. Link
  2. Gharbi, M., Chen, J., Barron, J. T., Hasinoff, S. W., & Durand, F. (2017). Deep bilateral learning for real-time image enhancement. ACM Transactions on Graphics (TOG), 36(4), 118. Link

Image Restoration papers:

  1. Mao, X., Shen, C., & Yang, Y. B. (2016). Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In Advances in neural information processing systems (pp. 2802-2810). Link
  2. Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017, July). Learning deep CNN denoiser prior for image restoration. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. 2). Link

Image Retrieval papers:

  1. Zhao, F., Huang, Y., Wang, L., & Tan, T. (2015). Deep semantic ranking based hashing for multi-label image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1556-1564). Link
  2. Liu, H., Wang, R., Shan, S., & Chen, X. (2016). Deep supervised hashing for fast image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2064-2072). Link
  3. Lin, K., Yang, H. F., Hsiao, J. H., & Chen, C. S. (2015). Deep learning of binary hash codes for fast image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 27-35). Link
  4. Yue-Hei Ng, J., Yang, F., & Davis, L. S. (2015). Exploiting local features from deep networks for image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 53-61). Link

Image Segmentation papers:

  1. Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018). Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence, 40(4), 834-848. Link
  2. Badrinarayanan, V., Kendall, A., & Cipolla, R. (2015). Segnet: A deep convolutional encoder-decoder architecture for image segmentation. arXiv preprint arXiv:1511.00561. Link

Object Detection papers:

  1. Girshick, R. (2015). Fast r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 1440-1448). Link
  2. Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems (pp. 91-99). Link
  3. He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017, October). Mask r-cnn. In Computer Vision (ICCV), 2017 IEEE International Conference on (pp. 2980-2988). IEEE. Link

Back to top


Homework:

Homework will be submitted through Gradescope. Send your name and email address to the GSI if you are not enrolled into Gradescope.
Zip and submit source codes to ee225bsp19@gmail.com.. The subject of the email should be 'FirstName_LastName_HW#'.

    1. Homework 1: Review the following papers, see Announcements at the top of the page for more information.

      1. Dong, C., Loy, C. C., He, K., & Tang, X. (2016). Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2), 295-307. Link
      2. Kim, J., Kwon Lee, J., & Mu Lee, K. (2016). Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1646-1654). Link
      You need to submit that through gradescope no later than 12 noon Wed. Jan 30th.
    2. Homework 2:
      It is due on Feb 6th 12:00pm. Download the homework description here. The images for the homework can be found here, and in case you don't have the book yet, the homework problems can be downloaded from here.

      Homework 2 solutions are available.

    3. Homework 3:
      Reading assignment this week, is paper 5 on the class web page. One page write up is due at 12:00pm noon on Wednesday Feb. 6th.

      1. Xie, J., Xu, L., & Chen, E. (2012). Image denoising and inpainting with deep neural networks. In Advances in neural information processing systems (pp. 341-349). Link
    4. Homework 4:
      Reading assignment this week, is paper 7 and 8 on the class web page. One page write up is due at 12:00pm noon on Wednesday Feb. 13th.

      1. Lore, K. G., Akintayo, A., & Sarkar, S. (2017). LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognition, 61, 650-662. Link
      2. Gharbi, M., Chen, J., Barron, J. T., Hasinoff, S. W., & Durand, F. (2017). Deep bilateral learning for real-time image enhancement. ACM Transactions on Graphics (TOG), 36(4), 118. Link
    5. Homework 5:
      It is due on Feb 13th 12:00pm. Download the homework description here.
      Project 3-3 is here, Project 3-7 is here, Project 3-8, 3-9 is here. Download checkerboard1024-shaded.tif, hidden-horse.tif, spillway-dark.tif, testpattern1024.tif, blurry-moon.tif,
      expected results for project 3.7 (c) is here and here

      Homework 5 solutions are available.

    6. Homework 6:
      Reading assignment this week, is paper 17 and 18 on the class web page. One page write up is due at 12:00pm noon on Wednesday Feb. 20th. The homework requirement can be found here.

    7. Homework 7:
      It is due on Feb 20th 12:00pm. Download the homework description here. If you don't have the textbook, you can download the homework questions here.

      Homework 7 solutions are available.

    8. Homework 8:
      Reading assignment this week, is paper 9 and 10 on the class web page. One page write up is due at 12:00pm noon on Wednesday Feb. 27th. The homework requirement can be found here.

    9. Homework 9:
      It is due on Feb 27th 12:00pm. Download the homework description here. If you don't have the textbook, you can download the homework questions here.

      Homework 9 solutions are available.

    10. Homework 10:
      It is due on March 6th 12:00pm. Download the homework description here. Please refer to the image restoration tutorial for further information. You will need the following images for this homework: NoisyImg.bmp,  NoisyBlur.bmp

      Homework 10 solutions are available.

    11. Homework 11:
      It is due on March 13th 12:00pm. Download the homework description here. Please refer to the tomography lecture slides for further information. You will need the following image for this homework: Pyramid.bmp

      Homework 11 solutions are available.

    12. Homework 12:
      It is due on April 3th 12:00pm. Download the homework description here. You will need the following image for this homework: hw12 images.

      Homework 12 solutions are available.

    13. Homework 13:
      Reading assignment this week, is paper [1] and [2]. It is due at 12:00pm noon on Wednesday April 10th.

    14. Homework 14:
      It is due on April 17th 12:00pm. Download the homework description here. You will need the following image for this homework: hw14 images.

      Homework 14 solutions are available.

    15. Homework 15:
      It is due on April 24th 12:00pm. Download the homework description here. You will need the following image for this homework: Phase.dat, Magnitude.dat, Test.bmp. Please also read this when you work on the homework.

      Homework 15 solutions are available.

Back to top


Handouts:

  • Signals, Systems and Fourier Transform
  • Multi-Dimensional Fourier Transform
  • Image Restoration
  • Embedded Image Coding Using Zerotrees of Wavelets Coefficients
  • Review of Algorithms for Reconstruction of images from Fourier Transform Magnitude

    Back to top


    Useful Links:
  • Mars and other interesting large images
  • Mistakes in the movies
  • Image processing and HIV research
  • Optical illusions
  • TED Talk: A cinematic journey through visual effects
  • Slides from the book by Gonzalez
  • Some popular image processing packages
    1. Image Processing On Line (IPOL)
    2. Matlab (general purpose)
    3. ITK (medical imaging)
    4. FSL (brain imaging)
    5. Adobe Products
    6. Python Imaging Library
    7. OpenCV
    8. Sparse Modeling 1
    9. Sparse Modeling 2
  • Back to top


     Last updated 1/18/19