Hardware for Machine Learning

AI Hardware
Course Overview

Machine learning has emerged to be one of the key approaches to solving complex cognition and learning problems. Deep neural networks, in particular, have become pervasive due to their successes across various applications, including computer vision, speech recognition, natural language processing, etc. This has been especially true in the recent years with the release of large language models such as ChatGPT, LLama, etc. Can you tell if this text was written with ChatGPT or not? While machine learning algorithms deliver impressive accuracy on many deployment scenarios, the complexity of the algorithms also poses a unique computational and energy efficiency challenge to state-of-the-art hardware design, especially on edge devices.

To this end, this course is designed to help students come up to speed on various aspects of hardware, software, and hardware-software co-design for machine learning. The course will explore the basics of deep learning models, deep learning frameworks, computer architecture for deep learning, hardware accelerators, co-optimization of algorithms and hardware, design space exploration, training, inference, and support for state-of-the-art deep learning networks. In particular, this course is structured around building hardware prototypes for machine learning systems using state-of-the-art platforms (e.g., FPGAs and ASICs). It's also a seminar-style course, so students are expected to present, discuss, and interact with research papers. At the end of the semester, students will present their work based on a class research project.

Instructor
Sophia Shao Ph.D.

Professor Sophia Shao

ysshao at berkeley dot edu

Co-instructor
Vikram Jain Ph.D.

Dr. Vikram Jain

vikramj at berkeley dot edu

Teaching Assistant
Vadim (Dima) Nikiforov

Vadim (Dima) Nikiforov

vnikiforov at berkeley dot edu

Guest Speakers
Bita Darvish Rouhani

Bita Darvish Rouhani

Distinguished Engineer, NVIDIA

Blaise-Pascal Tine

Blaise-Pascal Tine

Assistant Professor, UCLA

Jonathan Ross

Jonathan Ross

CEO, Founder, Groq

Richard Ho

Richard Ho

Head of Hardware, OpenAI

Lectures and Office Hours
Lectures Mon, Wed 9:30 am - 11:00 am Cory 293/Zoom Prof. Sophia Shao, Dr. Vikram Jain
Instructor Office Hours Tue 10:00 am - 11:00 am Cory 570 Prof. Sophia Shao
Instructor Office Hours Wed 2:00 pm - 3:00 pm Soda 580 Dr. Vikram Jain
TA Office Hours Tue 5:00 pm - 6:00 pm Soda 373 Dima Nikiforov
Grading
Readings 30 %
Labs 40 %
Projects 30 %
Course participation (e.g., Ed, Gemmini/Chipyard code contributions) 5% (extra credit)
  • Late day policy: Each student has a total of 14 late days for the entire semester. Late days can be used for labs, paper reviews, and project checkoffs but not for the project presentation and report.
  • Students may choose to allocate all 14 late days to one submission or evenly distribute them across multiple submissions.
  • If a student doesn’t have any remaining late days, late submissions will incur a 10% penalty per day.
  • The late days for a team of 2-3 students during the project phase are the minimum remaining late days of the team members.
Readings
Labs
  • Ask questions on our Ed forum.
  • Labs will be posted as links in the outline below. Please submit completed labs via Gradescope.
  • Labs will be challenging and graded for correctness.
Course Outline
Week Date Lecture Topic Paper Reading Labs/Projects
1 1/17 Class Introduction slides recording
2 1/22 Introduction to DNN + Quantization slides recording Lab 1 (due 2/2)
1/27 Kernel Computation slides recording cuDNN, arXiv'2014
3 1/29 Guest lecture from Bita Rouhani, NVIDIA recording
1/31 Paper Discussion: Accelerators Source of Inefficiency, ISCA'2010
4 2/4 Paper Discussion: Gemmini slides Gemmini, DAC'2021 Lab 2 (due 2/16)
2/7 Dataflow slides Eyeriss, ISCA'2016
5 2/12 Paper Discussion: MMA in POWER10 AI-Infused POWER10, ISCA'2021
2/14 Paper Discussion: Ara2 Ara2, 2023
6 2/19 Presidents' Day. Lab 3 (due 3/3)
2/23 ISSCC. No class!
7 2/26 Vortex GPU Guest Lecture slides recording Vortex, MICRO'21
2/28 TPUs slides Ten Lessons from TPUs, ISCA'21
Resources
Textbooks/Courses
Hardware
Tutorials
Honor Code
  • If you turn in someone else’s work as if it were your own, you are guilty of academic misconduct. This includes problem sets, answers on exams, lab exercise checks, project design, and any required course turn-in material.
  • Also, if you knowingly aid in academic misconduct, you are guilty.
  • We have software that compares your submitted work to others.
  • However, it is okay to discuss with others lab exercises and the project (obviously, okay to work with project partner). Okay to discuss homework with others. But everyone must turn in their own work.
  • Do not post your work on public repositories like github (private o.k.)
  • If we catch your academic misconduct, you will get negative points on the assignment: It is better to not do the work than to cheat! If it is a midterm exam, final exam, or final project, you get an F in the class. All cases of academic misconducts reported to the office of student conduct.
It is very important to the EECS Department that every student in the EECS community feels safe, respected and welcome. We recognize though that incidents happen, sometimes unintentionally, that run counter to that goal. There are many things we can do to try to improve department climate, but we need to understand where the challenges lie. If you experience a remark, or disrespectful treatment, or if you feel you are being ignored, excluded or marginalized in a course or department-related activity, please speak up. If you witness such treatment toward someone else, please also say something. If you notice course content (such as lectures, discussions, assignments, exams) that marginalize or alienate you, please point them out. This is particularly important if the offending actions/content are based on gender, race, age, disability, or other protected category but it’s not always possible to tell and so we encourage you to share with us in any case. The more we hear, the more we can do to improve the climate in our courses and department-supported activities. We also welcome suggestions on things we could do to improve department climate.