Hardware for Machine Learning

Machine learning has emerged to be one of the key approaches to solving complex cognition and learning problems. Deep neural networks, in particular, have become pervasive due to their successes across various applications, including computer vision, speech recognition, natural language processing, etc. This has been especially true in the recent years with the release of large language models such as ChatGPT, LLama, etc. Can you tell if this text was written with ChatGPT or not? While machine learning algorithms deliver impressive accuracy on many deployment scenarios, the complexity of the algorithms also poses a unique computational and energy efficiency challenge to state-of-the-art hardware design, especially on edge devices.

To this end, this course is designed to help students come up to speed on various aspects of hardware, software, and hardware-software co-design for machine learning. The course will explore the basics of deep learning models, deep learning frameworks, computer architecture for deep learning, hardware accelerators, co-optimization of algorithms and hardware, design space exploration, training, inference, and support for state-of-the-art deep learning networks. In particular, this course is structured around building hardware prototypes for machine learning systems using state-of-the-art platforms (e.g., FPGAs and ASICs). It's also a seminar-style course, so students are expected to present, discuss, and interact with research papers. At the end of the semester, students will present their work based on a class research project.

Lectures	Mon, Wed	9:30 am - 11:00 am	Cory 293/Zoom	Prof. Sophia Shao, Dr. Vikram Jain
Instructor Office Hours	Tue	10:00 am - 11:00 am	Cory 570	Prof. Sophia Shao
Instructor Office Hours	Wed	2:00 pm - 3:00 pm	Soda 580	Dr. Vikram Jain
TA Office Hours	Tue	5:00 pm - 6:00 pm	Soda 373	Dima Nikiforov

Readings	30 %
Labs	40 %
Projects	30 %
Course participation (e.g., Ed, Gemmini/Chipyard code contributions)	5% (extra credit)

• Late day policy: Each student has a total of 14 late days for the entire semester. Late days can be used for labs, paper reviews, and project checkoffs but not for the project presentation and report.
• Students may choose to allocate all 14 late days to one submission or evenly distribute them across multiple submissions.
• If a student doesn’t have any remaining late days, late submissions will incur a 10% penalty per day.
• The late days for a team of 2-3 students during the project phase are the minimum remaining late days of the team members.

• Please signup for paper review via Paper Signup.
• Please submit review via HotCRP.

• Ask questions on our Ed forum.
• Labs will be posted as links in the outline below. Please submit completed labs via Gradescope.
• Labs will be challenging and graded for correctness.

Week	Date	Lecture Topic	Paper Reading	Labs/Projects
1	1/17	Class Introduction slides recording
2	1/22	Introduction to DNN + Quantization slides recording		Lab 1 (due 2/2)
	1/27	Kernel Computation slides recording	cuDNN, arXiv'2014
3	1/29	Guest lecture from Bita Rouhani, NVIDIA recording
	1/31	Paper Discussion: Accelerators	Source of Inefficiency, ISCA'2010
4	2/4	Paper Discussion: Gemmini slides	Gemmini, DAC'2021	Lab 2 (due 2/16)
	2/7	Dataflow slides	Eyeriss, ISCA'2016
5	2/12	Paper Discussion: MMA in POWER10	AI-Infused POWER10, ISCA'2021
	2/14	Paper Discussion: Ara2	Ara2, 2023
6	2/19	Presidents' Day.		Lab 3 (due 3/3)
	2/23	ISSCC. No class!
7	2/26	Vortex GPU Guest Lecture slides recording	Vortex, MICRO'21
	2/28	TPUs slides	Ten Lessons from TPUs, ISCA'21
8	3/4	Paper Discussion: Accelerator Integration slides	AuRORA, MICRO'23
	3/6	Transformers slides recording	Transformer Accelerator, JSSC'23	Lab 4 (due 3/19)
9	3/11	Paper Discussion: Accelerator Evaluation slides recording	RoSE, ISCA'23
	3/13	On-device Training slides recording	Cambricon-Q, ISCA'21
10	3/18	HW-SW Co-design slides recording	Timeloop, ISPASS'19
	3/20	Guest Lecture on Energy-Efficient ML for Metaverse, Huichu Liu and Lita Yang, Meta slides recording		Lab 5 (due 4/2)
11	3/25	Spring Break
	3/27	Spring Break
12	4/1	Guest lecture on Democratizing Gen-AI with Hardware-Software Co-design for Efficient Large Language Model (LLM) Inference, Sudeep Bhoja, Akhil Arunkumar, Aayush Ankit, d-Matrix
	4/3	Near and In-memory Computing slides recording	DIANA, JSSC'23
13	4/8	Paper Discussion: TinyML	Benchmarking TinyML Systems, 2021
	4/10	Chiplets and the pursuit of more compute slides recording	Cerebras Deep Dive, MICRO '23
14	4/15	Guest lecture on On-device ML for 5G Modems, Ankita Nayak, Qualcomm
	4/17	Finale slides

• If you turn in someone else’s work as if it were your own, you are guilty of academic misconduct. This includes problem sets, answers on exams, lab exercise checks, project design, and any required course turn-in material.
• Also, if you knowingly aid in academic misconduct, you are guilty.
• We have software that compares your submitted work to others.
• However, it is okay to discuss with others lab exercises and the project (obviously, okay to work with project partner). Okay to discuss homework with others. But everyone must turn in their own work.
• Do not post your work on public repositories like github (private o.k.)
• If we catch your academic misconduct, you will get negative points on the assignment: It is better to not do the work than to cheat! If it is a midterm exam, final exam, or final project, you get an F in the class. All cases of academic misconducts reported to the office of student conduct.

It is very important to the EECS Department that every student in the EECS community feels safe, respected and welcome. We recognize though that incidents happen, sometimes unintentionally, that run counter to that goal. There are many things we can do to try to improve department climate, but we need to understand where the challenges lie. If you experience a remark, or disrespectful treatment, or if you feel you are being ignored, excluded or marginalized in a course or department-related activity, please speak up. If you witness such treatment toward someone else, please also say something. If you notice course content (such as lectures, discussions, assignments, exams) that marginalize or alienate you, please point them out. This is particularly important if the offending actions/content are based on gender, race, age, disability, or other protected category but it’s not always possible to tell and so we encourage you to share with us in any case. The more we hear, the more we can do to improve the climate in our courses and department-supported activities. We also welcome suggestions on things we could do to improve department climate.