EECS 127/227A  Optimization Models in Engineering
Spring 2020
Schedule
This schedule is tentative and subject to change.(Please scroll horizontally if you're viewing this on your phone.)
Week  Date  Lecture Topic  Reading  Section  Homework 

0 
01/21 Tu 
Introduction
(Notes) (Slides) 
C and EG Ch. 1 
Section 0
(Dis) (Sol) (Laura/Vignesh Slides) 

01/23 Th 
Leastsquares/linear algebra review
(Notes) 
C and EG Ch. 2, 3  
1 
01/28 Tu 
Linear algebra review: PCA and symmetric matrices
(Notes) 
C and EG Ch. 4 
Section 1
(Dis) (Sol) 
Homework 1
Due 01/31 Fr (PDF) Self grades Due 02/07 Fr (Sols) (Self Grade) 
01/30 Th 
Singular value decomposition
(Notes) (Notebook) 
C and EG Ch. 5  
2 
02/04 Tu 
Lowrank approximation
(Notes) (Extended notes on Minimum Norm and LowRank Approximation) 
C and EG Sec 5.3. Also https://en.wikipedia.org/wiki/Lowrank_approximation. 
Section 2
(Dis) (Sol) 
Homework 2
Due 02/07 Fr (PDF) (Code) Self grades Due 02/14 Fr (Sols) (Code Sols) (Self Grade) 
02/06 Th 
Vector calculus
(Notes) 
B and V Appendix A4  
3 
02/11 Tu 
Lowrank approximation
(Notes) 
C and EG Sec 5.3. Also https://en.wikipedia.org/wiki/Lowrank_approximation. 
Section 3
(Dis) (Sol) 
Homework 3
Due 02/14 Fr (PDF) (Code) Self grades Due 02/21 Fr (Sols) (Code Sols) (Self Grade) 
02/13 Th 
Leastsquares and variants: Ridge regression
(Notes) (Notebook) 
C and EG: Ch 6  
4 
02/18 Tu 
Leastsquares and variants
(Notes) (Notebook) 
C and EG Ch. 8.18.3, B and V Ch. 3 
Section 4
(Dis) (Sol) 
Homework 4
Due 02/21 Fr (PDF) (Code) Self grades Due 02/28 Fr (Sols) (Code Sols) (Self Grade) 
02/20 Th 
Convexity
(Notes) 
C and EG: Ch 8.18.3, B and V, Ch 4  
5 
02/25 Tu 
Convex optimization
(Notes) 
C and EG Ch. 12.2 
Section 5
(Dis) (Sol) 
Homework 5
Due 02/28 Fr (PDF) Self grades Due 03/06 Fr (Sols) (Self Grade) 
02/27 Th 
Gradient Descent
(Notes) 
C and EG: Ch 12.2  
6 
03/03 Tu 
Weak duality
(Notes) (Webcast) 
C and EG: Ch 8.5, B and V: Ch 5 
Section 6
(Dis) (Sol) 
Homework 6
Due 03/06 Fr (PDF) Self grades Due 03/13 Fr (Sols) (Self Grade) 
03/05 Th 
Strong duality
(Notes) (Notebook) (Webcast) 
C and EG: Ch 8.5, B and V: Ch 5  
7 TakeHome Midterm 03/12 5:107:15pm 
03/10 Tu 
Duality examples (Logistic regression) and Total Least Squares
(Notes) (Webcast) 
Section 7
(Dis) (Sol) (Friday) (Monday) 
Homework 7
Optional (PDF) (Sols) 

03/12 Th 
Midterm
(Exam Answer Template) 

8 
03/17 Tu 
KKT, optimality conditions
(Notes) (Webcast) 
B and V 5.5 
Section 8
(Dis) (Sol) (Friday) (Friday Int'l) 
Homework 8
Due 03/20 Fr (PDF) Self grades Due 03/27 Fr (Sols) (Self Grade) 
03/19 Th 
KKT + formulating optimization problems
(Notes) (Webcast) 
B and V 5.5  

Spring Break!


9 
03/31 Tu 
LPs
(Notes) (Webcast) 
C & EG: Ch. 9 
Section 9
(Dis) (Sol) (Laura's Notes) (Theo's Notes) (Laura) (Theo) 
Homework 9
Due 04/03 Fr (PDF) (Code) Self grades Due 04/10 Fr (Sols) (Code Sols) (Self Grade) 
04/02 Th 
L1 Norms, LASSO
(Notes) (Webcast) 
C & EG: 9.6.2, 13.4, 12.5  
10 
04/07 Tu 
More LASSO, QPs
(Notes) (Webcast) 
C & EG: Ch. 10 
Section 10
(Dis) (Sol) (Theo's Notes) (Vignesh's Notes) (Theo) (Vignesh) 
Homework 10
Due 04/10 Fr (PDF) (Code) Self grades Due 04/17 Fr (Sols) (Code Sols) (Self Grade) 
04/09 Th 
QPs
(Notes) (Webcast) 
C & EG: 10  
11 
04/14 Tu 
SOCPs, Newton's Method
(Notes) (Webcast) 
C & EG: 12.2.4 
Section 11
(Dis) (Sol) (Theo's Notes) (Laura's Notes) (Theo) (Laura) 
Homework 11
Due 04/17 Fr (PDF) (Code) (Sols) (Code Sols) (Self Grade) 
04/16 Th 
LQR/Control
(Notes) (Webcast) 

12 
04/21 Tu 
SVM
(Notes) (Webcast) 
Section 12
(Dis) (Sol) (Theo's Notes) (Vignesh's Notes) (Theo) (Vignesh) 
Homework 12
Due 04/25 Sa (PDF) (Code) Self grades Due 05/02 Sa (Sols) (Code Sols) (Self Grade) 

04/23 Th 
SVM
(Notes) (Webcast) 

13 
04/28 Tu 
Guest Lecture: Deep learning, linear algebra, optimization. Laurent El Ghaoui
(Slides) (Webcast) 
Homework 13
Optional (PDF) (Sols) 

04/30 Th 
Final Assessment
(Exam Outline) 

14 
05/05 Tu 


05/07 Th 


15 
05/15 Fr 
No Final Exam

Course Staff
Instructors
GSIs
Grace Zhang
Admin Head / Discussion TA
grace.zhang@
Suvansh Sanjeev
Admin Head / Discussion TA
suvansh@
Vignesh Subramanian
Content Head / Discussion TA
vignesh.subramanian@
Theo Cabannes
Content TA
theophile@
Armin Askari
Content TA
aaskari@
Elena Jia
Content TA
yiranjia@
Wilson Wu
Content TA
wilswu@
Yeshwanth Cherapanamjeri
Content / Discussion TA
yeshwanth@
Sean Farhat
Content TA
sfarhat@
Laura Hallock
Content / Discussion TA
laura.the.gsi@gmail.com
Sampada Deglurkar
Content TA
sampada_deglurkar@
David Yi
Reader
dyi@
Navneedh Maudgalya
Reader
navneedhm@
Ana Tudor
Reader
anamtudor@
Mandi Zhao
Reader
mandi.zhao@
Victor Li
Reader
lytz@
Daniel Jing
Reader
daniel.jing@
Please add berkeley.edu to the end of all emails, unless otherwise specified.
Resources
Piazza: ask questions here.Projects
Textbooks
 Optimization Models in Engineering (Giuseppe Calafiore and Laurent El Ghaoui)
 Convex Optimization (Stephen Boyd and Lieven Vandenberghe)
Prof. El Ghaoui's Slides
 Introduction
 Vectors
 Matrices
 Matrices (contd.)
 Symmetric Matrices
 Singular Value Decomposition
 Linear Equations
 Least Squares and Variants
 Linear Programs
 Convex Quadratic Programs
 Robust Optimization Models
 Review: Conic Models
 Convexity
 Convex Optimization Problems
 Weak Duality
 Strong Duality
 Optimality Conditions
Policies
Course Info
Course Title: Optimization Models in Engineering
Units: 4
Course Format: 3 hours of lecture, 1 hour discussion
Instructors: Gireeja Ranade
Contact Information: ee127227sp20@gmail.com, gireeja@eecs.berkeley.edu
Prerequisites: EE 16A, EE 16B, CS 70, Math 53
Online resources: Piazza, Gradescope (MYJ5YZ), and course website
Course Description
This course offers an introduction to optimization models and their applications, ranging from machine learning and statistics to decisionmaking and control, with emphasis on numerically tractable problems, such as linear or constrained leastsquares optimization. (4 Units.)
Course Objectives
 Develop a practical understanding of the applications and limitations of optimization as a solution approach to engineering analysis and design.
 Develop an ability to use rapid prototyping software to guide optimization solutions.
Desired Course Outcome:
By the end of the term, students having taken Optimization Models in Engineering are expected to be able to:
 Understand the basic concepts of linear algebra: vectors, matrices, rank, projections; symmetric matrices, positive semidefinite matrices, eigenvalues; singular value decomposition and principal component analysis.
 Understand three basic optimization models: leastsquares, linear programming, and quadratic programming; and develop an understanding of the more general convex optimization.
 Be aware of the wideranging applications where optimization models are useful, including in machine learning applications.
 Be able to use prototyping software such as CVX to develop optimizationbased solutions in concrete applications.
Course Content
The course content will roughly follow the outline posted on the website, but is subject to change and evolve throughout the semester.
 Eigenvalue and singular value decomposition
 Linear equations
 Least squares
 Convexity
 Gradient descent
 Linear programming
 Quadratic programming
 Duality
 Optimality conditions
 Applications: machine learning
 Applications: control
Some application topics might be added midsemester. It is strongly recommended that the students use the book Optimization Models, G.C. Calafiore and L. El Ghaoui, Cambridge University Press, October 2014.
Discussion
Discussion sections will take place every week. You may attend any one of the following sections, the material here is intended to help you with the class. You are strongly encouraged to attend and ask questions. Discussions will begin on Friday January 24, 2020. Discussion frequency and policy is as follows:
 Monday 10:00am  11:00pm, McCone 141, TA: Yeshwanth
 Monday 3:00pm  4:00pm, Tan 180, TA: Grace
 Monday 4:00pm  5:00pm, Etcheverry 3106, TA: Suvansh
 Friday 11:00pm  12:00pm, Goldman School 150, TA: Theo
 Friday 2:00pm  3:00pm, Barrows 126, TA: Laura/Vignesh
 Friday 3:00pm  4:00pm, Barrows 20, TA: Laura/Vignesh
Discussion material will be released on Friday.
Solutions to the discussion material will be released on Monday after the last discussion of the week. Please also see our link to Google Calendar.
Homework
Homework will be assigned every Friday here on the website and will be due the following Friday at 11:00pm. Homework will be electronically submitted to Gradescope (entry code: MYJ5YZ) as follows:
 A single PDF consisting of the following components:
 Scanned handwritten work or a LaTeX file containing answers to non IPython questions
 A PDF of all iPython notebooks
 All IPython notebooks in .ipynb format
You will need to select the pages associated with each question through Gradescope. Late homework will not be accepted, but to account for the vicissitudes of life, we will drop the 2 lowest homework scores from the final grade calculation.
Discussion and collaboration, as opposed to copying, of homework is encouraged. In other words, you are encouraged to discuss the homework with your classmates but you must write your own derivations and do your own calculations, etc. Do not hesitate to ask the professor or the GSIs for clarifications and hints for the homework problems during Homework Parties and Office hours. We encourage cooperation rather than competition.
Homework Grading
The point of homework in this class is for you to learn the material. To help you in doing this each student will grade their own homework in addition to being graded by readers.
Self Grades HowTo
After the HW deadline, official solutions will be posted online. You will then be expected to read them and enter your own scores and comments for every part of every problem in the homework on a simple coarse scale.
0 = didn't attempt or very very wrong,
2 = got started and made some progress, but went off in the wrong direction or with no clear direction,
5 = right direction and got halfway there,
8 = mostly right but a minor thing missing or wrong,
10 = 100% correct.
Note: You must justify selfgrades of 2, 5 or 8 with a comment. Grades of 0 and 10 do not need to be justified. If you are really confused about how to grade a particular problem, you should post on Piazza. This is not supposed to be a stressful process. See this video for a walkthrough of the selfgrade process.
Course readers are going to be grading and sending you occasional comments. Because we have reader grades, we will catch any attempts at trying to inflate your own scores. This will be considered cheating and is definitely not worth the risk. Your own scores will be used in computing your final grade for the course, adjusted by taking into account reader scores so that everyone is fairly graded effectively on the same scale. For example, if we notice that you tend to give yourself 5s on questions where readers looking at your homeworks tend to give you 8s, we will apply an upward correction to adjust.
You can find the link to the self grade form (where you will enter these scores) on the website next to the solutions. After completing a self grade form, it will generate a formatted .txt file with your responses. Download this file and submit it to Gradescope. There will be an assignment “HW X Self Grade” on Gradescope that you should submit this to. After uploading, make sure the autograder matches the scores you gave yourself in the self grade form. Your selfgrades will be due on the following Friday after the homework deadline at 11:00 PM sharp. We will accept late selfgrades up to 25 hours late (Saturday 11:59PM) for a 10% penalty on the associated homework assignment. If you don't enter a proper grade by this deadline, you are giving yourself a zero on that assignment. Merely doing the homework is not enough, you must do the homework; turn it in on time; read the solutions; do the selfgrade; and turn it in on time. Unless all of these steps are done, you will get a zero for that assignment.
Exam Policies
The 127/227AT Spring 2020 semester will have one midterm and one final. The midterm time will be during class hours and date will be communicated in class. Current schedule (below) is TENTATIVE. The final will be held during the designated final exam slot released by campus. Makeup/alternate exams will not be scheduled under any circumstances.
Please plan for exams at these times and email us during the first two weeks of the semester per university policy if you know about any exam conflicts. If an emergency arises that conflicts with the exam times, email us as soon as possible at ee127227sp20@gmail.com. Emergency exam conflicts will be handled on a casebycase basis. We do not provide specific accommodations for conflict with other classes, conflict with personal activities, even planned in advance, extracurricular or not. We try to accommodate situations of personal hardship to the best of our abilities. If you skip an exam without coordinating with us beforehand, you will receive an automatic zero.
Grading Breakdown
The course grade will be based on the following evaluation:
 Homework (30%)
 Midterm (30%)
 Final (40%)
There will also be a project option. If you choose to do the project, we will calculate the following breakdown:
 Homework (30%)
 Project (15%)
 Midterm (25%)
 Final (30%)
Your final grade will be the maximum of the two grading schemes.
This policy rewards consistency and regular work, as well as final knowledge acquisition.
Redemption policy: Exams cover all material until the exam. Grade of midterm will be replaced with a weighted average of the final and midterm grade if it is higher:
 Clobbered midterm grade = max(midterm grade, 60% final grade + 40% midterm grade)
DSP
We are happy to accommodate students with special needs as determined by the DSP office. Please email us in the first two weeks of class to request appropriate accommodations. Please include your accommodation letter in this email.
Course Materials
RECOMMENDED:
 Calafiore, Giuseppe and El Ghaoui, Laurent. Optimization Models.
 Boyd, Stephen, and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
 Livebook (You can register on thelivebook platform for free)
Course Communication
The instructors and TAs will post announcements, clarifications, hints, etc. on Piazza. You must check the EECS127/227A Piazza page frequently throughout the term.
If you have a question, your best option is to post a message there. The staff will check the forum regularly, and other students will be able to help you too. When using the forum, please avoid offtopic discussions, and please do not post answers to homework questions before the homework is due. Also, always look for a convenient category to post the question to (for example, each homework will have its own category, so please post there). That will ensure you get the answer faster.
If your question is personal or not of interest to other students, you may mark your question as private on Piazza, so only the instructors will see it. If you wish to talk with one of us individually, you are also welcome to come to our office hours. Please reserve email for the questions you can't get answered in office hours, in discussion sections, or through the forum.
It can be challenging for the instructors to gauge how smoothly the class is going. We always welcome any feedback on what we could be doing better. If you would like to send any feedback, you can send an email to: ee127227sp20@gmail.com
Collaboration
We encourage you to work on homework problems in study groups of two to four people; however, you must always write up the solutions on your own. Similarly, you may use books or online resources to help solve homework problems, but you must always credit all such sources in your writeup, and you may never copy material verbatim. Using previous homework and exam solutions is strictly prohibited, and will be considered academic dishonesty. This is not how you want to start your career as an engineer.
We expect that most students can distinguish between helping other students and cheating. Explaining the meaning of a question, discussing a way of approaching a solution, or collaboratively exploring how to solve a problem within your group is an interaction that we encourage strongly. But you should write your homework solution strictly by yourself so that your hands and eyes can help you internalize the subject matter. You should acknowledge everyone whom you have worked with, or who has given you any significant ideas about the homework. This is good scholarly conduct.
Don't Be Afraid to Ask for Help
Are you struggling? Please come talk with us! The earlier we learn about your struggles, the more likely it is that we can help you. Waiting until right before an exam or the last few weeks of the semester to let us know about your problems is not an effective strategy  the later it is, the less we will be able to help you.
Even if you are convinced that you are the only person in the class who is struggling, please overcome any feelings of embarrassment or guilt, and come ask for help as soon as you need it – we can almost guarantee you're not the only person who feels this way. Don't hesitate to ask us for help – we really do care that you thrive!
Advice
The following tips are offered based on our experience.
Do the homework! The homework is explicitly designed to help you to learn the material as you go along. There is usually a strong correlation between homework scores and final grades in the class.
Keep up with lectures! Discussion sections, labs and homework all touch on portions of what we discuss in lecture. Students do much better if they stay on track with the course. That will also help you keep the pace with your homework and study group.
Take part to discussion sections! Discussion sections are not auxiliary lectures. They are an opportunity for interactive learning. The success of a discussion section depends largely on the willingness of students to participate actively in it. As with office hours, the better prepared you are for the discussion, the more you are likely to benefit from it.
Come to office hours! We love to talk to you and do a deep dive to help you understand the material better.
Form study groups! As stated above, you are encouraged to form small groups (two to four people) to work together on homework and on understanding the class material on a regular basis. In addition to being fun, this can save you a lot of time by generating ideas quickly and preventing you from getting hung up on some point or other. Of course, it is your responsibility to ensure that you contribute actively to the group; passive listening will likely not help you much. Also recall the caveat above, that you must write up your solutions on your own. We strongly advise you to spend some time on your own thinking about each problem before you meet with your study partners; this way, you will be in a position to compare ideas with your partners, and it will get you in practice for the exams. Make sure you work through all problems yourself, and that your final writeup is your own. Some groups try to split up the problems ("you do Problem 1, I'll do Problem 2, then we'll swap notes"); not only is this a punishable violation of our collaboration policies, it also ensures you will learn a lot less from this course.