EECS 127/227A | Optimization Models in Engineering

Spring 2020

Schedule

This schedule is tentative and subject to change.
(Please scroll horizontally if you're viewing this on your phone.)

Calendar

Resources

Piazza: ask questions here.

Projects

• Project 1
• Project 2
• Project 3

Textbooks

• Optimization Models in Engineering (Giuseppe Calafiore and Laurent El Ghaoui)
• Convex Optimization (Stephen Boyd and Lieven Vandenberghe)

Prof. El Ghaoui's Slides

• Introduction
• Vectors
• Matrices
• Matrices (contd.)
• Symmetric Matrices
• Singular Value Decomposition
• Linear Equations
• Least Squares and Variants
• Linear Programs
• Convex Quadratic Programs
• Robust Optimization Models
• Review: Conic Models
• Convexity
• Convex Optimization Problems
• Weak Duality
• Strong Duality
• Optimality Conditions

Policies

Course Info

Course Title: Optimization Models in Engineering

Units: 4

Course Format: 3 hours of lecture, 1 hour discussion

Instructors: Gireeja Ranade

Contact Information: ee127227sp20@gmail.com, gireeja@eecs.berkeley.edu

Prerequisites: EE 16A, EE 16B, CS 70, Math 53

Online resources: Piazza, Gradescope (MYJ5YZ), and course website

Course Description

This course offers an introduction to optimization models and their applications, ranging from machine learning and statistics to decision-making and control, with emphasis on numerically tractable problems, such as linear or constrained least-squares optimization. (4 Units.)

Course Objectives

1. Develop a practical understanding of the applications and limitations of optimization as a solution approach to engineering analysis and design.
2. Develop an ability to use rapid prototyping software to guide optimization solutions.

Desired Course Outcome:

By the end of the term, students having taken Optimization Models in Engineering are expected to be able to:

1. Understand the basic concepts of linear algebra: vectors, matrices, rank, projections; symmetric matrices, positive semidefinite matrices, eigenvalues; singular value decomposition and principal component analysis.
2. Understand three basic optimization models: least-squares, linear programming, and quadratic programming; and develop an understanding of the more general convex optimization.
3. Be aware of the wide-ranging applications where optimization models are useful, including in machine learning applications.
4. Be able to use prototyping software such as CVX to develop optimization-based solutions in concrete applications.

Course Content

The course content will roughly follow the outline posted on the website, but is subject to change and evolve throughout the semester.

1. Eigenvalue and singular value decomposition
2. Linear equations
3. Least squares
4. Convexity
5. Gradient descent
6. Linear programming
7. Quadratic programming
8. Duality
9. Optimality conditions
10. Applications: machine learning
11. Applications: control

Some application topics might be added mid-semester. It is strongly recommended that the students use the book Optimization Models, G.C. Calafiore and L. El Ghaoui, Cambridge University Press, October 2014.

Discussion

Discussion sections will take place every week. You may attend any one of the following sections, the material here is intended to help you with the class. You are strongly encouraged to attend and ask questions. Discussions will begin on Friday January 24, 2020. Discussion frequency and policy is as follows:

• Monday 10:00am - 11:00pm, McCone 141, TA: Yeshwanth
• Monday 3:00pm - 4:00pm, Tan 180, TA: Grace
• Monday 4:00pm - 5:00pm, Etcheverry 3106, TA: Suvansh
• Friday 11:00pm - 12:00pm, Goldman School 150, TA: Theo
• Friday 2:00pm - 3:00pm, Barrows 126, TA: Laura/Vignesh
• Friday 3:00pm - 4:00pm, Barrows 20, TA: Laura/Vignesh

Discussion material will be released on Friday.

Solutions to the discussion material will be released on Monday after the last discussion of the week. Please also see our link to Google Calendar.

Homework

Homework will be assigned every Friday here on the website and will be due the following Friday at 11:00pm. Homework will be electronically submitted to Gradescope (entry code: MYJ5YZ) as follows:

1. A single PDF consisting of the following components:
1. Scanned handwritten work or a LaTeX file containing answers to non IPython questions
2. A PDF of all iPython notebooks
2. All IPython notebooks in .ipynb format

You will need to select the pages associated with each question through Gradescope. Late homework will not be accepted, but to account for the vicissitudes of life, we will drop the 2 lowest homework scores from the final grade calculation.

Discussion and collaboration, as opposed to copying, of homework is encouraged. In other words, you are encouraged to discuss the homework with your classmates but you must write your own derivations and do your own calculations, etc. Do not hesitate to ask the professor or the GSIs for clarifications and hints for the homework problems during Homework Parties and Office hours. We encourage cooperation rather than competition.

Homework Grading

The point of homework in this class is for you to learn the material. To help you in doing this each student will grade their own homework in addition to being graded by readers.

Self Grades How-To

After the HW deadline, official solutions will be posted online. You will then be expected to read them and enter your own scores and comments for every part of every problem in the homework on a simple coarse scale.

0 = didn't attempt or very very wrong,

2 = got started and made some progress, but went off in the wrong direction or with no clear direction,

5 = right direction and got half-way there,

8 = mostly right but a minor thing missing or wrong,

10 = 100% correct.

Note: You must justify self-grades of 2, 5 or 8 with a comment. Grades of 0 and 10 do not need to be justified. If you are really confused about how to grade a particular problem, you should post on Piazza. This is not supposed to be a stressful process. See this video for a walkthrough of the self-grade process.

Course readers are going to be grading and sending you occasional comments. Because we have reader grades, we will catch any attempts at trying to inflate your own scores. This will be considered cheating and is definitely not worth the risk. Your own scores will be used in computing your final grade for the course, adjusted by taking into account reader scores so that everyone is fairly graded effectively on the same scale. For example, if we notice that you tend to give yourself 5s on questions where readers looking at your homeworks tend to give you 8s, we will apply an upward correction to adjust.

You can find the link to the self grade form (where you will enter these scores) on the website next to the solutions. After completing a self grade form, it will generate a formatted .txt file with your responses. Download this file and submit it to Gradescope. There will be an assignment “HW X Self Grade” on Gradescope that you should submit this to. After uploading, make sure the autograder matches the scores you gave yourself in the self grade form. Your self-grades will be due on the following Friday after the homework deadline at 11:00 PM sharp. We will accept late self-grades up to 25 hours late (Saturday 11:59PM) for a 10% penalty on the associated homework assignment. If you don't enter a proper grade by this deadline, you are giving yourself a zero on that assignment. Merely doing the homework is not enough, you must do the homework; turn it in on time; read the solutions; do the self-grade; and turn it in on time. Unless all of these steps are done, you will get a zero for that assignment.

Exam Policies

The 127/227AT Spring 2020 semester will have one midterm and one final. The midterm time will be during class hours and date will be communicated in class. Current schedule (below) is TENTATIVE. The final will be held during the designated final exam slot released by campus. Makeup/alternate exams will not be scheduled under any circumstances.

Please plan for exams at these times and email us during the first two weeks of the semester per university policy if you know about any exam conflicts. If an emergency arises that conflicts with the exam times, email us as soon as possible at ee127227sp20@gmail.com. Emergency exam conflicts will be handled on a case-by-case basis. We do not provide specific accommodations for conflict with other classes, conflict with personal activities, even planned in advance, extracurricular or not. We try to accommodate situations of personal hardship to the best of our abilities. If you skip an exam without coordinating with us beforehand, you will receive an automatic zero.

Grading Breakdown

The course grade will be based on the following evaluation:

• Homework (30%)
• Midterm (30%)
• Final (40%)

There will also be a project option. If you choose to do the project, we will calculate the following breakdown:

• Homework (30%)
• Project (15%)
• Midterm (25%)
• Final (30%)

Your final grade will be the maximum of the two grading schemes.

This policy rewards consistency and regular work, as well as final knowledge acquisition.

Redemption policy: Exams cover all material until the exam. Grade of midterm will be replaced with a weighted average of the final and midterm grade if it is higher:

• Clobbered midterm grade = max(midterm grade, 60% final grade + 40% midterm grade)

DSP

We are happy to accommodate students with special needs as determined by the DSP office. Please email us in the first two weeks of class to request appropriate accommodations. Please include your accommodation letter in this email.

Course Materials

RECOMMENDED:

1. Calafiore, Giuseppe and El Ghaoui, Laurent. Optimization Models.
2. Boyd, Stephen, and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
3. Livebook (You can register on thelivebook platform for free)

Course Communication

The instructors and TAs will post announcements, clarifications, hints, etc. on Piazza. You must check the EECS127/227A Piazza page frequently throughout the term.

If you have a question, your best option is to post a message there. The staff will check the forum regularly, and other students will be able to help you too. When using the forum, please avoid off-topic discussions, and please do not post answers to homework questions before the homework is due. Also, always look for a convenient category to post the question to (for example, each homework will have its own category, so please post there). That will ensure you get the answer faster.

If your question is personal or not of interest to other students, you may mark your question as private on Piazza, so only the instructors will see it. If you wish to talk with one of us individually, you are also welcome to come to our office hours. Please reserve email for the questions you can't get answered in office hours, in discussion sections, or through the forum.

It can be challenging for the instructors to gauge how smoothly the class is going. We always welcome any feedback on what we could be doing better. If you would like to send any feedback, you can send an email to: ee127227sp20@gmail.com

Collaboration

We encourage you to work on homework problems in study groups of two to four people; however, you must always write up the solutions on your own. Similarly, you may use books or online resources to help solve homework problems, but you must always credit all such sources in your writeup, and you may never copy material verbatim. Using previous homework and exam solutions is strictly prohibited, and will be considered academic dishonesty. This is not how you want to start your career as an engineer.

We expect that most students can distinguish between helping other students and cheating. Explaining the meaning of a question, discussing a way of approaching a solution, or collaboratively exploring how to solve a problem within your group is an interaction that we encourage strongly. But you should write your homework solution strictly by yourself so that your hands and eyes can help you internalize the subject matter. You should acknowledge everyone whom you have worked with, or who has given you any significant ideas about the homework. This is good scholarly conduct.

Don't Be Afraid to Ask for Help

Are you struggling? Please come talk with us! The earlier we learn about your struggles, the more likely it is that we can help you. Waiting until right before an exam or the last few weeks of the semester to let us know about your problems is not an effective strategy - the later it is, the less we will be able to help you.

Even if you are convinced that you are the only person in the class who is struggling, please overcome any feelings of embarrassment or guilt, and come ask for help as soon as you need it – we can almost guarantee you're not the only person who feels this way. Don't hesitate to ask us for help – we really do care that you thrive!

Advice

The following tips are offered based on our experience.

Do the homework! The homework is explicitly designed to help you to learn the material as you go along. There is usually a strong correlation between homework scores and final grades in the class.

Keep up with lectures! Discussion sections, labs and homework all touch on portions of what we discuss in lecture. Students do much better if they stay on track with the course. That will also help you keep the pace with your homework and study group.

Take part to discussion sections! Discussion sections are not auxiliary lectures. They are an opportunity for interactive learning. The success of a discussion section depends largely on the willingness of students to participate actively in it. As with office hours, the better prepared you are for the discussion, the more you are likely to benefit from it.

Come to office hours! We love to talk to you and do a deep dive to help you understand the material better.

Form study groups! As stated above, you are encouraged to form small groups (two to four people) to work together on homework and on understanding the class material on a regular basis. In addition to being fun, this can save you a lot of time by generating ideas quickly and preventing you from getting hung up on some point or other. Of course, it is your responsibility to ensure that you contribute actively to the group; passive listening will likely not help you much. Also recall the caveat above, that you must write up your solutions on your own. We strongly advise you to spend some time on your own thinking about each problem before you meet with your study partners; this way, you will be in a position to compare ideas with your partners, and it will get you in practice for the exams. Make sure you work through all problems yourself, and that your final write-up is your own. Some groups try to split up the problems ("you do Problem 1, I'll do Problem 2, then we'll swap notes"); not only is this a punishable violation of our collaboration policies, it also ensures you will learn a lot less from this course.