Woah, my account still works, so I feel like putting up a little spiel on project grading. In my opinion, CS61C project grading seems to work for maybe 70-80% of people. People who do things completely correctly pass the autograder tests, and people who don't know what they're doing get low scores. However, in some cases, people who know mostly what they're doing can get really bad scores despite having a decent understanding of the material.
In an ideal world, students would know enough to be able to test their projects fully and hopefully they would always turn in project code that conforms to the autograder standards and at least gets most of everything correct. However, this is not an ideal world, and not every student knows exactly how to test their project. To complicate matters, sometimes students think they know how to test their code and write up simple tests which seem to work, which gives a false sense of security, but then the more complicated ones that the autograder uses may fail for reasons unknown to the student. At this point, the student must feel incredibly frustrated, because they thought they were doing everything correctly when in reality this was not the case. As a reader, this situation upsets me the most, because if students had access to at least some staff-provided tests prior to submission, this situation would not occur, and I feel that while an imperfect understanding of the material should definitely be penalized, the extent to which it is penalized currently is too severe.
As an example, in CS188 (and some other upper division CS classes, I am told), when you submit a project, you get the autograder feedback within 15 minutes via email, and you know exactly what you're doing correctly and what you're doing incorrectly. I find that this is a very helpful system for the student (myself), because if I'm not sure whether I've interpreted the question correctly, I can just submit my work and see whether my answers match up with what is expected. Now, I realize that this requires a lot of work for the teaching staff, who are usually more overworked in lower division classes than upper divisions (or so it seems to me). However, this process of grading is much more transparent and fair, in my humble opinion.
The thing that disappoints me most about autograder based testing is that it leaves very little room for partial credit and appreciation of the hard work that students put into their projects, and the amount that they have learned. The Logisim project is an incredibly nontrivial project for a lower division student, and a lot of work goes into it. Even slight errors in wiring can render the whole CPU useless, and finding such errors is a very difficult problem, even for experienced staff. The cache project is even more cryptic, and I'm not sure that the "oracle" actually helps in all cases.
In recognition of the above, my proposal is that there should be a pre-testing phase for the project grading, where students can receive partial credit before the final grading is done by the reader. Basically, the project spec would have a partial test case attached to it, which at least will test some baseline measure of performance (eg. running very simple programs just to make sure program execution and control flow work) and maybe 25-50% of the points for the project will come from this pretest. The reader will still check this pretest when grading, but by making it available to the students, we would be able to avoid many very silly mistakes that arise from unfortunate misunderstandings of the project specification. This way, the grading process becomes more transparent and the specification is made clearer by way of example.
I don't know if anyone will ever read this, but I felt the need to put it up here to ease my own mind. If I am ever a TA for this class, which I hope to be in the future, I will bring up this idea and try to push for it as much as possible, because it is beneficial for both the students and the reader, who will not have to deal with broken submissions that misunderstand the spec. It's an agonizing feeling, to have to assign low scores just because of small misunderstandings. Just because this is Berkeley does not mean that we have to crush student's spirits with every project. The exams do that well enough :-/. The goal of a lower division class, in my humble opinion, should be to educate students and provide them with a solid foundation to develop on, not to scare them away and weed them out.