CS 162 Lecture Notes for Monday, 29 January 2007 Transcribed by Chris Pennello, based on lecture notes by Alan Smith ************************* *** About these notes *** ************************* These notes cover pages 1.33 through 1.35 (lines 2,073 through 2,218) of `smith.1.txt' and pages .1 through .19 (lines 1 through 1,217) of `smith.2.txt'. Please note that this is essentially a diff between the lecture notes provided by Alan Smith and what was discussed in class. The headings are intended to make it easier to line up what's discussed in Smith's notes files and what's discussed here. *************************** *** BSD Unix Scheduling *** *************************** The reccomended text, Operating System Concepts by Silberschatz, Abraham, et al, discusses the BSD scheduling algorithm in more detail. Initially, it had 128 prioritiy levels. However, this was far too many, so the levels were grouped. In the computation formula for User priority, smaller numbers mean a higher priority. Decay filters, mentioned in the computation of PCPU, the load factor, can be viewed when running `uptime'. DEC had a system called ULTRIX, which was a version of Berkeley UNIX; it used pretty much the same scheduler as BSD. Before DEC was bought by Compaq (which was bought by HP), was very prominent computer maker, and they produced workstations that cost just under $10,000. Ken Olsen, who headed DEC, once asked ``who needs a computer on his desk?'' ********************************** *** Scheduling Countermeasures *** ********************************** How do you beat scheduling algorithms? How do you get more service out of the computer than the algorithm should give you? 1. One way is to break up your job into smaller ones so that the small jobs get higher priority and your entire task gets completed more quickly as a result. 2. Another is to do fake I/O so that your priority will be further increased. ************************** *** Scheduling Summary *** ************************** The different scheduling are arbitrary in the sense that they should all be ``correct''---the jobs that are submitted should be run. What differentiates them is performance. The purpose of HW1 is to compare these different performance factors and draw conclusions about the different algorithms based on them. Kaveh >> What's the difference between threads and jobs? Smith >> Short answer: Basically, a process comes with an address space, and threads share address space. There can be multiple threads per job. We'll spend 4 weeks later discussing these concepts. Kaveh >> But how does scheduling affect threads? Smith >> Thus far, we've covered a very idealized model of how scheduling of jobs works. We've discussed scheduling some kind of idealized entities: they can be jobs or threads. The only difference will be overheads. ********************************************* *** Independent and Cooperating Processes *** ********************************************* We'll be discussing concepts such as deadlock, semaphors, etc. There are two issues in dealing with processes. 1. Scheduling 2. Synchrnoization Synchronization affects correctness, so you have to get that right before you can schedule processes. When we discussed scheduling, we assumed every process was independent. In real systems, many things are dependent. We strive for ``cooperating'' processes, not necessarily competing ones. ***************************** *** Cooperating processes *** ***************************** Cooperating processes should yield the same correct results, independent of running time and how the processes are interleaved. These minute differences in initial conditions can cause different results, but they are fairly small. One exception, of course, is if you use random numbers in your computations. Cooperation of processes is difficult and creates complexity, but it allows for new benefits: shared resources, faster computations are among them. Example of modularity in UNIX piping as an instance of cooperating processes: `eqn' produces equations, `tbl' produces tables, and `troff' typesets, so these three processes interact in serial to produce a more complex result through piping: $ eqn | tbl | troff ************************** *** A counting contest *** ************************** Example of two competing processes sharing the same resource. There are a couple possibilities. There could actually be two different processors. There could be different chaches for each processor, so there could be cache consistency problems, depending on the algorithms dedicated to managing this. The variable could also simply just be in a register with a single CPU. The main point, however, is that this whole process is non-deterministic, which is clearly undesirable. The claim is that multiprogramming is just as difficult to manage (i.e., just as bad) as multiprocessing. There are shared resources, so managing the swapping of processes in and out while also balancing how they share resources is very difficult. ************************* *** Atomic operations *** ************************* We have the idea of atomic operations, ones that cannot be interrupted, such as memory load and store. Given some atomic operations, you can construct what you might consider to be larger atomic operations. However, we'll see that this isn't always the best thing to do. This is because the more complex `atomic' operations make synchronization much more difficult. Essentially, they don't remove any problems, they only make them harder and more subtle. ********************* *** Too much milk *** ********************* Example of the ``too much milk'' problem. Sample solution to this problem. It doesn't work because the roommates don't necessarily notice what the other is doing. I.e., processes can execute in parallel, and the fact that they share state becomes unimportant, since they ignore it in executing their algorithms in parallel. We need to synchronize the roommates: we must use atomic processes to ensure the correct cooperation. This is called mutual exclusion: the so-called ``critical section'' is a collection of operations such that only one process should be executing in it at one time, such as buying milk from the store. Only one roommate should be doing so at once; they shouldn't be doing it ``in parallel'' (whether they are truly acting in parallel, or just miss each other's notes on the refrigerator). Peter >> What if the processor is out-of-order? Smith >> We assume that statements are executed in serial as they are written. But even if you have an out-of-order processor, the hardware should ensure that you get the same result as a serial sequential execution, even if there are multiple out-of-order processors. Amy >> Do we assume there's no third roommate? Smith >> {laughs} Our conclusion after we finish this will be, ``boy, what if there IS a third roommate?'' I.e., things can get even more complicated. We do assume, however in the two-roommate scheme, that you have a reliable roommate. What if one goes on vacation? Our solution implicitly DEPENDS on both processes running properly. If one of them gets Ctrl-C'd, then we won't get the behavior we desire. The third attempt for to solve the milk problem uses two notes, but fails when both roommates leave notes indicating they are looking into buying milk simultaneously---there is deadlock. The fourth attempt uses further assymetry between the roommates, but an interrupted process within the critical section will leave the other process hanging forever. The reason these solutions were so complicated and fragile is because the synchronization mechanisms were too simple. ***************************************** *** Requirements for mutual exclusion *** ***************************************** In order to accomplish mutual exclusion, we must have only one process in a critical section at a time, and that one can't die within the critical section. Also, if two processes want to run at the same time, we can't have both wait forever. One must always be able to proceed. *************************** *** (General) Semaphore *** *************************** We need more a more complex synchronization method. One is (general) semaphores, invented by Edsger Dijkstra. Dijkstra died recently: 6 August 2002. Semaphores may be able to solve many more problems than mutual exclusion, but the application becomes more complex. Apparently, we'll doubt their ability to solve more complicated problems as it becomes more difficult to apply them. ************************************* *** Too much milk with semaphores *** ************************************* If our semaphor OKToBuyMilk was not initially set to 1 (e.g., it was set to 0), whichever process attempts to buy milk won't be able to start its task, because it'll believe that someone else is trying to buy milk. This is an example of a binary semaphor: they have values of only 0 or 1. There are other kinds as well, such as counting semaphores. If there is a critical section, semaphores can be used to prevent multiple processes from using it as the same time. You can use multiple semaphores as well, depending on the complexity that the synchronization problem demands. ******************************* *** Producers and Consumers *** ******************************* Example of the producer & consumer problem. The problem is that the producers of data and the consumers of data shouldn't interfere with the buffer pool at the same time. This example will be picked up in the next lecture.