Christopher Condap (-cw) Lecture Notes on 4/04/2005 +==========================================+ | Directories and Other File System Topics | +==========================================+ Naming ------ *Any system has file descriptors, a data structure or record to describe a file. *File descriptor information has to be stored on disk so it will stay around even when the operating system doesn't. (Assuming the disk contents are permanent.) *In Unix, the file description is stored in a fixed-size array on disk. File descriptors also contain protection and accounting information. *A special area of disk is used for this, so that it won't be overwritten. Inode Fields ------------ *Reference count (# time open) - how many processes open. *Number of links to file *Owner's user ID, owner's group ID *Number of bytes in file *Time file was last accessed, time last modified, last time Inode changed *Disk block address, indirect blocks *Flags: Inode is locked, modified, or waiting on another lock. *File mode: (type of file: char, special, directory, block special, regular, symbolic link, socket) *Protection information: (set user ID on execution , set group ID on execution, read, write, execute permission, sticky bit) *Count of shared locks in Inode *Count of exclusive locks on Inode *Unique identifier *File system associated with this Inode. *Quota structure controlling this file Process Open File Table and Inode Table --------------------------------------- Studies in file migration lead to the problem of identifying files. Inodes can be recycled. So, unique identifiers were put into the Inode, giving a unique representation for duplicate files. *When a file is open, its description is kept in main memory. When the file is closed, the descriptor is stored back to disk. *There is usually a per process table of open files. **In Unix, there is a PROCESS OPEN FILE TABLE, with one entry for each file open. The integer entry into that table is the handle for that file open. The integer entry into that table is the handle for that file open. Multiple opens for the file will get multiple entries (note that if a process forks, a given entry can be shared by several processes.) **Standard in is #0; standard out is #1; stderr is #2, must be per process. **Unix also has a System Open File Table, which points to the Inode for the file (in the Inode table). This table is system wide. Maps names to files. **There is also the INODE TABLE, which is a system-wide table holding active Descriptor is kept in OS space which is paged. So may be necessary to have page fault to get to descriptor info. File Names ---------- Users need a way of referencing files that they leave around on disk. One approach is just to have users remember descriptor indexes. I.e. the user would have to remember something like the number of the descriptor, or some such. Unfortunately, not very user friendly. Of course, users want to use text names to refer to files. Special disk structures called directories are used to tell what descriptor indices correspond to what names. Approach #1: Have a single directory for the whole disk. Use a special area of disk to hold the directory. *Directory contains pairs. *Problems **If one user uses a name, no one else can. **If you can't remember the name of a file, you may have to look through a very long list. **Security problem - people can see you file names (which can be dangerous) *Old personal computers (pre-Windows) work this way. Approach #2: Have a separate directory for each user (TOPS-10 approach). This is still clumsy: names form a user's different projects get confused. Still can't remember names of files. *IBM's VM is similar to this. Files have 3 part names: , where location A,B,A, etc. (i.e. which disk) Very painful. (Also file names limited to 8 characters). Approach #3: Unix Approach: Generalize the directory structure to a tree. *Directories are stored on disk just like regular files (i.e. file descripto with 13 pointers, etc) **User programs can manipulate directories almost like any other file. Only special system programs may write directories. *Each directory contains pairs. The file pointed to by the index may be in another directory. Hence, get hierarchical tree structure. Names have slashes separating the levels of the tree. *There is one special directory, called the root. This directory has no name, and it the file pointed to by descriptor 2 (descriptors 0 and 1 have other special purposes). **Note that we need ROOT. Otherwise, we would have no way to reach any files. From root, we can get anywhere in the file system, *Full name is the path name=, i.e. full name from root. **A directory consists of some number of blocks of DIRBLKSIZ bytes, where DIRBLKSIZ (Directory BLock Size) is chosen such that it can be transferred to disk in a single atomic operation (e.g. 512 bytes on most machines) **Each directory block contains some number directory entry structures, which are of variable length. Each directory entry has info at the front of it, containing its Inode number, the length of the entry, and the length of the name, contained in the entry. These are followed by the name padded to a 4-byte boundary with null bytes. All names are guaranteed null terminated. *Note that in a Unix, a file name is not the name of a file. It is only a name by which the kernel can search for the file. The Inode is really the "name" of the file. **Each pointer from a directory to a file is called a hard link. **In some systems, there is a distinction between a "branch" and a "link" where the link is a secondary access path, and the branch is the primary one (goes with ownership). **You "erase" a file by removing a link to it. In reality, a count is kept of the number of links to a file. It is only really erased when the last link is removed. **To really erase a file, we put the blocks of the file on the free list. Symbolic Links -------------- *There are two ways to "link" to another directory or file. One is a direct pointer. In Unix, such links are limited to not cross "file systems" - i.e. not to another disk. *We can use symbolic links, by which instead of pointing to the file of directory, we have a symbolic name for that file or directory. *We need to be careful not to create cycles in the directory system - otherwise recursive operations on the file system will loop. (E.g. cp -r). In Unix, this is solved by not permitting hard links to existing directories (except by the superuser). Pros and Cons of Tree Structured Directory Scheme ------------------------------------------------- *Can organize files in logical manner. Easy to find the file you're looking for, even if you don't exactly remember its name. *"Name" of the file is in fact a concatenation of the path from the root. Thus name isn't actually quite long - provides semantic info. *Can have duplicate names, if path to the file is different. *Can (assuming protection scheme permits) give away access to a subdirectory and the files under it, without giving access to other files. (Note: Unix does not permit multiple hard links to a directory, unless done by superuser). *Access to a file requires only reading the relevant directories, not the entire list of files. (My list of files prints out to a 1/2" printout - 20,000 files) *Structure: +-------+ | Inode |->-+ +-------+ +-------+ | +-->-->-->| Inode | V | +-------+ | ^ / | | \ +->-> +----------------+ | / | | \ | DIRECTORY | | / | | \ +----------------+ ^ _/ V_ V_ \_ | | | | |_| |_| |_| |_| +----------------+ | | | | ^ \__File Blocks__/ +----------------+ | | Name | Inode # |-->--+ +----------------+ | | | +----------------+ Working Directories ------------------- *It is cumbersome constantly to have to specify the full path name for all files. *In Unix, there is one directory per process, called the working directory, which the system remembers. *This is not the same as the home directory which is where you start at log-in time, and which is in effect the root of your personal file system. Every user has a search path which is a list of directories in which to look for a resolve a file name. The first element is almost always the working directory. *"/" is an escape to allow full path names. I.e. most names are relative file names. Ones with "/" are full (complete) path names. *Note that in Unix, the search path is maintained by the shell. If any other program wants to do the same, it has to rebuild the facilities from scratch. Should be in the OS. ("set path" is .cshrc or .login) **My path is: (. ~/bin /usr/new /usr/ucb /bin /usr/bin /usr/local /usr/hosts ~/com) **Basically, want to look in working directory, then system library directories. **We probably don't want a search strategy that actually searches more widely. If it did, it might find a file that wasn't really the target. *This is yet another example of locality. *Simple means to change working directory - "cd". Can also refer to directories of other users by prefacing their logins by "~". Operations On Files ------------------- *Open - put a file descriptor into your table of open files. Those are the files that you can use. May require that locks be set, and a user count be incremented. (If any locking is involved, may have to check for deadlock) *Close - Inverse of open. *Create a file - sometimes done automatically by open *Remove (rm) or erase - drop the link to the file. Put the blocks back on the free list if this is the last link. *Read - read a record from a file. (This usually means that there is a "access method" - i.e. I/O code - which deals with the user in terms of records, and the device in terms of physical blocks. *Write - like read, but may also require disk space allocation. *Rename ("mv" or "move") - rename the file. Unix combines two different operations here. Rename would strictly involve changing the file name within the same directory, "move" moves the file from one directory to another. Unix does both with one command. *Note that mv also destroys the old file if there is one with new name. **(Which could be considered a bug, not a feature) *Seek - move to a file location in the file *Synch - write blocks of file from disk cache back to disk *Change properties (e.g. protection info, owner) *Link - add a link to a file *Lock & Unlock - lock/unlock the file *Partial Erase (Truncate) Note that Commands such as "copy," "cat," etc are built out of simpler commands listed above. Pseudo Files ------------ *We have commands such as "read," and "write" for files. We want to do similar things to devices (e.g. terminal, printer, etc). There is no reason not to treat I/O devices as files, and we can do so. Called "pseudo files." +-----------------------------------------------------+ | Note: These notes correspond to the Smith notes #7, | | starting on page #14, labeled: | | "Directories and Other File System Topics" | +-----------------------------------------------------+