CS 471 -- Lecture 13

File System Implementations

Traditionally

Main memory small and volatile lose power, lose memory.
Secondary memory large (disk & tape) and stable. If shutdown right, no problem with loss of power.

Devices:

Tapes(usually back-up only)

header, and data divided into (usually) fixed size blocks
header includes info about file (e.g., when created, by whom) and protection.
access strictly sequential

Disks

usually several recording surfaces on one drive, moving heads.
lots of different organizations (unlike tape); discussed below
access can be "almost" random

Flash Memory

Developed by Toshiba 1980-84.
NOR
NAND

File system:

         Application Programs
                |
                V
         Logical file systems  <--- As provided by OS; some OSs provide
                |                   several types, some few.
                V
         File organization modules <- again, variety depends on OS
                |                     e.g.  provides "virtual
                V                     files" -- may be either in
         Basic file system            memory or in disk -- up to OS
                |
                V
         I/O control
                |
                V
         physical devices

System usually maintains an open-file table: list of currently open files, with permissions and locations.

OS usually provides mount commands: ties file names to physical device.

Disks usually several recording surfaces on drive. Disk space divided into

sectors (several/track; ~1k to 4k in size)

tracks (several/cylinder with around 30 sectors/trk)

cylinders (several/drive; ~200 to 36000)

I/O transfers done in multiples of sectors; addressing by cylinder number, surface (track) number, & sector number.

Files divided into blocks, with blocks usually same size as sector.

Disk Allocation Methods

Contiguous -- blocks allocated sequentially. Blocks from same file are contiguous.

Advantages: if you know block where file begins, you can determine location (sector) of any byte in the file. So almost random access.
Disadvantage: files go away or change size, fragmentation results.

Can talk about space allocation using just contiguous memory allocation: first fit, best fit, worst fit. And same fragmentation problems.

Linked -- Last several bytes of each block contains location of next block (usually).

Advantage: file can grow easily
Disadvantage: random access not possible. reliability reduced (why?)

Indexed -- File starts with index block which contains address of each block in file.

Advantages: almost random access (read index into memory, use it to compute address of each byte: if byte is in nth block (how determined?) the index has address of that block.
What about really BIG files?
- Multilevel -- first index points to blocks with more indices. problem with this is that it multiplies the number of disk accesses (think: really slow) required to get to real data (file contents).
- Combined -- (typical of UNIX systems) first n entries of index point to data, next few point to second level indices which point to data, next few point to second level indices which point to third level indices which point to data. Since most files (at least under UNIX) are small, most accesses will require reading in the index (one disk access) and then only one additional disk access for any byte in the file. Only for really large files will multiple disk accesses be needed.
Disadvantage: (potential) number of accesses required to get to particular block of file.

Directories

(and free space management)

Disk contains special files (often at sector 00001) which contain table of files on the disk. May also contain a "bad sector table" (since the technology often produces a record surface with tiny blemishes -- hence spots that can't be used). Disk is usable unless this first sector is flawed since the file table and back block table must be readable. (On many systems, the "format" command checks each sector -- write something then see if it can be read), and build these tables.)

Free space list

bit vector -- one byte per sector.
linked list -- pointer to first sector which as pointer to second sector, etc. or can group so block has addresses of n free blocks.
grouping -- first block has address of n free blocks, where first n-1 are actually free, but last is the next index.
counting -- since blocks are usually allocated and unallocated in groups (as files come and go), keep list of contiguous free blocks where each list entry says where and how many.

Directories

List of files. Some systems have single directory (like MS, CMS)
- Some have one directory per user.
- File naming restriction/conventions.
  
  Some systems restrict length of file names (old DOS 11 chars, UNIX 256 chars).
  In some systems, each file must have a type (which gives information about what kind of data it contains, e.g., object code, FORTRAN source, compress data, pictures). Sometimes enforced by OS, sometimes only a convention.
- Windows and UNIX have "tree structured" directories. Though not really trees since the same file can be in several directories. Really are acyclic graphs
  In UNIX, users control directories, which are "normal" files but can only be altered through the use of the appropriate tools. UNIX also maintains one i-node table per device. The directory contains the name of the file and its i-number (index to i-node table). The i-node table contains dates (create, reference, use), user id, group id, size, number of links, status, locks, and physical location of file. You can look in /usr/kvm/sys/ufs/inode.h to see actual structure.
Protection:
OS usually responsible for:
1. protecting file from being corrupted,
2. limiting use of files (according to users or according to use)
Mechanisms:
1. encryption (see crypt command in UNIX).
2. OS maintains access/use information on each file and checks it before allowing a process access.
3. Groups: In UNIX, file protection is poor (and crude). Protection for each file can be separately specified for the owner, for groups, and for general access. Each user can be a member of several groups (type "groups" to see your groups). The owner of a file (or root) can give read, write, or execute permissions separately at the three levels of owner, group, and others. For example, if you have a really important file which does not change often, you might consider removing your write access to it to prevent accidental modification or removal. If you need to change it later, you can restore your write access (since you own the file), make the change, then remove write access again.

UNIX: consider the following partial output of a ls -li command:

 41345 drwxr-xr-x 20 price    daemon       1024 Mar 26 13:57 .
     2 drwxr-xr-x  7 root     wheel         512 Jul  7  2016 ..
 41396 -rw-------  1 price    staff         264 Jul  7  2015 .Xauthority
 41397 -rw-r--r--  1 price    staff         262 Sep 25  2016 .Xdefaults
 41397 -rw-r--r--  1 price    staff         905 Feb 23 18:53 .cshrc
  2435 -rw-r--r--  1 price    staff          63 Mar 20  2017 .emacs
 41349 -rw-------  1 price    staff         596 Mar  8 15:06 .history
 41372 -rw-r--r--  1 price    staff         587 Jul  8  2017 .login
  3648 -rw-r--r--  1 price    staff         588 Feb 15 18:11 .xinitrc
 41348 -rw-r--r--  1 price    staff         517 Feb 17 19:04 .xplaces
 40198 drwxr-xr-x  2 price    staff         512 Mar 12 16:53 bin
 26778 drwxr-xr-x 14 price    staff         512 Jun 30  2016 classes
  9758 drwxr-xr-x  2 price    staff         512 Feb 18 13:01 correspondence
 24395 drwxrwxr-x  2 price    staff         512 Aug  1  2017 mailfolders
  6178 drwxr-xr-x  4 price    staff         512 Jan 21 18:24 misc
 35352 drwxr-xr-x  3 price    staff         512 Feb 18 13:00 misc.sailing
 12160 drwxr-sr-x  6 price    staff         512 Jul 28  2017 papers
  6165 drwxr-xr-x  2 price    staff         512 Jan 15  2017 presentations
 20726 drwxr-xr-x  6 price    staff         512 May 30  2017 proposals
 25592 drwxr-xr-x  3 price    staff         512 Mar 26 13:57 research.projects
 31695 drwxr-xr-x  7 price    staff         512 Sep 29  2017 src
   ^     ^  ^  ^   ^  ^         ^            ^        ^      ^
   |     |  |  |   |  |         |            |        |      |
   |     |  |  |   |  |         |            |        |       - name of file
   |     |  |  |   |  |         |            |         - last chg date (UNIX 
   |     |  |  |   |  |         |            |           also keeps create and
   |     |  |  |   |  |         |            |           last reference dates)
   |     |  |  |   |  |         |             -- size (in bytes)
   |     |  |  |   |  |          -- group owner
   |     |  |  |   |   -- owner
   |     |  |  |    -- number of directories in which this file appears (so OS
   |     |  |  |       knows when it can delete the file.  You can delete from
   |     |  |  |       your directory, but others may still have it in theirs.
   |     |  |   -- Others permissions.
   |     |   -- Group permissions.
   |      -- Owner permissions.
    -- i-number: index into i-node table.  If two directories share a file, then
       the two files have the same i-number.

Permissions changed with chmod, Group owner changed with chgrp. Groups on system contained in /etc/group file. Files are put in multiple directories with ln (for link) command.

OS responsible for enforcing "consistency semantics". If several users are using the same file at the same time, what happens? If all users are on the same machine, this is nontrivial, but easier than when users are on different machines accessing the file over a network. This is tricky because pieces of the file are sitting in buffers (with generally more and bigger buffers or caches on networks) to improve performance. More on this when we get to distributed systems.

Disk Scheduling

On a PC, disk scheduling is not a consideration. However, on a large multiuser system, since I/O tends to be very slow (relative to CPU speeds), and since a single drive may have a list of outstanding I/O requests, the order in which the requests are serviced can significantly effect performance. CPU speeds are significantly higher the disk drive speeds so that sufficient time exists or reorder requests to improve performance. What we are doing is potentially changing the movement of heads (since sectors cannot be reordered in response to read requests). Different strategies effect the amount of head motion.

The simplist approach: FCFS, i.e., don't reorder. Appropriate for single user systems.

SSTF (shortest seek time first) -- move the heads to the closest track with an outstanding I/O request. As new requests arrive, read order is changed accordingly. Disadvantage: can lead to starvation.

SCAN -- to avoid starvation of SSTF, once heads start moving in one direction keep going in that direction.

C-SCAN -- to provide more uniform response times, always move heads in one direction, then reset.

C-LOOK -- strict SCAN means that once head starts in a direction, continues until it gets to end (the innermost or the outermost track). LOOK modifies this by stopping motion on one direction if no more requests remain in that direction.

FIFO (FCFS) is appropriate if individual drives (many systems will have several drives) generally have few outstanding requests, as on a PC. C-SCAN and C-LOOK appropriate for heavily loaded drives.

Swap-Space Management

Disks are also used for memory management (for paging, swapping, or paged- swapping). Since this so effects overall system performance, it is given special consideration. Sometimes, swap space has its own preallocated disk partition, so no directories to traverse, maintain. Produces improved speed, but space is permanently allocated, and if sometimes too small, the OS will run out of memory. Changing its size will require halting the system and rebuilding the file system from backups, then rebooting.

Performance

Disk speeds (as measured by the rate at which data can be written or read) can be improved by use of disk striping: each block is divided into several subblocks which can be transferred in parallel to separate drives. This technique means that, for example, a system designer might choose to use several slow speed (and hence cheaper) drives instead of one high speed drive -- and still provide high speed disk transfers.

This technique can also be used to improve reliability of disks (a common weak point in system reliability) by adding parity bits and spreading each byte across several drives in parallel (so that no speed penalty results). If a single drive fails, then the parity can used, along with the good data from the other drives, to compute the failed bit.

Index Previous Next