Post on 14-Apr-2018
transcript
7/30/2019 Operating SystemsLectureNotes Session 10
1/50
OPERATING SYSTEM
FILE SYSTEM
Session 10(Chapter 9)
7/30/2019 Operating SystemsLectureNotes Session 10
2/50
Concept and Structure of a File
OperatingSystems-FileSystem
Contiguous logical
address space
Types: Data
numeric character
binary
Program
None - sequence of words,bytes
Simple record structure Lines Fixed length Variable length
Complex Structures Formatted document Re-locatable load file
Can simulate last two with first
method by inserting appropriatecontrol characters
Who decides: Operating system Program
2
7/30/2019 Operating SystemsLectureNotes Session 10
3/50
Attributes of a File
OperatingSystems-FileSystem
Name only information kept in human-readable form
Identifier unique tag (number) identifies file within file
system
Type needed for systems that support different types
Location pointer to file location on device Size current file size
Protection controls who can do reading, writing,
executing
Time, date, and user identification data forprotection, security, and usage monitoring
Information about files are kept in the directory structure,
which is maintained on the disk
3
7/30/2019 Operating SystemsLectureNotes Session 10
4/50
File Operation
OperatingSystems-FileSystem
File is an abstract data type
Create
Write
Read
Reposition within file Delete
Truncate
Open(Fi) search the directory structure on disk for
entry Fi, and move the content of entry to memory Close (Fi) move the content of entry Fi in memory to
directory structure on disk
4
7/30/2019 Operating SystemsLectureNotes Session 10
5/50
What is needed to Open Files
OperatingSystems-FileSystem
Several pieces of data are needed to manage open files: File pointer: pointer to last read/write location, per process that
has the file open
File-open count: counter of number of times a file is open to
allow removal of data from open-file table when last processescloses it
Disk location of the file: cache of data access information
Access rights: per-process access mode information
5
7/30/2019 Operating SystemsLectureNotes Session 10
6/50
Open File Locking
OperatingSystems-FileSystem
Provided by some operating systems and file systems
Mediates access to a file
Mandatory or advisory: Mandatory access is denied depending on locks held and
requested Advisory processes can find status of locks and decide what
to do
6
7/30/2019 Operating SystemsLectureNotes Session 10
7/50
File typeName and Extension
OperatingSystems-FileSystem
7
7/30/2019 Operating SystemsLectureNotes Session 10
8/50
What is File System?
OperatingSystems-FileSystem
The file system is a mechanism for storing data and programs.
Main characteristics of Secondary storage Large storage capacity
Non-volatile nature.
Consequence? Give raise to the convenience of storing data and programs on secondary
storage.
8
7/30/2019 Operating SystemsLectureNotes Session 10
9/50
What do you Require of File
Systems?
Naming Should be flexible, e.g., allow multiple names for same files
Support hierarchy for easy of use
Persistence Want to be sure data has been written to disk in case crash occurs
Sharing/Protection Want to restrict who has access to files
Want to share files with other users
Speed & Efficiency for different access patterns Sequential access
Random access
Sequential is most common & Random next
Other pattern is Keyed access (not usually provided by OS
OperatingSystems-FileSystem
9
7/30/2019 Operating SystemsLectureNotes Session 10
10/50
What do you Require of File
Systems?
Minimum Space Overhead Disk space needed to store metadata is lost for user data
Twist: all metadata that is required to do translation must be stored
on disk Translation scheme should minimize number of additional accesses for
a given access pattern
Harder than, say page tables where we assumed page tables
themselves are not subject to paging!
OperatingSystems-FileSystem
10
7/30/2019 Operating SystemsLectureNotes Session 10
11/50
File System Organization
The file system on the disk of an operating system allows data to be stored,
searched and retrieved.
One part of the file system is the user interface.
This interface defines the definition of a file, file attributes, operations
allowed on files and a directory structure for organizing files.
The other part of the file system has algorithms and data structures to map
the logical file system onto the physical storage devices.
A file system has a layered design Each layer uses the features provided by the layer below it to provide features to the
layer above it.
The lowest layer is the I/O control. It consists of device drivers and interrupt handlers for information transfer between
the memory and the disk.
A device driver is a translator between the file system and the actual disk
hardware.
The device driver takes high level instructions as input.
OperatingSystems-FileSystem
11
7/30/2019 Operating SystemsLectureNotes Session 10
12/50
File System Organization
The next layer, the basic file system issues generic commands to the
appropriate device driver to access (read / write) physical blocks on the disk. The layer above the basic file system is the file organization module.
This module has information about files, the logical records and the physical
blocks. The file organization module translates / maps logical addresses to physical block
addresses for the basic file system.
This module also keeps track of free blocks.
The logical file system makes use of the directory structure of the file system
and provides information to the file organization module given a file name.
Protection and security are also part of the logical file system.
OperatingSystems-FileSystem
12
7/30/2019 Operating SystemsLectureNotes Session 10
13/50
File System Organization
OperatingSystems-FileSystem
13
7/30/2019 Operating SystemsLectureNotes Session 10
14/50
Over-View
OperatingSystems-FileSystem
14
7/30/2019 Operating SystemsLectureNotes Session 10
15/50
OperatingSystems-FileSystem15
7/30/2019 Operating SystemsLectureNotes Session 10
16/50
Steps in Opening and Reading a
File
Look-up (via-directory) Find on disks file descriptors block number
Find entry in open file table. Create one if none, else increment ref count
Find where file data is located By reading on-disk file descriptor
Read data & return to user
OperatingSystems-FileSystem
16
7/30/2019 Operating SystemsLectureNotes Session 10
17/50
Open File Table
inoderepresents file at most 1 in-memory instance per unique file
#number of openers & other properties
filerepresents one or more processes using an file
With separate offsets forbyte-stream dirrepresents an open directory file
Generally: None of data in OFT is persistent
Reflects how processes are currently using files
Lifetime of objects determined by open/close Reference counting is used
OperatingSystems-FileSystem
17
7/30/2019 Operating SystemsLectureNotes Session 10
18/50
File Descriptors (inodes)
Term inode can refer to 3 things: 1.in-memory inode
Store information about an open file, such as how many openers, corresponds
to on-disk file descriptor
2.on-disk inode Region on disk, entry in file descriptor table, that stores persistent information
about a filewho owns it, where to find its data blocks, etc.
3.on-disk inode, when cached in buffer cache A bytewise copy of 2. in memory
Q.: Should in-memory inode store a pointer to cached on-disk
inode?
OperatingSystems-FileSystem
18
7/30/2019 Operating SystemsLectureNotes Session 10
19/50
What information does
File System contain?
Contains super blockstores information such
assize of entire file system, etc. Location of file descriptor table & free map
Free Block Map Bitmap used to find free blocks
Typically cached in memory
Superblock & free map often replicated in different
positions on disk
OperatingSystems-FileSystem
19
7/30/2019 Operating SystemsLectureNotes Session 10
20/50
Allocation of Disk Space
These mechanisms looks at how efficiently files are
stored and accessed. Contiguous
Linked
Indexed Commonly referred as file allocation strategies
OperatingSystems-FileSystem
20
7/30/2019 Operating SystemsLectureNotes Session 10
21/50
Contiguous Allocation
Idea: allocate files in contiguous blocks
File Descriptor = (first block, length)
Good sequential & random access
Problems: hard to extend filesmay require expensive compaction
external fragmentation
analogous to segmentation-based VM
OperatingSystems-FileSystem
21
7/30/2019 Operating SystemsLectureNotes Session 10
22/50
Linked Files
Idea: implement linked list either with variable sized blocks
or fixed sized blocks (clusters)
Solves fragmentation problem, but now need lots of seeks for sequential accesses and random
accesses
unreliable: lose first block, may lose file
Solution: keep linked list in memory DOS: FAT File Allocation Table
OperatingSystems-FileSystem
22
7/30/2019 Operating SystemsLectureNotes Session 10
23/50
OperatingSystems-FileSystem23
7/30/2019 Operating SystemsLectureNotes Session 10
24/50
Indexed Allocation
Single-index: specify maximum file-size, create index
array, then note blocks in indexRandom access okone translation step
Sequential access requires more seeksdepending oncontiguous allocation
Drawback: hard to grow beyond maximum
OperatingSystems-FileSystem
24
7/30/2019 Operating SystemsLectureNotes Session 10
25/50
Directory Structure
Directory is collection of nodes containing
information about files.
OperatingSystems-FileSystem
Both the directory structureand the files reside on disk
Backups of these twostructures are kept on tapes
25
7/30/2019 Operating SystemsLectureNotes Session 10
26/50
Disk Structure
Disk can be subdivided into partitions
Disks or partitions can be RAID protected against failure
Disk or partition can be used raw without a file system, or
formatted with a file system
Partitions also known as minidisks, slices Entity containing file system known as a volume
Each volume containing file system also tracks that file
systems info in device directory or volume table of
contents As well as general-purpose file systems there are many
special-purpose file systems, frequently all within the same
operating system or computer.
OperatingSystems-FileSystem
26
7/30/2019 Operating SystemsLectureNotes Session 10
27/50
Disk Schematics
OperatingSystems-FileSystem
27
7/30/2019 Operating SystemsLectureNotes Session 10
28/50
Tracks Sectors and Cylinders
OperatingSystems-FileSystem
28
7/30/2019 Operating SystemsLectureNotes Session 10
29/50
From O.S Perspective
Disks are big & slow -compared to RAM
Access to disk requires Seek (move arm to track)to cross all tracks anywhere from 20-50ms, on
average takes 1/3.
Rotational delay (wait for sector to appear under track) 7,200rpm is 8.3ms
per rotation, on average takes : 4.15ms rot delay Transfer time (fast: 512 bytes at 998 Mbit/s is about 3.91us)
Seek+Rot Delay dominates
Random Access is expensive And unlikely to get better
Consequence: Avoid seeks
Seek to short distances
Amortize seeks by doing bulk transfers
OperatingSystems-FileSystem
29
7/30/2019 Operating SystemsLectureNotes Session 10
30/50
Disk Scheduling
Can use priority scheme
Can reduce avg access time by sending requests to disk controller
in certain order Or, more commonly, have disk itself reorder requests
SSTF: shortest seek time first Like SJF in CPU scheduling, guarantees minimum avg seek time, but
can lead to starvation
SCAN: elevator algorithm Process requests with increasing track numbers until highest reached,
then decreasing etc.repeat
Variations: LOOKdont go all the way to the top without passengers
C-SCAN: -only take passengers when going up
OperatingSystems-FileSystem
30
T i l Fil S t
7/30/2019 Operating SystemsLectureNotes Session 10
31/50
Typical File System
Organization
OperatingSystems-FileSystem
31
7/30/2019 Operating SystemsLectureNotes Session 10
32/50
Operations Performed on a Directory
OperatingSystems-FileSystem
Search for a file
Create a file
Delete a file
List a directory
Rename a file Traverse the file system
32
7/30/2019 Operating SystemsLectureNotes Session 10
33/50
Organize the Directory (Logically) to Obtain
OperatingSystems-FileSystem
Efficiency locating a file quickly
Naming convenient to users Two users can have same name for different files
The same file can have several different names
Grouping logical grouping of files by properties, (e.g.,all Java programs, all games, )
33
7/30/2019 Operating SystemsLectureNotes Session 10
34/50
Single Directory
A single directory for all users
Naming and grouping problem
OperatingSystems-FileSystem
34
7/30/2019 Operating SystemsLectureNotes Session 10
35/50
Single Directory
Separate directory for each user. Path name
Can have the same file name for different user Efficient searching
No grouping capability
OperatingSystems-FileSystem
35
7/30/2019 Operating SystemsLectureNotes Session 10
36/50
Tree Structured Directories
OperatingSystems-FileSystem
Efficient searching
Grouping Capability
Current directory (working
directory)
cd /spell/mail/prog
type list
36
7/30/2019 Operating SystemsLectureNotes Session 10
37/50
Tree Structured Directories
OperatingSystems-FileSystem
Absolute orrelative path name
Creating a new file is done in current directory
Delete a file
rm
Creating a new subdirectory is done in current directorymkdir
Example: if in current directory /mail
mkdir count
Deleting maildeleting the entire subtree rooted by mail
37
7/30/2019 Operating SystemsLectureNotes Session 10
38/50
Acyclic Graph Directories
OperatingSystems-FileSystem
They have sharedSub-directories and files
38
7/30/2019 Operating SystemsLectureNotes Session 10
39/50
Acyclic Graph Directories
Two different names (aliasing)
Ifdict deletes list dangling pointer
Solutions: Back-pointers, so we can delete all pointers
Variable size records a problem Back-pointers using a daisy chain organization
Entry-hold-count solution
New directory entry type Link another name (pointer) to an existing file
Resolve the link
follow pointer to locate the file
OperatingSystems-FileSystem
39
7/30/2019 Operating SystemsLectureNotes Session 10
40/50
General Graph Directory
How do we guarantee no
cycles?Allow only links to file not
subdirectories
Garbage collection
Every time a new link is
added use a cycle detection
algorithm to determine
whether it is OK.
OperatingSystems-FileSystem
40
7/30/2019 Operating SystemsLectureNotes Session 10
41/50
Directory Implementation
Linear List
list of file names with pointers to the data blocks is one way to implement a
directory.
linear search is necessary to find for a particular file
The method is simple but the search is time consuming.
To create a file a linear search is made to look for the existence of a file with
the same file name and if no such file is found the new file created is added to
the directory at the end.
To delete a file a linear search for the file name is made and if found allocated
space is released.
Every time making a linear search consumes time and increases access time
that is not desirable since a directory information is frequently used.
A sorted list allows for a binary search that is time efficient compared to the
linear search.
But maintaining a sorted list is an overhead especially because of file creations and
deletions. OperatingSystems-FileSystem
41
7/30/2019 Operating SystemsLectureNotes Session 10
42/50
Directory Implementation
Hash Table
A linear list is used to store directory entries.
A hash table takes a value computed from the file name and returns a
pointer to the file name in the linear list.
Thus search time is greatly reduced.
Insertions are prone to collisions that are resolved.
The main problem is the hash function that is dependent on the hash
table size.
A solution to the problem is to allow for chained overflow with each hash
entry being a linked list.
Directory lookups in a hash table are faster than in a linear list.
OperatingSystems-FileSystem
42
7/30/2019 Operating SystemsLectureNotes Session 10
43/50
How is File System Mounted?
A file system must be
mounted before it can be
accessed
A un-mounted file system ismounted at a mount point
OperatingSystems-FileSystem
(a) Existing. (b) Unmounted Partition
Mount point
43
7/30/2019 Operating SystemsLectureNotes Session 10
44/50
File Sharing
Sharing of files on multi-user systems is desirable
Sharing may be done through a protection scheme
On distributed systems, files may be shared across a
network
Network File System (NFS) is a common distributed file-sharing method.
Multiple Users
User IDsidentify users, allowing permissions and protections to be
per-user
Group IDsallow users to be in groups, permitting group access
rights
OperatingSystems-FileSystem
44
File Sharing Remote File
7/30/2019 Operating SystemsLectureNotes Session 10
45/50
Uses networking to allow file system access between systems Manually via programs like FTP Automatically, seamlessly using distributed file systems Semi automatically via theworld wide web
Client-servermodel allows clients to mount remote file systemsfrom servers Server can serve multiple clients Client and user-on-client identification is insecure or complicated NFS is standard UNIX client-server file sharing protocol CIFSis standard Windows protocol Standard operating system file calls are translated into remote calls
Distributed Information Systems (distributed naming services)
such as LDAP, DNS, NIS, Active Directory implement unified accessto information needed for remote computing
File Sharing Remote File
Systems
OperatingSystems-FileSystem
45
7/30/2019 Operating SystemsLectureNotes Session 10
46/50
File Sharing Failure Modes
Remote file systems add new failure modes, due to
network failure, server failure
Recovery from failure can involve state information about
status of each remote request
Stateless protocols such as NFS include all information ineach request, allowing easy recovery but less security
OperatingSystems-FileSystem
46
File Sharing Consistency
7/30/2019 Operating SystemsLectureNotes Session 10
47/50
File Sharing Consistency
Semantics
Consistency semanticsspecify how multiple users areto access a shared file simultaneously Similar to process synchronization algorithms
Tend to be less complex due to disk I/O and network latency (forremote file systems
Andrew File System (AFS) implemented complex remote filesharing semantics Unix file system (UFS) implements:
Writes to an open file visible immediately to other users of the sameopen file
Sharing file pointer to allow multiple users to read and writeconcurrently
AFS has session semantics Writes only visible to sessions starting after the file is closed
OperatingSystems-FileSystem
47
7/30/2019 Operating SystemsLectureNotes Session 10
48/50
Protection
File owner/creator should be able to control:
what can be done
by whom
Types of access
Read
Write
Execute
Append
Delete
List
OperatingSystems-FileSystem
48
7/30/2019 Operating SystemsLectureNotes Session 10
49/50
Access Lists and Groups
Mode of access: read, write, execute Three classes of users
Ask manager to create a group (unique name), say G, and addsome users to the group.
For a particular file (say game) or subdirectory, define anappropriate access.
Attach a group to a file
OperatingSystems-FileSystem
49
A Sample UNIX Directory
7/30/2019 Operating SystemsLectureNotes Session 10
50/50
A Sample UNIX Directory
Listing50