File System

WashingtonWASHINGTON UNIVERSITY IN ST LOUIS

File Systems

Fred Kuhns([email protected], http://www.arl.wustl.edu/~fredk)

Department of Computer Science and EngineeringWashington University in St. Louis

Fred Kuhns (04/08/23) Cs422 – Operating Systems Organization 2

Why a file system

• There is a general need for long-term and shared data storage:– need to store large amount of

information– persistent storage (outlives process and

system reboots)– concurrent sharing of information

• Files meet these requirements• The file manager or file system within

the OS


File Concept

• Abstraction presented to the user• Named collection of related information

on secondary storage.• File name may encode the file type

– file extensions in UNIX and Windows

• Common examples of File types– Regular files, directories– Executable files– special files (block and character)– Archives


File Structure

• None - sequence of words, bytes• Simple record structure

– Lines, Fixed length, Variable length• Complex Structures

– Formatted document, multi-media documents

• Who decides:– Operating system– Application– “Middleware”– DBMS


File Attributes• Name – only information kept in human-

readable form.• Type – needed for systems that support

different types.• Location – pointer to file location on device.• Size – current file size.• Protection – controls who can do reading,

writing, executing.• Time, date, and user identification – data

for protection, security, and usage monitoring.• Information about files are kept in the

directory structure, which is maintained on the disk.


File Operations• create• write• read• reposition within file – file seek• delete• truncate• open(Fi) – search the directory structure

on disk for entry Fi, and move the content of entry to memory.

• close (Fi) – move the content of entry Fi in memory to directory structure on disk.


File Types – name, extension

Executable exe, com, bin ornone

ready-to-run machine-language program

Object obj, o complied, machinelanguage, not linked

Source code c, p, pas, 177,asm, a

source code in variouslanguages

Batch bat, sh commands to thecommand interpreter

Text txt, doc textual data documents

Word processor wp, tex, rrf, etc. various word-processorformats

Library lib, a libraries of routines

Print or view ps, dvi, gif ASCII or binary file

Archive arc, zip, tar related files groupedinto one file, sometimescompressed.

File Type Usual extension Function


Access Methods• Sequential Access -

– read next – write next – reset– no read after last write– (rewrite)

• Direct Access: n = relative block number– read n– write n– position to n– read next– write next – rewrite n


Directory Structure• A collection of nodes containing information

about all files.

F 1 F 2F 3

F 4

F n

Directory

Files

Both the directory structure and the files reside on disk.Backups of these two structures are kept on tapes.


Information in a Device Directory

• Name • Type• Address • Current length• Maximum length• Date last accessed (for archival)• Date last updated (for dump)• Owner ID (who pays)• Protection information (discuss later)


Operations Performed on Directory

• Search for a file• Create a file• Delete a file• List a directory• Rename a file• Traverse the file system


Organize the Directory (Logically)

• Efficiency – locating a file quickly.• Naming – convenient to users.

– Two users can have same name for different files.

– The same file can have several different names.

• Grouping – logical grouping of files by properties, (e.g., all Pascal programs, all games, …)


Single-Level Directory

• A single directory for all users.

• Naming problem• Grouping problem


Two-Level Directory

• Separate directory for each user.

•Path name•Can have the same file name for different

user•Efficient searching•No grouping capability


Tree-Structured Directories


Tree-Structured Directories

• Efficient searching• Grouping Capability• Current directory (working directory)

– cd /spell/mail/prog– type list


Tree-Structured Directories• Absolute or relative path name• Creating a new file is done in current directory.• Delete a file

rm <file-name>• Creating a new subdirectory is done in current

directory.mkdir <dir-name>

Example: if in current directory /spell/mailmkdir count

mail

prog copy prt exp count

Deleting mail deleting the entire subtree rooted by ‘mail’


Acyclic-Graph Directories

• Have shared subdirectories and files.


Acyclic-Graph Directories

• Two different names (aliasing)• If dict deletes all dangling pointer.

Solutions:– Backpointers, so we can delete all

pointers.Variable size records a problem.

– Backpointers using a daisy chain organization.

– Entry-hold-count solution.


General Graph Directory


General Graph Directory (Cont.)

• How do we guarantee no cycles?– Allow only links to file not

subdirectories.– Garbage collection.– Every time a new link is added use a

cycle detection algorithm to determine whether it is OK.


Protection

• File owner/creator should be able to control:– what can be done– by whom

• Types of access– Read– Write– Execute– Append– Delete– List


Access Lists and Groups• Mode of access: read, write, execute• Three classes of users

RWXa) owner access 7 1 1 1

RWXb) groups access 6 1 1 0

RWXc) public access 1 0 0 1

• Ask manager to create a group (unique name), say G, and add some users to the group.

• For particular file or subdirectory, define an appropriate access. owner group public

chmod 761 game

Attach a group to a filechgrp G game


File-System Structure

• Disk divided into one or more partitions– independent FS on each partition– Sector 0 contains the Master Boot Record

(MBR)– MBR contains partition table– one partition marked as active– boot block – first block of active partition – BIOS reads and executes MBR, which reads

boot block and executes it.– program in boot block loads OS and runs it.– Often FS contains superblock which contains

key FS parameters


Example Disk and Filesystem Layout

boot block super block inode list root dir

MBR

Partition Table

partition 1 partition 2 (active) partition 3

free space management files & dirs


Files: Contiguous Allocation• Each file occupies a set of contiguous blocks

on the disk.• Simple – only starting location (block #) and

length (number of blocks) are required.• Random access.• Wasteful of space external

fragmentation, may use compaction to fix.

• Files cannot grow.• Mapping from logical to physical.


Linked Allocation

• Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk.

pointerblock =

Allocate as needed, link together; e.g., file starts at block 9


Linked Allocation (Cont.)


Linked Allocation (Cont.)• Simple – need only starting address• Free-space management system – no

waste of space • No random access• Mapping


Indexed Allocation

• Brings all pointers together into the index block.

• Logical view.

index table


Example of Indexed Allocation


Indexed Allocation (Cont.)• Need index table• Random access• Dynamic access without external

fragmentation, but have overhead of index block.

• Wasted space• Index levels


Indexed Allocation – Mapping

outer-index

index table file


Combined Scheme: UNIX


Disk Space Management

• Block size – disk utilization and performance dependent on file size.– For example, assume medium size of 2KB– 130,000 B/track, 7200RPM (8.33ms),

10ms average seek time– Txfer time = 10 + 4.165 + 8.33*k/130000– rate = k/(10 + 4.165 + 8.33*k/130000)

• Disk space utilization– with larger blocks you increase internal

fragmentation


Free-Space Management

• Bit vector (n blocks)

…

0 1 2 n-1

bit[i] = 0 block[i] free

1 block[i] occupied


Free-Space Management (Cont.)

• Bit map requires extra space. Example:block size = 212 bytes (4096 Bytes)disk size = 234 bytes (16 GByte)n = 234/212 = 222 bits (4Mbits=512

KBytes)• Linked list (free list)

– Cannot get contiguous space easily– No waste of space

• Grouping • Counting


Free-Space Management (Cont.)

• Need to protect:– Pointer to free list– Bit map

• Must be kept on disk• Copy in memory and disk may differ.• Cannot allow for block[i] to have a situation

where bit[i] = 1 in memory and bit[i] = 0 on disk.

– Solution:• Set bit[i] = 1 in disk.• Allocate block[i]• Set bit[i] = 1 in memory


Directory Implementation

• Linear list of file names with pointer to the data blocks.– simple to program– time-consuming to execute

• Hash Table – linear list with hash data structure.– decreases directory search time– collisions – situations where two file names

hash to the same location– fixed size


Efficiency and Performance

• Efficiency dependent on:– disk allocation and directory algorithms– types of data kept in file’s directory entry

• Performance– disk cache – separate section of main

memory for frequently sued blocks– free-behind and read-ahead – techniques

to optimize sequential access– improve PC performance by dedicating

section of memory as virtual disk, or RAM disk.


Various Disk-Caching Locations


Recovery

• Consistency checker – compares data in directory structure with data blocks on disk, and tries to fix inconsistencies.

• Use system programs to back up data from disk to another storage device (floppy disk, magnetic tape).

• Recover lost file or disk by restoring data from backup.


File System Implementations

• UNIX Examples - SVR4 and BSD


UNIX FS Framework

Provides persistent storageFacilities for managing dataInterface exported abstractions: files, directories, file descriptors and file systemskernel does not interpret file contentsfiles and directories form tree structure


File and Directory Organization

/

bin etc dev usr vmunix

etclocal

bin

sh

bash

/usr/local/bin/bash

(hard) links


File Attributes

Type - for example regular, FIFO, special.Reference countsize in bytesdevice idownershipaccess modestimestamps


User View of Files

File Descriptors (open, dup, dup2, fork)All I/O is through file descriptorsreferences the open file objectper process object

File Object - holds contextcreated by an open() system callstores file offsetreference to vnode

vnode - abstract representation of a file


How it works

File Descriptors{{0, uf_ofile} {1, uf_ofile} {2 , uf_ofile} {3 , uf_ofile} {4 , uf_ofile}

{5 , uf_ofile}}

Open File Objects{*f_vnode,f_offset,f_count,...},{*f_vnode,f_offset,f_count,...},{*f_vnode,f_offset,f_count,...},{*f_vnode,f_offset,f_count,...},{*f_vnode,f_offset,f_count,...}}

Vnode/vfsIn-memory

representationof file

Vnode/vfsIn-memory


Vnode/vfsIn-memory


Vnode/vfsIn-memory


Vnode/vfsIn-memory



File Systems

File hierarchy composed of one or more File SystemsOne File System is designated the Root File SystemAttached to mount points File can not span multiple File Systemsresides on one logical disk


Logical Disks

Viewed as linear sequence of fixed sized, randomly accessible blocks.A file system must reside in a logical disk, however a logical disk need not contain a file system.Typically physical disks divided into partitions that correspond to logical disks


FS Overview

System calls

vnode interface

/procPCFSHSFStmpfs swapfs UFS RFS NFS

Anonymousmemory

Processaddressspace

disk cdrom diskette

Example from Solaris


Local Filesystems

S5fs - System V file system. Based on the original implementation.FFS/UFS - BSD developed filesystem with optimized disk usage algorithms


S5fs - Disk layout

Viewed as a linear array of blocksTypical disk block size 512, 1024, 2048 bytesPhysical block number is the block’s indexdisk uses cylinder, track and sectorfirst few blocks are the boot area, which is followed by the inode list (fixed size)


Disk Layout

tract

cylinder

sector heads

plattersRotational speeddisk seek time


bootarea superblock inode list

S5fs disk layout

data

Boot area - code to initialize bootstrap the system

Superblock - metadata for filesystem. Size of FS, sizeof inode list, number of free blocks/inodes, free block/inode list

inode list - linear array of 64byte inode structs


s5fs - some details

name

2 byte

inode

14byte

8450

...“”

myfile123

directory

Di_mode (2)di_nlinks (2)di_uid (2)di_gid (2)di_size (4)di_addr (39)di_gen (1)di_atime (4)di_mtime (4)di_ctime (4)

On-disk inode


Locating file data blocks

0 12 3 45678910 - indirect11 - double indirect12 - triple indirect

256

bloc

ks

65,536 blocks

16,777,216 blocks

Assume 1024 Byte Blocks


BSD - FFS

Disk partition divided into cylinder groupssuperblocks restructured and replicated across partition

Constant informationcylinder group summary info such as free inodes and free block

support block fragmentsLong file namesnew disk block allocation strategy


FFS Allocation strategy

Goal: Collocate similar data/info.file inodes located in same cyl group as dir.new dirs created in different cyl groups.Place file data blocks/inode in same cyl group - for size < 48Kallocate sequential blocks at a rotationally optimal position.Choose cyl group with “best” free count


Is FFS/UFS Better?

Measurements have shown substantial performance benefits over s5fsFFS however, is sub-optimal when the disk is nearly full. Thus 10% is always kept free. Modern disks however, no longer match the underlying assumptions of FFS


Example Disk Buffer Cache

Hash (device,inode)

Free(LRU)

Date post:	14-Nov-2014
Category:	Documents
Upload:	madhur-shailesh-dwivedi
View:	618 times
Download:	1 times

File System

Documents