CS444/CS544Operating Systems
Memory Management &
File Systems
4/09/2007
Prof. Searleman
Outline Memory Management
page replacement algorithms: approximating LRU summary
Introduction to File Systems
Read: Disk scheduling: Chapter 12, sections 12.1 – 12.4 File-System Interface: Chapter 10 File-System Implementation: Chapter 11
HW#10, due Wednesday, 4/11 Exam#3: Tuesday, 4/17, 7:00 pm, SC160
Approximating LRU Remember the reference bit in the PTE
Set if read or written At some regular interval (much much less often
than for each access) clear all the reference bits Only PTE without the ref bit clear are eligible for
eviction More than 1 bit of state?
Associate some number of counter bits At regular interval, if ref bit is 0 increment counter and
if ref bit is 1 then zero counter Counter tells you # intervals since the last reference More bits you give to counter = more accurate
approximation
LRU Clock
Also called Second Chance Logically put all physical page frames in a circle
(clock) Maintain a pointer to a current page (clock
hand) When need to evict a page, look at current
page If ref bit off then evict If ref bit on clear it and move on (second chance)
LRU Clock (con’t)
Arm moves as quickly as evictions are requested
If evictions rarely requested then arm moves slowly and pages have a long time to prove their worth by being referenced
If evictions frequently requested then arm moves fast and little time before the second chance is up
9.6 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Second-Chance (clock) Page-Replacement AlgorithmSecond-Chance (clock) Page-Replacement Algorithm
9.7 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Counting AlgorithmsCounting Algorithms
Keep a counter of the number of references that have been made to each page
LFU Algorithm: replaces page with smallest count
MFU Algorithm: based on the argument that the page with the smallest count was probably just brought in and has yet to be used
Best page replacement?
Of course it depends Interestingly if have too much memory it
doesn’t matter anything you do will be ok (overprovisioning)
Also doesn’t matter if have too little memory Thrashing and nothing you can do to stop it
(overcommitted) So much does it cost just to overprovision?
Fairness?
All the replacement policies we’ve looked at so far just try to pick the page to evict regardless of which process the page belongs to
What if demand page in from one process causes the eviction of another processes page? Is that fair?
On the other hand is it fair for one process to have 2 times their working set while another process has ½ their working set and is paging heavily?
Fixed vs Variable Space
Fixed space algorithms Give each process a limit of pages it can use When it reaches its limit, it replaces LRU or FIFO or
whatever from its pages May be more natural to give process a say in the
replacement policy used for its pages Variable space algorithms
Processes set of pages grows and shrinks One process can ruin it for the rest but opportunity
to make globally better decisions
9.11 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Priority AllocationPriority Allocation
Use a proportional allocation scheme using priorities rather than size
If process Pi generates a page fault, select for replacement one of its frames
select for replacement a frame from a process with lower priority number
9.12 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Global vs. Local AllocationGlobal vs. Local Allocation
Global replacement – process selects a replacement frame from the set of all frames; one process can take a frame from another
Local replacement – each process selects from only its own set of allocated frames
9.13 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
ThrashingThrashing
If a process does not have “enough” pages, the page-fault rate is very high. This leads to: low CPU utilization
operating system thinks that it needs to increase the degree of multiprogramming
another process added to the system
Thrashing a process is busy swapping pages in and out
9.14 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Thrashing (Cont.)Thrashing (Cont.)
9.15 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Demand Paging and Thrashing Demand Paging and Thrashing
Why does demand paging work?Locality model Process migrates from one locality to another
Localities may overlap
Why does thrashing occur? size of locality > total memory size
Use Working Set
Could ask each process to inform the OS of the size of its working set
OS only allow a process to start if it can allocate the complete working set
How easy for processes to report this?
9.17 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Working-Set ModelWorking-Set Model
working-set window a fixed number of page references (e.g.: 10,000 instruction)
WSSi (working set of Process Pi) =total number of pages referenced in the most recent (varies in time) if too small will not encompass entire locality if too large will encompass several localities if = will encompass entire program
D = WSSi total demand frames if D > m Thrashing Policy if D > m, then suspend one of the
processes
9.18 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Working-set modelWorking-set model
9.19 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Keeping Track of the Working SetKeeping Track of the Working Set
Approximate with interval timer + a reference bit Example: = 10,000
Timer interrupts after every 5000 time units Keep in memory 2 bits for each page Whenever a timer interrupts copy and sets the values of all reference bits
to 0 If one of the bits in memory = 1 page in working set
Why is this not completely accurate? Improvement = 10 bits and interrupt every 1000
time units
Page Fault Frequency (PFF)
PFF is a variable-space algorithm that tries to determine the working set size dynamically
Monitor page fault rate for each process If fault rate is above a given threshold, give it
more memory If fault rate is below threshhold, take away
memory Constant adjustment? Dampening factor so
only changes occasionally
9.21 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Page-Fault Frequency SchemePage-Fault Frequency Scheme
Establish “acceptable” page-fault rate If actual rate too low, process loses frame If actual rate too high, process gains frame
9.22 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Memory-Mapped FilesMemory-Mapped Files
Memory-mapped file I/O allows file I/O to be treated as routine memory access by mapping a disk block to a page in memory
A file is initially read using demand paging. A page-sized portion of the file is read from the file system into a physical page. Subsequent reads/writes to/from the file are treated as ordinary memory accesses.
Simplifies file access by treating file I/O through memory rather than read() write() system calls
Also allows several processes to map the same file allowing the pages in memory to be shared
9.23 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Memory Mapped FilesMemory Mapped Files
9.24 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Other Issues -- PrepagingOther Issues -- Prepaging
Prepaging To reduce the large number of page faults that occurs at process startup Prepage all or some of the pages a process will need, before they are
referenced But if prepaged pages are unused, I/O and memory was wasted Assume s pages are prepaged and α of the pages is used
Is cost of s * α save pages faults > or < than the cost of prepaging s * (1- α) unnecessary pages?
α near zero prepaging loses
9.25 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Other Issues – Page SizeOther Issues – Page Size
Page size selection must take into consideration: fragmentation
table size
I/O overhead
locality
9.26 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Other Issues – TLB Reach Other Issues – TLB Reach
TLB Reach - The amount of memory accessible from the TLB
TLB Reach = (TLB Size) X (Page Size) Ideally, the working set of each process is stored
in the TLB. Otherwise there is a high degree of page faults.
Increase the Page Size. This may lead to an increase in fragmentation as not all applications require a large page size
Provide Multiple Page Sizes. This allows applications that require larger page sizes the opportunity to use them without an increase in fragmentation.
9.27 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Other Issues – Program StructureOther Issues – Program Structure
Program structure Int[128,128] data; Each row is stored in one page Program 1 for (j = 0; j <128; j++)
for (i = 0; i < 128; i++) data[i,j] = 0;128 x 128 = 16,384 page faults
Program 2 for (i = 0; i < 128; i++) for (j = 0; j < 128; j++) data[i,j] = 0;
128 page faults
9.28 Silberschatz, Galvin and Gagne ©2005Operating System Concepts
Other Issues – I/O interlockOther Issues – I/O interlock
I/O Interlock – Pages must sometimes be locked into memory
Consider I/O. Pages that are used for copying a file from a device must be locked from being selected for eviction by a page replacement algorithm.
What’s the situation?(1) CPU utilization 20% Paging disk 97.5% Other I/O devices 5%
(2) CPU utilization 15% Paging disk 4% Other I/O devices 86%
Summary
Demand paging Start with no physical memory pages mapped
and load them in on demand Page replacement Algorithms
Belady – optimal but unrealizable FIFO – replace page loaded earliest LRU – replace page referenced earliest Working Set – keep set of pages in memory that
induces minimal fault rate (need program specification)
PFF – Grow/shrink page set as a function of fault rate
Fairness – globally optimal replacement vs protecting processes from each other?
File Systems
Despite complex internals, disks export a simple array of sectors
How do we go from that to a file system? What do we exactly do we expect from a file
system?
File System Basics
FS are probably the OS abstraction that average user is most familiar with Files Directories Access controls (owners, groups, permissions)
Files
A file is a collection of data with system maintained properties like Owner, size, name, last read/write time, etc.
Files often have “types” which allow users and applications to recognize their intended use
Some file types are understood by the file system (mount point, symbolic link, directory)
Some file types are understood by applications and users (.txt, .jpg, .html, .doc, …) Could the system understand these types and customize its
handling?
Basic File Operations
UNIX create (name) open (name, mode) read (fd) write(fd) sync(fd) seek(fd, pos) close(fd) unlink (name) rename (old, new)
Windows CreateFile(name, CREATE) CreateFile(name, OPEN) ReadFile(handle) WriteFile (handle) FlushFileBuffers(handle) SetFilePointer(handle) CloseHandle(handl) DeleteFile(name) CopyFile(name) MoveFile(name)
Directories
Directories provide a way for users to organize their files *and* a convenient way for users to identify and share data
Most file systems support hierarchical directories (/usr/local/bin or C:\WINNT) People like to organize information hierarchically
Recall: OS often records a current working directory for each process Can therefore refer to files by absolute and
relative names