+ All Categories
Home > Documents > CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman...

CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman...

Date post: 21-Dec-2015
Category:
View: 214 times
Download: 0 times
Share this document with a friend
35
CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman [email protected]
Transcript
Page 1: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

CS444/CS544Operating Systems

Memory Management &

File Systems

4/09/2007

Prof. Searleman

[email protected]

Page 2: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

Outline Memory Management

page replacement algorithms: approximating LRU summary

Introduction to File Systems

Read: Disk scheduling: Chapter 12, sections 12.1 – 12.4 File-System Interface: Chapter 10 File-System Implementation: Chapter 11

HW#10, due Wednesday, 4/11 Exam#3: Tuesday, 4/17, 7:00 pm, SC160

Page 3: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

Approximating LRU Remember the reference bit in the PTE

Set if read or written At some regular interval (much much less often

than for each access) clear all the reference bits Only PTE without the ref bit clear are eligible for

eviction More than 1 bit of state?

Associate some number of counter bits At regular interval, if ref bit is 0 increment counter and

if ref bit is 1 then zero counter Counter tells you # intervals since the last reference More bits you give to counter = more accurate

approximation

Page 4: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

LRU Clock

Also called Second Chance Logically put all physical page frames in a circle

(clock) Maintain a pointer to a current page (clock

hand) When need to evict a page, look at current

page If ref bit off then evict If ref bit on clear it and move on (second chance)

Page 5: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

LRU Clock (con’t)

Arm moves as quickly as evictions are requested

If evictions rarely requested then arm moves slowly and pages have a long time to prove their worth by being referenced

If evictions frequently requested then arm moves fast and little time before the second chance is up

Page 6: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.6 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Second-Chance (clock) Page-Replacement AlgorithmSecond-Chance (clock) Page-Replacement Algorithm

Page 7: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.7 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Counting AlgorithmsCounting Algorithms

Keep a counter of the number of references that have been made to each page

LFU Algorithm: replaces page with smallest count

MFU Algorithm: based on the argument that the page with the smallest count was probably just brought in and has yet to be used

Page 8: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

Best page replacement?

Of course it depends Interestingly if have too much memory it

doesn’t matter anything you do will be ok (overprovisioning)

Also doesn’t matter if have too little memory Thrashing and nothing you can do to stop it

(overcommitted) So much does it cost just to overprovision?

Page 9: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

Fairness?

All the replacement policies we’ve looked at so far just try to pick the page to evict regardless of which process the page belongs to

What if demand page in from one process causes the eviction of another processes page? Is that fair?

On the other hand is it fair for one process to have 2 times their working set while another process has ½ their working set and is paging heavily?

Page 10: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

Fixed vs Variable Space

Fixed space algorithms Give each process a limit of pages it can use When it reaches its limit, it replaces LRU or FIFO or

whatever from its pages May be more natural to give process a say in the

replacement policy used for its pages Variable space algorithms

Processes set of pages grows and shrinks One process can ruin it for the rest but opportunity

to make globally better decisions

Page 11: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.11 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Priority AllocationPriority Allocation

Use a proportional allocation scheme using priorities rather than size

If process Pi generates a page fault, select for replacement one of its frames

select for replacement a frame from a process with lower priority number

Page 12: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.12 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Global vs. Local AllocationGlobal vs. Local Allocation

Global replacement – process selects a replacement frame from the set of all frames; one process can take a frame from another

Local replacement – each process selects from only its own set of allocated frames

Page 13: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.13 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

ThrashingThrashing

If a process does not have “enough” pages, the page-fault rate is very high. This leads to: low CPU utilization

operating system thinks that it needs to increase the degree of multiprogramming

another process added to the system

Thrashing a process is busy swapping pages in and out

Page 14: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.14 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Thrashing (Cont.)Thrashing (Cont.)

Page 15: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.15 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Demand Paging and Thrashing Demand Paging and Thrashing

Why does demand paging work?Locality model Process migrates from one locality to another

Localities may overlap

Why does thrashing occur? size of locality > total memory size

Page 16: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

Use Working Set

Could ask each process to inform the OS of the size of its working set

OS only allow a process to start if it can allocate the complete working set

How easy for processes to report this?

Page 17: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.17 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Working-Set ModelWorking-Set Model

working-set window a fixed number of page references (e.g.: 10,000 instruction)

WSSi (working set of Process Pi) =total number of pages referenced in the most recent (varies in time) if too small will not encompass entire locality if too large will encompass several localities if = will encompass entire program

D = WSSi total demand frames if D > m Thrashing Policy if D > m, then suspend one of the

processes

Page 18: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.18 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Working-set modelWorking-set model

Page 19: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.19 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Keeping Track of the Working SetKeeping Track of the Working Set

Approximate with interval timer + a reference bit Example: = 10,000

Timer interrupts after every 5000 time units Keep in memory 2 bits for each page Whenever a timer interrupts copy and sets the values of all reference bits

to 0 If one of the bits in memory = 1 page in working set

Why is this not completely accurate? Improvement = 10 bits and interrupt every 1000

time units

Page 20: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

Page Fault Frequency (PFF)

PFF is a variable-space algorithm that tries to determine the working set size dynamically

Monitor page fault rate for each process If fault rate is above a given threshold, give it

more memory If fault rate is below threshhold, take away

memory Constant adjustment? Dampening factor so

only changes occasionally

Page 21: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.21 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Page-Fault Frequency SchemePage-Fault Frequency Scheme

Establish “acceptable” page-fault rate If actual rate too low, process loses frame If actual rate too high, process gains frame

Page 22: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.22 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Memory-Mapped FilesMemory-Mapped Files

Memory-mapped file I/O allows file I/O to be treated as routine memory access by mapping a disk block to a page in memory

A file is initially read using demand paging. A page-sized portion of the file is read from the file system into a physical page. Subsequent reads/writes to/from the file are treated as ordinary memory accesses.

Simplifies file access by treating file I/O through memory rather than read() write() system calls

Also allows several processes to map the same file allowing the pages in memory to be shared

Page 23: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.23 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Memory Mapped FilesMemory Mapped Files

Page 24: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.24 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Other Issues -- PrepagingOther Issues -- Prepaging

Prepaging To reduce the large number of page faults that occurs at process startup Prepage all or some of the pages a process will need, before they are

referenced But if prepaged pages are unused, I/O and memory was wasted Assume s pages are prepaged and α of the pages is used

Is cost of s * α save pages faults > or < than the cost of prepaging s * (1- α) unnecessary pages?

α near zero prepaging loses

Page 25: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.25 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Other Issues – Page SizeOther Issues – Page Size

Page size selection must take into consideration: fragmentation

table size

I/O overhead

locality

Page 26: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.26 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Other Issues – TLB Reach Other Issues – TLB Reach

TLB Reach - The amount of memory accessible from the TLB

TLB Reach = (TLB Size) X (Page Size) Ideally, the working set of each process is stored

in the TLB. Otherwise there is a high degree of page faults.

Increase the Page Size. This may lead to an increase in fragmentation as not all applications require a large page size

Provide Multiple Page Sizes. This allows applications that require larger page sizes the opportunity to use them without an increase in fragmentation.

Page 27: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.27 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Other Issues – Program StructureOther Issues – Program Structure

Program structure Int[128,128] data; Each row is stored in one page Program 1 for (j = 0; j <128; j++)

for (i = 0; i < 128; i++) data[i,j] = 0;128 x 128 = 16,384 page faults

Program 2 for (i = 0; i < 128; i++) for (j = 0; j < 128; j++) data[i,j] = 0;

128 page faults

Page 28: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

9.28 Silberschatz, Galvin and Gagne ©2005Operating System Concepts

Other Issues – I/O interlockOther Issues – I/O interlock

I/O Interlock – Pages must sometimes be locked into memory

Consider I/O. Pages that are used for copying a file from a device must be locked from being selected for eviction by a page replacement algorithm.

Page 29: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

What’s the situation?(1) CPU utilization 20% Paging disk 97.5% Other I/O devices 5%

(2) CPU utilization 15% Paging disk 4% Other I/O devices 86%

Page 30: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

Summary

Demand paging Start with no physical memory pages mapped

and load them in on demand Page replacement Algorithms

Belady – optimal but unrealizable FIFO – replace page loaded earliest LRU – replace page referenced earliest Working Set – keep set of pages in memory that

induces minimal fault rate (need program specification)

PFF – Grow/shrink page set as a function of fault rate

Fairness – globally optimal replacement vs protecting processes from each other?

Page 31: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

File Systems

Despite complex internals, disks export a simple array of sectors

How do we go from that to a file system? What do we exactly do we expect from a file

system?

Page 32: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

File System Basics

FS are probably the OS abstraction that average user is most familiar with Files Directories Access controls (owners, groups, permissions)

Page 33: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

Files

A file is a collection of data with system maintained properties like Owner, size, name, last read/write time, etc.

Files often have “types” which allow users and applications to recognize their intended use

Some file types are understood by the file system (mount point, symbolic link, directory)

Some file types are understood by applications and users (.txt, .jpg, .html, .doc, …) Could the system understand these types and customize its

handling?

Page 34: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

Basic File Operations

UNIX create (name) open (name, mode) read (fd) write(fd) sync(fd) seek(fd, pos) close(fd) unlink (name) rename (old, new)

Windows CreateFile(name, CREATE) CreateFile(name, OPEN) ReadFile(handle) WriteFile (handle) FlushFileBuffers(handle) SetFilePointer(handle) CloseHandle(handl) DeleteFile(name) CopyFile(name) MoveFile(name)

Page 35: CS444/CS544 Operating Systems Memory Management & File Systems 4/09/2007 Prof. Searleman jets@clarkson.edu.

Directories

Directories provide a way for users to organize their files *and* a convenient way for users to identify and share data

Most file systems support hierarchical directories (/usr/local/bin or C:\WINNT) People like to organize information hierarchically

Recall: OS often records a current working directory for each process Can therefore refer to files by absolute and

relative names


Recommended