Opera

ting

Syst

em

s

Ste

ven

Han

d

Michaelm

asTerm

2010

12le

cture

sfo

rC

ST

IA

Oper

atin

gSys

tem

s—

N/H

/MW

F@

12

Cours

eA

ims

•T

his

cours

eai

ms

to:

–ex

pla

inth

est

ruct

ure

and

funct

ions

ofan

oper

atin

gsy

stem

,

–ill

ust

rate

key

oper

atin

gsy

stem

aspec

tsby

concr

ete

exam

ple

,an

d

–pr

epar

eyo

ufo

rfu

ture

cours

es...

•A

tth

een

dof

the

cours

eyo

ush

ould

be

able

to:

–co

mpar

ean

dco

ntr

ast

CP

Usc

hed

ulin

gal

gorith

ms

–ex

pla

inth

efo

llow

ing:

proce

ss,ad

dre

sssp

ace,

file

.

–distinguish

pag

edan

dse

gmen

ted

virt

ual

mem

ory.

–discu

ssth

ere

lative

mer

its

ofU

nix

and

NT

...

Oper

atin

gSys

tem

s—

Aim

si

Cours

eO

utlin

e

•In

troduct

ion

toO

per

atin

gSys

tem

s.

•P

roce

sses

&Sch

edulin

g.

•M

emor

yM

anag

emen

t.

•I/

O&

Dev

ice

Man

agem

ent.

•P

rote

ctio

n.

•Fili

ng

Sys

tem

s.

•C

ase

Stu

dy:

Unix

.

•C

ase

Stu

dy:

Win

dow

sN

T.

Oper

atin

gSys

tem

s—

Outlin

eii

Reco

mm

ended

Readin

g

•Concurren

tSystems

orOperatingSystems

Bac

onJ

[an

dH

arris

T],

Addison

Wes

ley

1997

[200

3]

•OperatingSystemsConcepts

(5th

Ed.)

Silb

ersc

hat

zA

,Pet

erso

nJ

and

Gal

vin

P,A

ddison

Wes

ley

199

8.

•TheDesignandIm

plemen

tationofthe4.3BSD

UNIX

Operating

System

Leffl

erS

J,A

ddison

Wes

ley

1989

•InsideWindows2000(3rd

Ed)

orWindowsInternals

(4th

Ed)

Sol

omon

Dan

dR

uss

inov

ich

M,M

icro

soft

Pre

ss20

00[2

005]

Oper

atin

gSys

tem

s—

Book

siii

What

isan

Opera

ting

Sys

tem

?

•A

prog

ram

whic

hco

ntr

ols

the

exec

ution

ofal

lot

her

prog

ram

s

(applic

atio

ns)

.

•A

cts

asan

inte

rmed

iary

bet

wee

nth

euse

r(s)

and

the

com

pute

r.

•O

bje

ctiv

es:

–co

nve

nie

nce

,

–effi

cien

cy,

–ex

tensibili

ty.

•Sim

ilar

toa

gov

ernm

ent.

..

Oper

atin

gSys

tem

s—

Intr

oduct

ion

1

An

Abst

ract

Vie

w

Oper

ati

ng S

yste

m

Hard

ware

App 2

App N

App 1

•T

he

Oper

atin

gSys

tem

(OS):

–co

ntr

ols

allex

ecution

.

–m

ultip

lexe

sre

sourc

esbet

wee

nap

plic

atio

ns.

–ab

stra

cts

away

from

com

ple

xity

.

•Typ

ical

lyal

sohav

eso

melibraries

and

som

etools

prov

ided

with

OS.

•A

reth

ese

par

tof

the

OS?

IsIE

ato

ol?

–no-

one

can

agre

e...

•For

us,

the

OS≈

thekernel

.

Oper

atin

gSys

tem

s—

Intr

oduct

ion

2

InT

he

Begin

nin

g...

•19

49:

First

stor

ed-p

rogr

amm

achin

e(E

DSA

C)

•to∼

1955:

“Open

Shop

”.

–la

rge

mac

hin

esw

ith

vacu

um

tubes

.

–I/

Oby

pap

erta

pe

/punch

card

s.

–use

r=

prog

ram

mer

=op

erat

or.

•To

reduce

cost

,hire

anoperator:

–pr

ogra

mm

ers

write

prog

ram

san

dsu

bm

itta

pe/

card

sto

oper

ator

.

–op

erat

orfe

eds

card

s,co

llect

sou

tput

from

prin

ter.

•M

anag

emen

tlik

eit.

•P

rogr

amm

ers

hat

eit.

•O

per

ator

shat

eit.

⇒nee

dso

met

hin

gbet

ter.

Oper

atin

gSys

tem

s—

Evo

lution

3

Batc

hSys

tem

s

•In

troduct

ion

ofta

pe

drive

sal

low

batching

ofjo

bs:

–pr

ogra

mm

ers

put

jobs

on

card

sas

bef

ore.

–al

lca

rds

read

onto

ata

pe.

–op

erat

orca

rrie

sin

put

tape

toco

mpute

r.

–re

sults

writt

ento

outp

ut

tape.

–ou

tput

tape

take

nto

prin

ter.

•C

ompute

rnow

has

aresiden

tmonitor:

–in

itia

llyco

ntr

olis

inm

onitor

.

–m

onitor

read

sjo

ban

dtr

ansf

erco

ntr

ol.

–at

end

ofjo

b,co

ntr

oltr

ansf

ers

bac

kto

mon

itor

.

•Eve

nbet

ter:

spoolingsystem

s.

–use

inte

rrupt

drive

nI/

O.

–use

mag

net

icdisk

toca

che

input

tape.

–fire

oper

ator

.

•M

onitor

now

schedules

jobs.

..

Oper

atin

gSys

tem

s—

Evo

lution

4

Multi-Pro

gra

mm

ing

Op

era

tin

g

Sys

tem

Job 1

Job 2

Job 3

Job 4

Op

era

tin

g

Sys

tem

Job 1

Job 2

Job 3

Job 4

Op

era

tin

g

Sys

tem

Job 1

Job 2

Job 3

Job 4

Tim

e

•U

sem

emor

yto

cach

ejo

bs

from

disk⇒

mor

eth

anon

ejo

bac

tive

sim

ultan

eously.

•T

wo

stag

esc

hed

ulin

g:

1.se

lect

jobs

tolo

ad:jobscheduling.

2.se

lect

residen

tjo

bto

run:CPU

scheduling.

•U

sers

wan

tm

ore

inte

ract

ion⇒

time-sharing:

•e.

g.C

TSS,T

SO

,U

nix

,V

MS,W

indow

sN

T...

Oper

atin

gSys

tem

s—

Evo

lution

5

Today

and

Tom

orr

ow

•Sin

gle

use

rsy

stem

s:ch

eap

and

chee

rful.

–per

sonal

com

pute

rs.

–no

other

use

rs⇒

ignor

epr

otec

tion

.

–e.

g.D

OS,W

indow

s,W

in95

/98,

...

•R

TSys

tem

s:pow

eris

not

hin

gw

ithou

tco

ntr

ol.

–har

d-r

ealtim

e:nucl

ear

reac

tor

safe

tym

onitor

.

–so

ft-r

ealtim

e:m

p3

pla

yer.

•Par

alle

lP

roce

ssin

g:th

enee

dfo

rsp

eed.

–SM

P:2–8

proce

ssor

sin

abox

.

–M

IMD

:su

per

-com

puting.

•D

istr

ibute

dco

mputing:

glob

alpr

oce

ssin

g?

–Ja

va:

the

net

wor

kis

the

com

pute

r.

–C

lust

erin

g:th

enet

wor

kis

the

bus.

–C

OR

BA

:th

eco

mpute

ris

the

net

wor

k.

–.N

ET

:th

enet

wor

kis

anen

ablin

gfr

amew

ork.

..

Oper

atin

gSys

tem

s—

Evo

lution

6

Monolith

icO

pera

ting

Syst

em

s

H/W

S/W

App.

App.

App.

Schedule

r

Devic

e D

river

Devic

e D

river

App.

•O

ldes

tki

nd

ofO

Sst

ruct

ure

(“m

oder

n”

exam

ple

sar

eD

OS,or

igin

alM

acO

S)

•P

roble

m:

applic

atio

ns

can

e.g.

–tr

ash

OS

soft

war

e.

–tr

ash

anot

her

applic

atio

n.

–hoa

rdC

PU

tim

e.

–ab

use

I/O

dev

ices

.

–et

c...

•N

ogo

od

for

fault

conta

inm

ent

(or

multi-use

r).

•N

eed

abet

ter

solu

tion

...

Oper

atin

gSys

tem

s—

Str

uct

ure

s&

Pro

tect

ion

Mec

han

ism

s7

Dual-M

ode

Opera

tion

•W

ant

tost

opbugg

y(o

rm

alic

ious)

prog

ram

from

doi

ng

bad

thin

gs.

⇒pr

ovid

ehardware

suppor

tto

distingu

ish

bet

wee

n(a

tle

ast)

two

diff

eren

tm

odes

ofop

erat

ion:

1.UserMode

:w

hen

exec

uting

onbeh

alf

ofa

use

r(i.e

.ap

plic

atio

npr

ogra

ms)

.

2.Kernel

Mode

:w

hen

exec

uting

onbeh

alf

ofth

eop

erat

ing

syst

em.

•H

ardw

are

conta

ins

am

ode-

bit,e.

g.0

mea

ns

kern

el,1

mea

ns

use

r.

Ker

nel

Mo

de

Use

r

Mo

de

rese

t

inte

rru

pt

or

fau

lt

se

t u

se

r m

od

e

•M

ake

cert

ain

mac

hin

ein

stru

ctio

ns

only

pos

sible

inke

rnel

mode.

..

Oper

atin

gSys

tem

s—

Str

uct

ure

s&

Pro

tect

ion

Mec

han

ism

s8

Pro

tect

ing

I/O

&M

em

ory

•First

try:

mak

eI/

Oin

stru

ctio

ns

priv

ilege

d.

–ap

plic

atio

ns

can’t

mas

kin

terr

upts

.

–ap

plic

atio

ns

can’t

contr

olI/

Odev

ices

.

•B

ut:

1.A

pplic

atio

nca

nre

write

inte

rrupt

vect

ors.

2.Som

edev

ices

acce

ssed

viamem

ory

•H

ence

nee

dto

prot

ect

mem

ory

also

,e.

g.defi

nebase

andlimit

for

each

prog

ram

:

Op

era

tin

g

Sys

tem

Job 1

Job 2

Job 3

Job 4

0x0000

0x3000

0x5000

0x9800

0xD800

0xFFFF

0x5000

0x4800

lim

it r

egis

ter

ba

se r

egis

ter

•A

cces

ses

outs

ide

allo

wed

range

are

prot

ecte

d.

Oper

atin

gSys

tem

s—

Str

uct

ure

s&

Pro

tect

ion

Mec

han

ism

s9

Mem

ory

Pro

tect

ion

Har

dwar

e

CP

U

vect

or

to O

S (

ad

dre

ss e

rro

r)

yes

no

yes

no

base

base+limit

Memory

•H

ardw

are

chec

ksev

ery

mem

ory

refe

rence

.

•A

cces

sou

tof

range⇒

vect

orin

toop

erat

ing

syst

em(j

ust

asfo

ran

inte

rrupt)

.

•O

nly

allo

wupdate

ofbas

ean

dlim

itre

gist

ers

inke

rnel

mode.

•Typ

ical

lydisab

lem

emor

ypr

otec

tion

inke

rnel

mode

(althou

gha

bad

idea

).

•In

real

ity,

mor

eco

mple

xpr

otec

tion

h/w

use

d:

–m

ain

schem

esar

esegm

entation

andpaging

–(c

over

edla

ter

onin

cours

e)

Oper

atin

gSys

tem

s—

Str

uct

ure

s&

Pro

tect

ion

Mec

han

ism

s10

Pro

tect

ing

the

CPU

•N

eed

toen

sure

that

the

OS

stay

sin

contr

ol.

–i.e.

nee

dto

prev

ent

any

am

alic

ious

orbad

ly-w

ritt

enap

plic

atio

nfr

om‘h

oggi

ng’

the

CP

Uth

ew

hol

etim

e.

⇒use

atimer

dev

ice.

•U

sual

lyuse

acountdown

tim

er,e.

g.

1.se

ttim

erto

initia

lva

lue

(e.g

.0xFFFF).

2.ev

erytick

(e.g

.1µs)

,tim

erdec

rem

ents

valu

e.

3.w

hen

valu

ehits

zero

,in

terr

upt.

•(M

oder

ntim

ers

hav

epr

ogra

mm

able

tick

rate

.)

•H

ence

OS

gets

toru

nper

iodic

ally

and

do

its

stuff.

•N

eed

toen

sure

only

OS

can

load

tim

er,an

dth

atin

terr

upt

cannot

be

mas

ked.

–use

sam

esc

hem

eas

for

other

dev

ices

.

–(v

iz.pr

ivile

ged

inst

ruct

ions,

mem

ory

prot

ection

)

•Sam

esc

hem

eca

nbe

use

dto

imple

men

ttim

e-sh

arin

g(m

ore

onth

isla

ter)

.

Oper

atin

gSys

tem

s—

Str

uct

ure

s&

Pro

tect

ion

Mec

han

ism

s11

Kern

el-B

ase

dO

pera

ting

Sys

tem

s

H/W

S/W

App.

Priv

Unpriv

App.

App.

App.

Kern

el

Schedule

r

Devic

e D

river

Devic

e D

river

Syste

m C

alls

File

Syste

mP

roto

col C

ode

•A

pplic

atio

ns

can’t

do

I/O

due

topr

otec

tion

⇒op

erat

ing

syst

emdoes

iton

thei

rbeh

alf.

•N

eed

secu

rew

ayfo

rap

plic

atio

nto

invo

keop

erat

ing

syst

em:

⇒re

quire

asp

ecia

l(u

npr

ivile

ged)

inst

ruct

ion

toal

low

tran

sition

from

use

rto

kern

elm

ode.

•G

ener

ally

calle

dasoftware

interrupt

since

oper

ates

sim

ilarly

toa

real

(har

dw

are)

inte

rrupt.

..

•Set

ofO

Sse

rvic

esac

cess

ible

via

soft

war

ein

terr

upt

mec

han

ism

calle

dsystem

calls.

Oper

atin

gSys

tem

s—

Str

uct

ure

s&

Pro

tect

ion

Mec

han

ism

s12

Mic

roke

rnelO

pera

ting

Sys

tem

s

H/W

S/W

Ap

p.

Priv

Unpriv

Se

rve

rD

evic

eD

rive

r

Se

rve

rS

erv

erA

pp

.A

pp

.A

pp

.

Kern

el

Schedule

r

De

vic

eD

rive

r

•A

lter

nat

ive

stru

cture

:

–push

som

eO

Sse

rvic

esin

toservers.

–se

rver

sm

aybe

priv

ilege

d(i.e

.op

erat

ein

kern

elm

ode)

.

•In

crea

ses

bot

hmodularity

andextensibility

.

•Still

acce

sske

rnel

via

syst

emca

lls,but

nee

dnew

way

toac

cess

serv

ers:

⇒in

terp

roce

ssco

mm

unic

atio

n(I

PC

)sc

hem

es.

Oper

atin

gSys

tem

s—

Str

uct

ure

s&

Pro

tect

ion

Mec

han

ism

s13

Kern

els

vers

us

Mic

roke

rnels

So

why

isn’t

ever

ythin

ga

mic

roke

rnel

?

•Lot

sof

IPC

adds

over

hea

d

⇒m

icro

kern

els

usu

ally

per

form

less

wel

l.

•M

icro

kern

elim

ple

men

tation

som

etim

estr

icky

:nee

dto

wor

ryab

out

concu

rren

cyan

dsy

nch

ronisat

ion.

•M

icro

kern

els

ofte

nen

dup

with

redundan

tco

pie

sof

OS

dat

ast

ruct

ure

s.

Hen

ceto

day

mos

tco

mm

onoper

atin

gsy

stem

sblu

rth

edistinct

ion

bet

wee

nke

rnel

and

mic

roke

rnel

.

•e.

g.lin

ux

isa

“ker

nel

”,but

has

kern

elm

odule

san

dce

rtai

nse

rver

s.

•e.

g.W

indow

sN

Tw

asor

igin

ally

mic

roke

rnel

(3.5

),but

now

(4.0

onw

ards)

push

edlo

tsbac

kin

toke

rnel

for

per

form

ance

.

•Still

not

clea

rw

hat

the

bes

tO

Sst

ruct

ure

is,or

how

much

itre

ally

mat

ters

...

Oper

atin

gSys

tem

s—

Str

uct

ure

s&

Pro

tect

ion

Mec

han

ism

s14

Opera

ting

Syst

em

Funct

ions

•R

egar

dle

ssof

stru

cture

,O

Snee

ds

tosecurely

multiplexresources

:

1.pr

otec

tap

plic

atio

ns

from

each

other

,ye

t

2.sh

are

phys

ical

reso

urc

esbet

wee

nth

em.

•A

lso

usu

ally

wan

tto

abstract

away

from

grungy

har

war

e,i.e.

OS

provi

des

avirtualmachine:

–sh

are

CP

U(in

tim

e)an

dpr

ovid

eea

chap

pw

ith

avi

rtual

proce

ssor

,

–al

loca

tean

dpr

ote

ctm

emor

y,an

dpr

ovid

eap

plic

atio

ns

with

thei

r

own

virt

ual

addre

sssp

ace,

–pr

esen

ta

set

of(r

elat

ivel

y)har

dw

are

indep

enden

tvi

rtual

dev

ices

,

–div

ide

up

stor

age

spac

eby

using

filin

gsy

stem

s,an

d

–do

allth

isw

ithin

the

conte

xtof

ase

curity

fram

ewor

k.

•R

emai

nder

ofth

ispar

tof

the

cours

ew

illlo

ok

atea

chof

the

abov

e

area

sin

turn

...

Oper

atin

gSys

tem

s—

Funct

ions

15

Pro

cess

Conce

pt

•Fro

ma

use

r’s

poi

nt

ofvi

ew,th

eop

erat

ing

syst

emis

ther

eto

exec

ute

prog

ram

s:

–on

bat

chsy

stem

,re

fer

tojobs

–on

inte

ract

ive

syst

em,re

fer

toprocesses

–(w

e’ll

use

bot

hte

rms

fairly

inte

rchan

geab

ly)

•P

roce

ss6=

Pro

gram

:

–a

prog

ram

isstatic,

while

apr

oce

ssisdyn

amic

–in

fact

,a

proce

ss△ =

“a

prog

ram

inex

ecution

”

•(N

ote:

“pro

gram

”her

eis

pret

tylo

wle

vel,

i.e.

nat

ive

mac

hin

eco

de

orexecutable)

•P

roce

ssin

cludes

:

1.pr

ogra

mco

unte

r

2.st

ack

3.dat

ase

ctio

n

•P

roce

sses

exec

ute

onvirtualprocessors

Oper

atin

gSys

tem

s—

Pro

cess

es16

Pro

cess

Sta

tes

Exit

Ru

nn

ing

New

Read

y

Blo

cked

dis

patc

h

tim

eout

or

yie

ld

rele

ase

adm

it

event-

wait

event

•A

sa

proce

ssex

ecute

s,it

chan

gesstate

:

–N

ew:

the

proce

ssis

bei

ng

crea

ted

–R

unnin

g:

inst

ruct

ions

are

bei

ng

exec

ute

d

–R

eady:

the

proce

ssis

wai

ting

for

the

CP

U(a

nd

ispr

epar

edto

run

atan

ytim

e)

–B

lock

ed:

the

proce

ssis

wai

ting

for

som

eev

ent

toocc

ur

(and

cannot

run

untilit

does

)

–Exi

t:th

epr

oce

sshas

finished

exec

ution

.

•T

he

oper

atin

gsy

stem

isre

spon

sible

for

mai

nta

inin

gth

est

ate

ofea

chpr

oce

ss.

Oper

atin

gSys

tem

s—

Pro

cess

es17

Pro

cess

Contr

olB

lock

Pro

cess N

um

ber

(or

Pro

cess ID

)

Cu

rren

t P

rocess S

tate

Oth

er

CP

U R

eg

iste

rs

Mem

ory

Man

gem

en

t In

form

ati

on

CP

U S

ch

ed

ulin

g In

form

ati

on

Pro

gra

m C

ou

nte

r

Oth

er

Info

rmati

on

(e

.g. list

of

op

en

files, n

am

e o

f

execu

tab

le, id

en

tity

of

ow

ner,

CP

U

tim

e u

sed

so

far,

devic

es o

wn

ed

)

Refs

to

pre

vio

us a

nd

next

PC

Bs

OS

mai

nta

ins

info

rmat

ion

abou

tev

ery

proce

ssin

adat

ast

ruct

ure

calle

daprocess

controlblock

(PC

B):

•U

niq

ue

proce

ssid

entifier

•P

roce

ssst

ate

(Running,Ready,

etc.

)

•C

PU

sched

ulin

g&

acco

unting

info

rmat

ion

•P

rogr

amco

unte

r&

CP

Ure

gist

ers

•M

emor

ym

anag

emen

tin

form

atio

n

•...

Oper

atin

gSys

tem

s—

Pro

cess

es18

Conte

xtSw

itch

ing

Pro

ce

ss

AP

roc

es

s B

Op

era

tin

g S

ys

tem

Sa

ve

Sta

te i

nto

PC

B A

Re

sto

re S

tate

fro

m P

CB

B

Sa

ve

Sta

te i

nto

PC

B B

Re

sto

re S

tate

fro

m P

CB

A

idle

idle

idle

exe

cu

tin

g

exe

cu

tin

g

exe

cu

tin

g

•Process

Context

=m

achin

een

viro

nm

ent

during

the

tim

eth

epr

oce

ssis

active

lyusing

the

CP

U.

•i.e.

conte

xtin

cludes

prog

ram

counte

r,ge

ner

alpurp

ose

regi

ster

s,pr

oce

ssor

stat

us

regi

ster

(withC,N

,Van

dZ

flag

s),...

•To

switch

bet

wee

npr

oce

sses

,th

eO

Sm

ust

:

a)sa

veth

eco

nte

xtof

the

curr

ently

exec

uting

proce

ss(if

any)

,an

db)

rest

ore

the

conte

xtof

that

bei

ng

resu

med

.

•T

ime

take

ndep

ends

on

h/w

suppor

t.

Oper

atin

gSys

tem

s—

Pro

cess

es19

Sch

eduling

Queues

ad

mit

CP

Ure

lea

se

tim

eou

t or

yie

ld

dis

pa

tch

Rea

dy Q

ueu

e

even

t-w

ait

even

t

Wait

Qu

eue(

s)

Job

Qu

eue

crea

te(b

atc

h)

(in

tera

ctiv

e)cr

eate

•Jo

bQ

ueu

e:bat

chpr

oce

sses

awai

ting

adm

ission

.

•R

eady

Queu

e:se

tof

allpr

oce

sses

residin

gin

mai

nm

emor

y,re

ady

toex

ecute

.

•W

ait

Queu

e(s)

:se

tof

proce

sses

wai

ting

for

anI/

Odev

ice

(or

for

other

proce

sses

)

•Lon

g-te

rm&

shor

t-te

rmsc

hed

ule

rs:

–Jo

bsc

hed

ule

rse

lect

sw

hic

hpr

oce

sses

shou

ldbe

brou

ght

into

the

read

yqueu

e.

–C

PU

sched

ule

rdec

ides

whic

hpr

oce

sssh

ould

be

exec

ute

dnex

tan

dal

loca

tes

the

CP

Uto

it.

Oper

atin

gSys

tem

s—

Pro

cess

Life-

cycl

e20

Pro

cess

Cre

ation

•N

early

allsy

stem

sar

ehierarchical:

par

ent

proce

sses

crea

tech

ildre

npr

oce

sses

.

•R

esou

rce

shar

ing:

–par

ent

and

child

ren

shar

eal

lre

sourc

es,or

–ch

ildre

nsh

are

subse

tof

par

ent’

sre

sourc

es,or

–par

ent

and

child

shar

eno

reso

urc

es.

•Exe

cution

:

–par

ent

and

child

ren

exec

ute

concu

rren

tly,

or

–par

ent

wai

tsuntilch

ildre

nte

rmin

ate.

•A

ddre

sssp

ace:

–ch

ildis

duplic

ate

ofpar

ent

or

–ch

ildhas

apr

ogra

mlo

aded

into

it.

•e.

g.on

Unix

:fork()

syst

emca

llcr

eate

sa

new

proce

ss

–al

lre

sourc

essh

ared

(i.e

.ch

ildis

acl

one)

.

–execve()

syst

emca

lluse

dto

repla

cepr

oce

ss’m

emor

yw

ith

anew

prog

ram

.

•N

T/2

K/X

P:CreateProcess()

sysc

allin

cludes

nam

eof

prog

ram

tobe

exec

ute

d.

Oper

atin

gSys

tem

s—

Pro

cess

Life-

cycl

e21

Pro

cess

Term

ination

•P

roce

ssex

ecute

sla

stst

atem

ent

and

asks

the

oper

atin

gsy

stem

todel

ete

it(e

xit)

:

–ou

tput

dat

afr

omch

ildto

par

ent

(wai

t)

–pr

oce

ss’re

sourc

esar

edea

lloca

ted

byth

eO

S.

•P

roce

ssper

form

san

illeg

alop

erat

ion,e.

g.

–m

akes

anat

tem

pt

toac

cess

mem

ory

tow

hic

hit

isnot

auth

orised

,

–at

tem

pts

toex

ecute

apr

ivile

ged

inst

ruct

ion

•Par

ent

may

term

inat

eex

ecution

ofch

ildpr

oce

sses

( abor

t,ki

ll),e.

g.bec

ause

–ch

ildhas

exce

eded

allo

cate

dre

sourc

es

–ta

skas

sign

edto

child

isno

longe

rre

quired

–par

ent

isex

itin

g(“

casc

adin

gte

rmin

atio

n”)

–(m

any

oper

atin

gsy

stem

sdo

not

allo

wa

child

toco

ntinue

ifits

par

ent

term

inat

es)

•e.

g.U

nix

has

wait(),exit()

andkill()

•e.

g.N

T/2

K/X

Phas

ExitProcess()

for

self

term

inat

ion

and

TerminateProcess()

for

killi

ng

other

s.

Oper

atin

gSys

tem

s—

Pro

cess

Life-

cycl

e22

Pro

cess

Blo

ckin

g

•In

gener

ala

proce

ssblo

cks

onan

even

t,e.

g.

–an

I/O

dev

ice

com

ple

tes

anoper

atio

n,

–an

other

proce

ssse

nds

am

essa

ge

•A

ssum

eO

Spr

ovid

esso

me

kind

ofge

ner

al-p

urp

ose

blo

ckin

gpr

imitiv

e,e.

g.await().

•N

eed

care

han

dlin

gconcurren

cyissu

es,e.

g.

if(no

key

being

pressed)

{

await(keypress);

print("Key

has

been

pressed!\n");

} //

handle

keyboard

input

What

hap

pen

sif

ake

yis

pres

sed

atth

efirs

t’{

’?

•(T

his

isabig

area

:lo

tsm

ore

det

ailnex

tye

ar.)

•In

this

cours

ew

e’ll

gener

ally

assu

me

that

prob

lem

sof

this

sort

do

not

arise.

Oper

atin

gSys

tem

s—

Pro

cess

Life-

cycl

e23

CPU

-I/O

Burs

tCyc

le

CP

U B

urs

t D

ura

tio

n (

ms

)

Frequency

24

68

10

12

14

16

•C

PU

-I/O

Burs

tC

ycle

:pr

oce

ssex

ecution

consist

sof

anon

-goi

ngcycle

ofC

PU

exec

ution

,I/

Ow

ait,

CP

Uex

ecution

,...

•P

roce

sses

can

be

des

crib

edas

eith

er:

1.I/

O-b

ound:

spen

ds

mor

etim

edoi

ng

I/O

than

com

puta

tion

;has

man

ysh

ort

CP

Uburs

ts.

2.C

PU

-bound:

spen

ds

mor

etim

edoi

ng

com

puta

tion

s;has

few

very

long

CP

Uburs

ts.

•O

bse

rve

mos

tpr

oce

sses

exec

ute

for

atm

ost

afe

wm

illisec

onds

bef

ore

blo

ckin

g

⇒nee

dm

ultip

rogr

amm

ing

toob

tain

dec

ent

over

allC

PU

utiliz

atio

n.

Oper

atin

gSys

tem

s—

Pro

cess

Life-

cycl

e24

CPU

Sch

edule

r

Rec

all:

CP

Usc

hed

ule

rse

lect

son

eof

the

read

ypr

oce

sses

and

allo

cate

sth

eC

PU

toit.

•T

her

ear

ea

num

ber

ofocc

asio

ns

when

we

can/m

ust

choos

ea

new

proce

ssto

run:

1.a

runnin

gpr

oce

ssblo

cks

(running→

blocked)

2.a

tim

erex

pires

(running→

ready)

3.a

wai

ting

proce

ssunblo

cks

(blocked→

ready)

4.a

proce

sste

rmin

ates

(running→

exit)

•If

only

mak

esc

hed

ulin

gdec

isio

nunder

1,4⇒

hav

eanon-preem

ptive

sched

ule

r:

✔sim

ple

toim

ple

men

t

✘open

toden

ialof

serv

ice

–e.

g.W

indow

s3.

11,ea

rly

Mac

OS.

•O

ther

wise

the

sched

ule

rispreem

ptive

.

✔so

lves

den

ialof

serv

ice

prob

lem

✘m

ore

com

plic

ated

toim

ple

men

t

✘in

troduce

sco

ncu

rren

cypr

oble

ms.

..

Oper

atin

gSys

tem

s—

CP

USch

edulin

g25

Idle

syst

em

What

do

we

do

ifth

ere

isno

read

ypr

oce

ss?

•hal

tpr

oce

ssor

(untilin

terr

upt

arrive

s)

✔sa

ves

pow

er(a

nd

hea

t!)

✔in

crea

ses

proce

ssor

lifet

ime

✘m

ight

take

too

long

tost

opan

dst

art.

•busy

wai

tin

sched

ule

r

✔quic

kre

spon

setim

e

✘ugl

y,use

less

•in

vent

idle

proce

ss,al

way

sav

aila

ble

toru

n

✔gi

ves

unifor

mst

ruct

ure

✔co

uld

use

itto

run

chec

ks

✘use

sso

me

mem

ory

✘ca

nslow

inte

rrupt

resp

onse

Inge

ner

alth

ere

isa

trad

e-off

bet

wee

nre

spon

sive

nes

san

duse

fuln

ess.

Oper

atin

gSys

tem

s—

CP

USch

edulin

g26

Sch

eduling

Crite

ria

Ava

riet

yof

met

rics

may

be

use

d:

1.C

PU

utiliz

atio

n:

the

frac

tion

ofth

etim

eth

eC

PU

isbei

ng

use

d(a

nd

not

for

idle

proce

ss!)

2.T

hro

ugh

put:

#of

proce

sses

that

com

ple

teth

eir

exec

ution

per

tim

eunit.

3.Turn

arou

nd

tim

e:am

ount

oftim

eto

exec

ute

apar

ticu

lar

proce

ss.

4.W

aiting

tim

e:am

ount

oftim

ea

proce

sshas

bee

nw

aiting

inth

ere

ady

queu

e.

5.R

espon

setim

e:am

ount

oftim

eit

take

sfr

omw

hen

are

ques

tw

assu

bm

itte

duntil

the

firs

tre

spon

seis

produce

d(in

tim

e-sh

arin

gsy

stem

s)

Sen

sible

sched

ulin

gst

rate

gies

mig

ht

be:

•M

axim

ize

thro

ugh

put

orC

PU

utiliz

atio

n

•M

inim

ize

aver

age

turn

arou

nd

tim

e,w

aiting

tim

eor

resp

onse

tim

e.

Also

nee

dto

wor

ryab

outfairness

andliveness.

Oper

atin

gSys

tem

s—

CP

USch

edulin

g27

First

-Com

eFirst

-Serv

ed

Sch

eduling

•FC

FS

dep

ends

onor

der

proce

sses

arrive

,e.

g.

Pro

cess

Burs

tT

ime

Pro

cess

Burs

tT

ime

Pro

cess

Burs

tT

ime

P1

25P

24

P3

7

•If

proce

sses

arrive

inth

eor

der

P1,P2,P3:

P 1P 2

P 3

025

29

36

–W

aiting

tim

efo

rP1=

0;P2=

25;P3=

29;

–A

vera

ge

wai

ting

tim

e:(0

+25

+29)/3=

18.

•If

proce

sses

arrive

inth

eor

der

P3,P2,P1:

P 1P 2

P 3

07

11

36

–W

aiting

tim

efo

rP1=

11;P2=

7;P3=

0;

–A

vera

ge

wai

ting

tim

e:(11+7+0)/3=

6.

–i.e.

thre

etim

esas

good!

•First

case

poor

due

toconvoyeff

ect.

Oper

atin

gSys

tem

s—

CP

USch

edulin

g28

SJF

Sch

eduling

Intu

itio

nfr

om

FC

FS

lead

sus

toshortestjobfirst

(SJF

)sc

hed

ulin

g.

•A

ssoci

ate

with

each

proce

ssth

ele

ngt

hof

its

nex

tC

PU

burs

t.

•U

seth

ese

lengt

hs

tosc

hed

ule

the

proce

ssw

ith

the

shor

test

tim

e(F

CFS

can

be

use

dto

brea

kties

).

For

exam

ple

:

Pro

cess

Arr

ival

Tim

eB

urs

tT

ime

P1

07

P2

24

P3

41

P4

54

P 1P 3

P 2

0

P 4

78

12

16

•W

aiting

tim

efo

rP1=

0;P2=

6;P3=

3;P4=

7;

•A

vera

gew

aiting

tim

e:(0

+6+

3+

7)/4=

4.

SJF

isop

tim

alin

the

sense

that

itgiv

esth

em

inim

um

aver

age

wai

ting

tim

efo

ran

ygi

ven

set

ofpr

oce

sses

...

Oper

atin

gSys

tem

s—

CP

USch

edulin

g29

SRT

FSch

eduling

•SR

TF

=Shor

test

Rem

ainin

g-T

ime

First

.

•Ju

sta

pree

mptive

vers

ion

ofSJF

.

•i.e.

ifa

new

proce

ssar

rive

sw

ith

aC

PU

burs

tle

ngt

hle

ssth

anth

eremainingtime

ofth

ecu

rren

tex

ecuting

proce

ss,pr

eem

pt.

For

exam

ple

:

Pro

cess

Arr

ival

Tim

eB

urs

tT

ime

P1

07

P2

24

P3

41

P4

54

P 1P 3

P 2

0

P 4

24

57

11

16

P 2P 1

•W

aiting

tim

efo

rP1=

9;P2=

1;P3=

0;P4=

2;

•A

vera

gew

aiting

tim

e:(9

+1+

0+

2)/4=

3.

What

are

the

prob

lem

sher

e?

Oper

atin

gSys

tem

s—

CP

USch

edulin

g30

Pre

dic

ting

Burs

tLength

s

•For

bot

hSJF

and

SR

TF

require

the

nex

t“b

urs

tle

ngt

h”

for

each

proce

ss⇒

nee

dto

com

eup

with

som

ew

ayto

pred

ict

it.

•C

anbe

don

eby

using

the

lengt

hof

prev

ious

CP

Uburs

tsto

calc

ula

tean

expon

ential

ly-w

eigh

ted

mov

ing

aver

age

(EW

MA

):

1.t n

=ac

tual

lengt

hof

nth

CP

Uburs

t.

2.τ n

+1

=pr

edic

ted

valu

efo

rnex

tC

PU

burs

t.

3.For

α,0≤

α≤

1defi

ne:

τ n+1=

αt n

+(1−

α)τ

n

•If

we

expan

dth

efo

rmula

we

get:

τ n+1=

αt n

+...+

(1−

α)jαt n

−j+...+

(1−

α)n

+1τ 0

wher

eτ 0

isso

me

const

ant.

•C

hoos

eva

lue

ofα

acco

rdin

gto

our

bel

ief

abou

tth

esy

stem

,e.

g.if

we

bel

ieve

histo

ryirre

leva

nt,

choos

eα≈

1an

dth

enget

τ n+1≈

t n.

•In

gener

alan

EW

MA

isa

good

pred

icto

rif

the

varian

ceis

smal

l.

Oper

atin

gSys

tem

s—

CP

USch

edulin

g31

Round

Robin

Sch

eduling

Defi

ne

asm

allfixe

dunit

oftim

eca

lled

aquantum

(ortime-slice),

typic

ally

10-1

00m

illisec

onds.

Then

:

•P

roce

ssat

hea

dof

the

read

yqueu

eis

allo

cate

dth

eC

PU

for

(up

to)

one

quan

tum

.

•W

hen

the

tim

ehas

elap

sed,th

epr

oce

ssis

pree

mpte

dan

dad

ded

toth

eta

ilof

the

read

yqueu

e.

Rou

nd

robin

has

som

enic

epr

oper

ties

:

•Fai

r:if

ther

ear

en

proce

sses

inth

ere

ady

queu

ean

dth

etim

equan

tum

isq,

then

each

proce

ssge

ts1/n

thof

the

CP

U.

•Liv

e:no

proce

ssw

aits

mor

eth

an(n−1)q

tim

eunits

bef

ore

rece

ivin

ga

CP

Ual

loca

tion

.

•Typ

ical

lyge

thig

her

aver

age

turn

arou

nd

tim

eth

anSR

TF,but

bet

ter

aver

age

response

time.

But

tric

kych

oos

ing

corr

ect

size

quan

tum

:

•q

too

larg

e⇒

FC

FS/F

IFO

•q

too

smal

l⇒

conte

xtsw

itch

over

hea

dto

ohig

h.

Oper

atin

gSys

tem

s—

CP

USch

edulin

g32

Sta

tic

Priority

Sch

eduling

•A

ssoci

ate

an(inte

ger)

prio

rity

with

each

proce

ss

•For

exam

ple

:

Prior

ity

Typ

eP

rior

ity

Typ

e

0sy

stem

inte

rnal

proce

sses

2in

tera

ctiv

epr

oce

sses

(stu

den

ts)

1in

tera

ctiv

epr

oce

sses

(sta

ff)

3bat

chpr

oce

sses

.

•T

hen

allo

cate

CP

Uto

the

hig

hes

tpr

iority

proce

ss:

–‘h

ighes

tpr

iority

’ty

pic

ally

mea

ns

smal

lest

inte

ger

–ge

tpr

eem

ptive

and

non

-pre

emptive

varian

ts.

•e.

g.SJF

ispr

iority

sched

ulin

gw

her

epr

iority

isth

epr

edic

ted

nex

tC

PU

burs

ttim

e.

•Pro

ble

m:

how

tore

solv

eties

?

–ro

und

robin

with

tim

e-slic

ing

–al

loca

tequan

tum

toea

chpr

oce

ssin

turn

.

–P

roble

m:

bia

sed

tow

ards

CP

Uin

tensive

jobs.

∗per

-pro

cess

quan

tum

bas

edon

usa

ge?

∗ig

nor

e?

•Pro

ble

m:

star

vation

...

Oper

atin

gSys

tem

s—

CP

USch

edulin

g33

Dyn

am

icPriority

Sch

eduling

•U

sesa

me

sched

ulin

gal

gorith

m,but

allo

wpr

iorities

toch

ange

over

tim

e.

•e.

g.sim

ple

agin

g:

–pr

oce

sses

hav

ea

(sta

tic)

base

priority

and

adyn

amic

effective

priority

.

–if

proce

ssst

arve

dfo

rk

seco

nds,

incr

emen

teff

ective

prio

rity

.

–on

cepr

oce

ssru

ns,

rese

teff

ective

prio

rity

.

•e.

g.co

mpute

dpr

iority

:

–firs

tuse

din

Dijks

tra’

sT

HE

–tim

eslots

:...,t,t+

1,...

–in

each

tim

eslot

t,m

easu

reth

eC

PU

usa

geof

proce

ssj:

uj

–pr

iority

for

proce

ssj

inslot

t+

1:

pj t+

1=

f(u

j t,p

j t,u

j t−1,p

j t−1,...)

–e.

g.pj t+

1=

pj t/2

+kuj t

–pen

alises

CP

Ubou

nd→

suppor

tsI/

Obou

nd.

•to

day

such

com

puta

tion

consider

edac

cepta

ble

...

Oper

atin

gSys

tem

s—

CP

USch

edulin

g34

Mem

ory

Managem

ent

Ina

multip

rogr

amm

ing

syst

em:

•m

any

proce

sses

inm

emor

ysim

ultan

eously,

and

ever

ypr

oce

ssnee

ds

mem

ory

for:

–in

stru

ctio

ns

(“co

de”

or“t

ext”

),

–st

atic

dat

a(in

prog

ram

),an

d

–dyn

amic

dat

a(h

eap

and

stac

k).

•in

additio

n,op

erat

ing

syst

emitse

lfnee

ds

mem

ory

for

inst

ruct

ions

and

dat

a.

⇒m

ust

shar

em

emor

ybet

wee

nO

San

dk

proce

sses

.

The

mem

ory

mag

agem

ent

subsy

stem

han

dle

s:

1.R

eloca

tion

2.A

lloca

tion

3.P

rote

ctio

n

4.Shar

ing

5.Log

ical

Org

anisat

ion

6.P

hys

ical

Org

anisat

ion

Oper

atin

gSys

tem

s—

Mem

ory

Man

agem

ent

35

The

Addre

ssB

indin

gPro

ble

m

Con

sider

the

follo

win

gsim

ple

prog

ram

:

int

x,

y;

x=

5;

y=

x+

3;

We

can

imag

ine

this

wou

ldre

sult

inso

me

asse

mbly

code

whic

hlo

oks

som

ethin

glik

e:

str

#5,

[Rx]

//store

5into

’x’

ldr

R1,

[Rx]

//load

value

of

xfrom

memory

add

R2,

R1,

#3

//and

add

3toit

str

R2,

[Ry]

//and

store

result

in

’y’

wher

eth

eex

pres

sion

‘[addr

]’sh

ould

be

read

tom

ean

“the

conte

nts

of

the

mem

ory

atad

dre

ssaddr”.

Then

the

addre

ssbin

din

gpr

oble

mis:

whatvalues

dowegive

RxandRy

?

This

isa

proble

mbec

ause

we

don

’tkn

oww

her

ein

mem

ory

our

prog

ram

will

be

load

edw

hen

we

run

it:

•e.

g.if

load

edat

0x10

00,th

enx

andy

mig

ht

be

stor

edat

0x20

00,0x

200

4,but

iflo

aded

at0x

5000

,th

enx

andy

mig

ht

be

at0x

600

0,0x

6004

.

Oper

atin

gSys

tem

s—

Rel

oca

tion

36

Addre

ssB

indin

gand

Relo

cation

To

solv

eth

epr

oble

m,w

enee

dto

set

up

som

eki

nd

ofco

rres

pon

den

cebet

wee

n“pr

ogra

mad

dre

sses

”an

d“re

alad

dre

sses

”.T

his

can

be

don

e:

•at

com

pile

tim

e:

–re

quires

know

ledge

ofab

solu

tead

dre

sses

;e.

g.D

OS

.com

file

s

•at

load

tim

e:

–w

hen

prog

ram

load

ed,w

ork

out

pos

itio

nin

mem

ory

and

updat

eev

ery

rele

vant

inst

ruct

ion

inco

de

with

corr

ect

addre

sses

–m

ust

be

don

eev

ery

tim

epr

ogra

mis

load

ed

–ok

for

embed

ded

syst

ems

/boot

-loa

der

s

•at

run-t

ime:

–ge

tso

me

har

dw

are

toau

tom

atic

ally

tran

slat

ebet

wee

npr

ogra

mad

dre

sses

and

real

addre

sses

.

–no

chan

ges

atal

lre

quired

topr

ogra

mitse

lf.

–m

ost

pop

ula

ran

dflex

ible

schem

e,pr

ovid

ing

we

hav

eth

ere

quisite

har

dw

are,

viz.

am

emor

ym

anag

emen

tunit

orM

MU

.

Oper

atin

gSys

tem

s—

Rel

oca

tion

37

Logic

alvs

Phys

icalA

ddre

sses

Map

pin

gof

logi

calto

phys

ical

addre

sses

isdon

eat

run-t

ime

byM

emor

yM

anag

emen

tU

nit

(MM

U),

e.g.

CP

U

ad

dre

ss f

au

ltno

yes

ph

ysic

al

ad

dre

ss

limit

Memory

base

+

log

ica

la

dd

ress

Rel

oca

tio

n R

egis

ter

1.R

eloca

tion

regi

ster

hol

ds

the

valu

eof

the

bas

ead

dre

ssow

ned

byth

epr

oce

ss.

2.R

eloca

tion

regi

ster

conte

nts

are

added

toea

chm

emor

yad

dre

ssbef

ore

itis

sent

tom

emor

y.

3.e.

g.D

OS

on80

x86

—4

relo

cation

regi

ster

s,lo

gic

alad

dre

ssis

atu

ple

(s,o).

4.N

B:pr

oce

ssnev

erse

esphys

ical

addre

ss—

sim

ply

man

ipula

tes

logi

calad

dre

sses

.

5.O

Shas

priv

ilege

toupdat

ere

loca

tion

regi

ster

.

Oper

atin

gSys

tem

s—

Rel

oca

tion

38

Contiguous

Alloca

tion

Giv

enth

atw

ew

ant

multip

levi

rtual

proce

ssor

s,how

can

we

suppor

tth

isin

asingl

ead

dre

sssp

ace?

Wher

edo

we

put

proce

sses

inm

emor

y?

•O

Sty

pic

ally

must

be

inlo

wm

emor

ydue

tolo

cation

ofin

terr

upt

vect

ors

•Eas

iest

way

isto

stat

ical

lydiv

ide

mem

ory

into

multip

lefixe

dsize

par

tition

s:

–ea

chpar

tition

span

sa

contigu

ous

range

ofphys

ical

mem

ory

–bot

tom

par

tition

conta

ins

OS,re

mai

nin

gpar

tition

sea

chco

nta

inex

actly

one

proce

ss.

–w

hen

apr

oce

sste

rmin

ates

its

par

tition

bec

omes

avai

lable

tonew

proce

sses

.

–e.

g.O

S/3

60M

FT

.

•N

eed

topr

otec

tO

San

duse

rpr

oce

sses

from

mal

icio

us

prog

ram

s:

–use

bas

ean

dlim

itre

gist

ers

inM

MU

–updat

eva

lues

when

anew

proce

sses

issc

hed

ule

d

–N

B:so

lvin

gbot

hre

loca

tion

and

prot

ection

proble

ms

atth

esa

me

tim

e!

Oper

atin

gSys

tem

s—

Con

tigu

ous

Allo

cation

39

Sta

tic

Multip

rogra

mm

ing

Pa

rtit

ion

edM

emo

ryRun

Queue

Blocked

Queue

A B C D

Backing

Store

Main

Store

OS

•par

tition

mem

ory

when

inst

allin

gO

S,an

dal

loca

tepie

ces

todiff

eren

tjo

bqueu

es.

•as

soci

ate

jobs

toa

job

queu

eac

cord

ing

tosize

.

•sw

apjo

bbac

kto

disk

when

:

–blo

cked

onI/

O(a

ssum

ing

I/O

isslow

erth

anth

ebac

king

stor

e).

–tim

eslic

ed:

larg

erth

ejo

b,la

rger

the

tim

eslic

e

•ru

njo

bfr

oman

other

queu

ew

hile

swap

pin

gjo

bs

•e.

g.IB

MO

S/3

60M

FT

,IC

LSys

tem

4

•pro

ble

ms:

frag

men

tation

(par

tition

too

big

),ca

nnot

grow

(par

tition

too

smal

l).

Oper

atin

gSys

tem

s—

Con

tigu

ous

Allo

cation

40

Dyn

am

icPar

titionin

g

Get

mor

eflex

ibili

tyif

allo

wpar

tition

size

sto

be

dyn

amic

ally

chos

en,e.

g.O

S/3

60M

VT

(“M

ultip

leV

aria

ble

-siz

edTas

ks”)

:

•O

Ske

eps

trac

kof

whic

har

eas

ofm

emor

yar

eav

aila

ble

and

whic

har

eocc

upie

d.

•e.

g.use

one

orm

orelinkedlists:

0000

0C04

2200

3810

4790

91E8

B0F0

B130

D708

FFFF

•W

hen

anew

proce

ssar

rive

sin

toth

esy

stem

,th

eO

Sse

arch

esfo

ra

hol

ela

rge

enou

ghto

fit

the

proce

ss.

•Som

eal

gorith

ms

todet

erm

ine

whic

hhol

eto

use

for

new

proce

ss:

–firs

tfit:

stop

sear

chin

glis

tas

soon

asbig

enou

ghhol

eis

found.

–best

fit:

sear

chen

tire

list

tofind

“bes

t”fitt

ing

hol

e(i.e

.sm

alle

sthol

ew

hic

his

larg

een

ough

)

–wors

tfit:

counte

rintu

itiv

ely

allo

cate

larg

est

hol

e(a

gain

must

sear

chen

tire

list)

.

•W

hen

proce

sste

rmin

ates

its

mem

ory

retu

rns

onto

the

free

list,

coal

esci

ng

hol

esto

geth

erw

her

eap

prop

riat

e.

Oper

atin

gSys

tem

s—

Con

tigu

ous

Allo

cation

41

Sch

eduling

Exa

mple

0

400K

1000K

2000K

2300K

2560K

OSP1

P2

P3

OSP1

P3

OSP1

P4

P3

OSP3

OSP5

P3

P4

P4

0

400K

1000K

2000K

2300K

2560K

1700K

0

400K

1000K

2000K

2300K

2560K

1700K

900K

•C

onsider

mac

hin

ew

ith

tota

lof

2560K

mem

ory,

wher

eO

Sre

quires

400K.

•T

he

follo

win

gjo

bs

are

inth

equeu

e:

Pro

cess

Mem

ory

Req

dTot

alExe

cution

Tim

eP1

600K

10P2

1000

K5

P3

300K

20P4

700K

8P5

500K

15

Oper

atin

gSys

tem

s—

Con

tigu

ous

Allo

cation

42

Ext

ern

alFra

gm

enta

tion

OSP1

P2

P3

OSP1

P3

OSP1

P4

P3

OSP3

P4

P4

P5

P6

OSP5

P3

P4

OSP5

P3

P4

•D

ynam

icpar

tition

ing

algo

rith

ms

suffer

from

exte

rnal

frag

men

tation

:as

proce

sses

are

load

edth

eyle

ave

little

frag

men

tsw

hic

hm

aynot

be

use

d.

•Ext

ernal

frag

men

tation

exists

when

the

tota

lav

aila

ble

mem

ory

issu

ffici

ent

for

are

ques

t,but

isunusa

ble

bec

ause

itis

split

into

man

yhol

es.

•C

anal

sohav

epr

oble

ms

with

tiny

hol

es

Sol

ution

:co

mpac

thol

esper

iodic

ally.

Oper

atin

gSys

tem

s—

Con

tigu

ous

Allo

cation

43

Com

pact

ion

0

300K

1000K

1500K

1900K

2100K

OSP1

P3

P4

500K

600K

P2

1200K

400K

300K

200K

0

300K

800K

2100K

OSP1

P3

P4

500K

600K

P2

1200K

900K

0

300K

1000K

2100K

OSP1

P4

P3

500K

600K

P2

1200K

900K

0

300K

2100K

OSP1

P4

P3

500K

600K

P2

1500K

900K

1900K

Choos

ing

optim

alst

rate

gyquite

tric

ky...

Not

eth

at:

•W

ere

quire

run-t

ime

relo

cation

for

this

tow

ork.

•C

anbe

done

mor

eeffi

cien

tly

when

proce

ssis

move

din

tom

emor

yfr

oma

swap

.

•Som

em

achin

esuse

dto

hav

ehar

dw

are

suppor

t(e

.g.C

DC

Cyb

er).

Also

get

frag

men

tation

inbackingstore

,but

inth

isca

seco

mpac

tion

not

real

lya

viab

leop

tion

...

Oper

atin

gSys

tem

s—

Con

tigu

ous

Allo

cation

44

Paged

Virtu

alM

em

ory

CP

U

Memory

logical address

physical

address

p

f

Page

Table

po

fo

1

Anot

her

solu

tion

isto

allo

wa

proce

ssto

exist

innon

-con

tiguou

sm

emor

y,i.e.

•div

ide

phys

ical

mem

ory

into

rela

tive

lysm

allblo

cks

offixe

dsize

,ca

lled

fram

es

•div

ide

logi

calm

emor

yin

toblo

cks

ofth

esa

me

size

calle

dpag

es

•(t

ypic

alpag

esize

sar

ebet

wee

n51

2byt

esan

d8K

)

•ea

chad

dre

ssge

ner

ated

byC

PU

com

prises

apag

enum

ber

pan

dpag

eoff

seto.

•M

MU

use

sp

asan

index

into

apag

eta

ble

.

•pag

eta

ble

conta

ins

asso

ciat

edfr

ame

num

ber

f

•usu

ally

hav

e|p|>>|f|⇒

nee

dva

lidbit

Oper

atin

gSys

tem

s—

Pag

ing

45

Pagin

gPro

sand

Cons

Page 0

Page 0

Page 1

Page 2

Page n-1

Page 3

Page 4

Page 1

Page 4

Page 3

0 1 2 3 4 5 6 7 8

1 1 0 1 1 0

4 6 2 1

Vir

tua

l M

emo

ry

Ph

ysic

al

Mem

ory

✔m

emor

yal

loca

tion

easier

.

✘O

Sm

ust

keep

pag

eta

ble

per

proce

ss

✔no

exte

rnal

frag

men

tation

(in

phys

ical

mem

ory

atle

ast)

.

✘but

getin

tern

alfragm

enta

tion.

✔cl

ear

separ

atio

nbet

wee

nuse

ran

dsy

stem

view

ofm

emor

yusa

ge.

✘ad

ditio

nal

over

hea

don

conte

xtsw

itch

ing

Oper

atin

gSys

tem

s—

Pag

ing

46

Str

uct

ure

ofth

ePage

Table

Diff

eren

tki

nds

ofhar

dw

are

suppor

tca

nbe

prov

ided

:

•Sim

ple

stca

se:

set

ofded

icat

edre

loca

tion

regi

ster

s

–on

ere

giste

rper

pag

e

–O

Slo

ads

the

regi

ster

son

conte

xtsw

itch

–fine

ifth

epag

eta

ble

issm

all.

..but

what

ifhav

ela

rge

num

ber

of

pag

es?

•A

lter

nat

ivel

yke

eppag

eta

ble

inm

emor

y

–on

lyon

ere

gist

ernee

ded

inM

MU

(pag

eta

ble

bas

ere

gist

er(P

TB

R))

–O

Ssw

itch

esth

isw

hen

switch

ing

proce

ss

•Pro

ble

m:

pag

eta

ble

sm

ight

still

be

very

big

.

–ca

nke

epa

pag

eta

ble

lengt

hre

gist

er(P

TLR

)to

indic

ate

size

ofpag

eta

ble

.

–or

can

use

mor

eco

mple

xst

ruct

ure

(see

late

r)

•Pro

ble

m:

nee

dto

refe

rto

mem

orytwice

for

ever

y‘a

ctual

’m

emor

yre

fere

nce

...

⇒use

atr

anslat

ion

look

asid

ebuffer

(TLB

)

Oper

atin

gSys

tem

s—

Pag

ing

47

TLB

Opera

tion

CP

U

Memory

logical address

physical address

p

po

fo

f

Page

Table

1

TL

B

p1p2p3p4

f1f2f3f4

•O

nm

emor

yre

fere

nce

pres

ent

TLB

with

logi

calm

emor

yad

dre

ss

•If

pag

eta

ble

entr

yfo

rth

epag

eis

pres

ent

then

get

anim

med

iate

resu

lt

•If

not

then

mak

em

emor

yre

fere

nce

topag

eta

ble

s,an

dupdat

eth

eT

LB

Oper

atin

gSys

tem

s—

Pag

ing

48

TLB

Issu

es

•U

pdat

ing

TLB

tric

kyif

itis

full:

nee

dto

disca

rdso

met

hin

g.

•C

onte

xtsw

itch

may

requires

TLB

flush

soth

atnex

tpr

oce

ssdoes

n’t

use

wro

ng

pag

eta

ble

entr

ies.

–Today

man

yT

LB

ssu

ppor

tpr

oce

ssta

gs

(som

etim

esca

lled

addre

ss

spac

enum

ber

s )to

impr

ove

per

form

ance

.

•H

itra

tio

isth

eper

centa

ge

oftim

ea

pag

een

try

isfo

und

inT

LB

•e.

g.co

nsider

TLB

sear

chtim

eof

20ns,

mem

ory

acce

sstim

eof

100ns,

and

ahit

ratio

of

80%

⇒as

sum

ing

one

mem

ory

refe

rence

required

for

pag

eta

ble

look

up,th

e

effective

mem

ory

acce

sstim

eis0.8×120+0.2×220=

140n

s.

•In

crea

sehit

ratio

to98%

give

seff

ective

acce

sstim

eof

122ns

—only

a13

%im

prov

emen

t.

Oper

atin

gSys

tem

s—

Pag

ing

49

Multileve

lPage

Table

s

•M

ost

moder

nsy

stem

sca

nsu

ppor

tve

ryla

rge

(232,2

64)

addre

sssp

aces

.

•Sol

ution

–sp

litpag

eta

ble

into

seve

ralsu

b-p

arts

•T

wo

leve

lpag

ing

–pag

eth

epag

eta

ble

P1

Offset

Vir

tual A

dd

ress

L2 Address

L1 P

ag

e T

ab

le0 n N

P2

L1 Address

Base R

eg

iste

r

L2 P

ag

e T

ab

le0 n N

Leaf PTE

•For

64bit

arch

itec

ture

sa

two-

leve

lpag

ing

schem

eis

not

suffi

cien

t:nee

dfu

rther

leve

ls(u

sual

ly4,

orev

en5).

•(e

ven

som

e32

bit

mac

hin

eshav

e>

2le

vels,e.

g.

x86

PA

Em

ode)

.

Oper

atin

gSys

tem

s—

Pag

ing

50

Exa

mple

:x8

6

PTA

V DR W

U SW T

C DA C

Z OP S

IGN

Page

Dir

ecto

ry (

Lev

el 1

)

10

24

entr

ies

L1

L2

Offset

Vir

tua

l A

dd

ress 20

bit

s

•Pag

esize

4K(o

r4M

b).

•First

look

up

isin

thepage

directory

:in

dex

using

mos

t10

sign

ifica

nt

bits.

•A

ddre

ssof

pag

edirec

tory

stor

edin

inte

rnal

proce

ssor

regi

ster

(cr3).

•R

esults

(nor

mal

ly)

inth

ead

dre

ssof

apage

table.

Oper

atin

gSys

tem

s—

Pag

ing

51

Exa

mple

:x8

6(2

)

PFA

V DR W

U SW T

C DA C

D YZ O

IGN

Page

Table

(L

evel

2)

10

24

entr

ies

G L

L1

L2

Offset

Vir

tua

l A

dd

ress

20

bit

s

•U

senex

t10

bits

toin

dex

into

pag

eta

ble

.

•O

nce

retr

ieve

pag

efr

ame

addre

ss,ad

din

the

offse

t(i.e

.th

elo

w12

bits)

.

•N

otic

epag

edirec

tory

and

pag

eta

ble

sar

eex

actly

one

pag

eea

chth

emse

lves

.

Oper

atin

gSys

tem

s—

Pag

ing

52

Pro

tect

ion

Issu

es

•A

ssoci

ate

prot

ection

bits

with

each

pag

e–

kept

inpag

eta

ble

s(a

nd

TLB

).

•e.

g.on

ebit

for

read

,on

efo

rw

rite

,on

efo

rex

ecute

.

•M

ayal

sodistingu

ish

whet

her

apag

em

ayonly

be

acce

ssed

when

exec

uting

inkernel

mode,

e.g.

apag

e-ta

ble

entr

ym

aylo

oklik

e:

Frame Number

VX

WR

K

•A

tth

esa

me

tim

eas

addre

ssis

goin

gth

rough

pag

etr

anslat

ion

har

dw

are,

can

chec

kpr

otec

tion

bits.

•A

ttem

pt

tovi

olat

epr

otec

tion

cause

sh/w

trap

toop

erat

ing

syst

emco

de

•A

sbef

ore,

hav

evalid/invalid

bit

det

erm

inin

gif

the

pag

eis

map

ped

into

the

proce

ssad

dre

sssp

ace:

–if

inva

lid⇒

trap

toO

Shan

dle

r

–ca

ndo

lots

ofin

tere

stin

gth

ings

her

e,par

ticu

larly

with

rega

rdto

shar

ing.

..

Oper

atin

gSys

tem

s—

Pag

ing

53

Shar

ed

Pages

Anot

her

adva

nta

geof

pag

edm

emor

yis

code/

dat

ash

arin

g,fo

rex

ample

:

•bin

arie

s:ed

itor

,co

mpile

ret

c.

•lib

raries

:sh

ared

obje

cts,

dlls

.

So

how

does

this

wor

k?

•Im

ple

men

ted

astw

olo

gica

lad

dre

sses

whic

hm

apto

one

phys

ical

addre

ss.

•If

code

isre-entrant

(i.e

.st

atel

ess,

non

-sel

fm

odifyi

ng)

itca

nbe

easily

shar

edbet

wee

nuse

rs.

•O

ther

wise

can

use

copy

-on-w

rite

tech

niq

ue:

–m

ark

pag

eas

read

-only

inal

lpr

oce

sses

.

–if

apr

oce

sstr

ies

tow

rite

topag

e,w

illtr

apto

OS

fault

han

dle

r.

–ca

nth

enal

loca

tenew

fram

e,co

pydat

a,an

dcr

eate

new

pag

eta

ble

map

pin

g.

•(m

ayuse

this

for

lazy

dat

ash

arin

gto

o).

Req

uires

additio

nal

book

-kee

pin

gin

OS,but

wor

thit,e.

g.ov

er10

0MB

of

shar

edco

de

onm

ylin

ux

box

.

Oper

atin

gSys

tem

s—

Pag

ing

54

Virtu

alM

em

ory

•V

irtu

alad

dre

ssin

gal

low

sus

toin

troduce

the

idea

ofvi

rtual

mem

ory:

–al

read

yhav

eva

lidor

inva

lidpag

es;in

troduce

anew

“ non

-res

iden

t”des

ignat

ion

–su

chpag

esliv

eon

anon

-vol

atile

bac

king

stor

e,su

chas

ahar

d-d

isk.

–pr

oce

sses

acce

ssnon

-res

iden

tm

emor

yju

stas

ifit

wer

e‘t

he

real

thin

g’.

•V

irtu

alm

emor

y(V

M)

has

anum

ber

ofben

efits

:

–por

tabili

ty:

progr

ams

wor

kre

gar

dle

ssof

how

much

actu

alm

emor

ypr

esen

t

–co

nve

nie

nce

:pr

ogra

mm

erca

nuse

e.g.

larg

esp

arse

dat

ast

ruct

ure

sw

ith

impunity

–effi

cien

cy:

no

nee

dto

was

te(r

eal)

mem

ory

onco

de

ordat

aw

hic

hisn’t

use

d.

•V

Mty

pic

ally

imple

men

ted

via

dem

and

pag

ing:

–pr

ogra

ms

(exe

cuta

ble

s)re

side

ondisk

–to

exec

ute

apr

oce

ssw

elo

adpag

esin

ondem

and;i.e.

asan

dw

hen

they

are

refe

rence

d.

•A

lso

getdem

andsegm

entation,but

rare

.

Oper

atin

gSys

tem

s—

Dem

and

Pag

edV

irtu

alM

emor

y55

Dem

and

Pagin

gD

eta

ils

When

load

ing

anew

proce

ssfo

rex

ecution

:

•w

ecr

eate

its

addre

sssp

ace

(e.g

.pag

eta

ble

s,et

c),but

mar

kal

lP

TEs

asei

ther

“inva

lid”

or“n

on-r

esid

ent”

;an

dth

en

•ad

dits

proce

ssco

ntr

olblo

ck(P

CB

)to

the

read

y-queu

e.

Then

when

ever

we

rece

ive

apag

efa

ult:

1.ch

eck

PT

Eto

det

erm

ine

if“i

nva

lid”

ornot

2.if

anin

valid

refe

rence⇒

kill

proce

ss;

3.ot

her

wise

‘pag

ein

’th

edes

ired

pag

e:

•find

afr

eefr

ame

inm

emor

y

•in

itia

tedisk

I/O

tore

adin

the

des

ired

pag

ein

toth

enew

fram

e

•w

hen

I/O

isfinished

modify

the

PT

Efo

rth

ispag

eto

show

that

itis

now

valid

•re

star

tth

epr

oce

ssat

the

faultin

gin

stru

ctio

n

Sch

eme

des

crib

edab

ove

ispure

dem

and

pag

ing:

•nev

erbr

ings

ina

pag

euntilre

quired⇒

get

lots

ofpag

efa

ults

and

I/O

when

the

proce

ssfirs

tbeg

ins.

•hen

cem

any

real

syst

ems

explic

itly

load

som

eco

repar

tsof

the

proce

ssfirs

t

Oper

atin

gSys

tem

s—

Dem

and

Pag

edV

irtu

alM

emor

y56

Page

Repla

cem

ent

•W

hen

pag

ing

infr

omdisk,

we

nee

da

free

fram

eof

phys

ical

mem

ory

tohol

dth

edat

aw

e’re

read

ing

in.

•In

real

ity,

size

ofphys

ical

mem

ory

islim

ited⇒

–nee

dto

disca

rdunuse

dpag

esif

tota

ldem

and

exce

eds

phys

ical

mem

ory

size

–(a

lter

nat

ivel

yco

uld

swap

out

aw

hol

epr

oce

ssto

free

som

efr

ames

)

•M

odifi

edal

gorith

m:

ona

pag

efa

ult

we

1.lo

cate

the

des

ired

repla

cem

ent

pag

eon

disk

2.to

sele

cta

free

fram

efo

rth

ein

com

ing

pag

e:

(a)

ifth

ere

isa

free

fram

euse

it(b

)ot

her

wise

sele

cta

vict

impag

eto

free

,(c

)w

rite

the

vict

impag

ebac

kto

disk,

and

(d)

mar

kit

asin

valid

inits

proce

sspag

eta

ble

s

3.re

addes

ired

pag

ein

tofr

eed

fram

e

4.re

star

tth

efa

ultin

gpr

oce

ss

•C

anre

duce

over

hea

dby

addin

ga

dirty

bit

toP

TEs

(can

pot

ential

lyom

itst

ep2c

)

•Q

uest

ion:

how

do

we

choos

eou

rvi

ctim

pag

e?

Oper

atin

gSys

tem

s—

Dem

and

Pag

edV

irtu

alM

emor

y57

Page

Repla

cem

ent

Alg

orith

ms

•First

-In

First

-Out

(FIF

O)

–ke

epa

queu

eof

pag

es,disca

rdfr

omhea

d

–per

form

ance

diffi

cult

topr

edic

t:hav

eno

idea

whet

her

pag

ere

pla

ced

will

be

use

dag

ain

ornot

–disca

rdis

indep

enden

tof

pag

euse

freq

uen

cy

–in

gener

al:

pret

tybad

,al

thou

ghve

rysim

ple

.

•O

ptim

alA

lgor

ithm

(OP

T)

–re

pla

ceth

epag

ew

hic

hw

illnot

be

use

dag

ain

for

longe

stper

iod

oftim

e

–ca

non

lybe

don

ew

ith

anor

acle

,or

inhin

dsight

–se

rves

asa

good

com

par

ison

for

other

algo

rith

ms

•Lea

stR

ecen

tly

Use

d(L

RU

)

–LR

Ure

pla

ces

the

pag

ew

hic

hhas

not

bee

nuse

dfo

rth

elo

nge

stam

ount

oftim

e

–(i.e

.LR

Uis

OP

Tw

ith

-ve

tim

e)

–as

sum

espas

tis

ago

od

pred

icto

rof

the

futu

re

–Q

uest

ion:

how

do

we

det

erm

ine

the

LR

Uor

der

ing?

Oper

atin

gSys

tem

s—

Pag

eR

epla

cem

ent

Alg

orithm

s58

Imple

menting

LRU

•C

ould

try

using

counte

rs

–gi

veea

chpag

eta

ble

entr

ya

tim

e-of

-use

fiel

dan

dgi

veC

PU

alo

gica

lcl

ock

(e.g

.an

n-b

itco

unte

r)

–w

hen

ever

apag

eis

refe

rence

d,its

PT

Eis

updat

edto

clock

valu

e

–re

pla

cepag

ew

ith

smal

lest

tim

eva

lue

–pro

ble

m:

requires

ase

arch

tofind

min

imum

valu

e

–pro

ble

m:

adds

aw

rite

tom

emor

y(P

TE)

onev

ery

mem

ory

refe

rence

–pro

ble

m:

clock

over

flow

...

•O

ra

pag

est

ack:

–m

ainta

inastack

ofpag

es(a

dou

bly

-lin

ked

list)

–updat

est

ack

onev

ery

refe

rence

toen

sure

new

(MR

U))

pag

eon

top

–disca

rdfr

ombot

tom

ofst

ack

–pro

ble

m:

requires

chan

ging

6poi

nte

rsper

[new

]re

fere

nce

–pos

sible

with

h/w

suppor

t,but

slow

even

then

(and

extr

emel

yslow

withou

tit!)

•N

eith

ersc

hem

ese

ems

prac

tica

lon

ast

andar

dpr

oce

ssor⇒

nee

dan

other

way

.

Oper

atin

gSys

tem

s—

Pag

eR

epla

cem

ent

Alg

orithm

s59

Appro

xim

ating

LRU

(1)

•M

any

syst

ems

hav

ea

refe

rence

bit

inth

eP

TE

whic

his

set

byh/w

when

ever

the

pag

eis

touch

ed

•T

his

allo

ws

not

rece

ntly

use

d(N

RU

)re

pla

cem

ent:

–per

iodic

ally

(e.g

.20

ms)

clea

ral

lre

fere

nce

bits

–w

hen

choos

ing

avi

ctim

tore

pla

ce,pr

efer

pag

esw

ith

clea

rre

fere

nce

bits

–if

we

also

hav

ea

modifi

edbit

(or

dirty

bit)

inth

eP

TE,w

eca

nex

tend

NR

Uto

use

that

too:

Ref

?D

irty

?C

omm

ent

no

no

bes

tty

pe

ofpag

eto

repla

ceno

yes

nex

tbes

t(r

equires

write

bac

k)ye

sno

prob

ably

code

inuse

yes

yes

bad

choi

cefo

rre

pla

cem

ent

•O

rca

nex

tend

bym

ainta

inin

gm

ore

histo

ry,e.

g.

–fo

rea

chpag

e,th

eop

erat

ing

syst

emm

ainta

ins

an8-

bit

valu

e,in

itia

lized

toze

ro

–per

iodic

ally

(e.g

.ev

ery

20m

s),sh

ift

the

refe

rence

bit

onto

mos

t-sign

ifica

nt

bit

ofth

eby

te,an

dcl

ear

the

refe

rence

bit

–se

lect

low

est

valu

epag

e(o

ron

eof

them

)to

repla

ce

Oper

atin

gSys

tem

s—

Pag

eR

epla

cem

ent

Alg

orithm

s60

Appro

xim

ating

LRU

(2)

•Pop

ula

rN

RU

schem

e:se

cond-c

han

ceFIF

O

–st

ore

pag

esin

queu

eas

per

FIF

O–

bef

ore

disca

rdin

ghea

d,ch

eck

its

refe

rence

bit

–if

refe

rence

bit

is0,th

endisca

rdit,ot

her

wise:

∗re

set

refe

rence

bit,an

dad

dpag

eto

tail

ofqueu

e∗

i.e.

give

it“a

seco

nd

chan

ce”

•O

ften

imple

men

ted

with

circ

ula

rqueu

ean

dhea

dpoi

nte

r:th

enca

lled

clock

.

•If

no

h/w

prov

ided

refe

rence

bit

can

emula

te:

–to

clea

r“r

efer

ence

bit”,

mar

kpag

eno

acce

ss–

ifre

fere

nce

d⇒

trap

,updat

eP

TE,an

dre

sum

e–

toch

eck

ifre

fere

nce

d,ch

eck

per

mission

s–

can

use

sim

ilar

schem

eto

emula

tem

odifi

edbit

Oper

atin

gSys

tem

s—

Pag

eR

epla

cem

ent

Alg

orithm

s61

Oth

er

Repla

cem

ent

Sch

em

es

•C

ounting

Alg

orithm

s:ke

epa

count

ofth

enum

ber

ofre

fere

nce

sto

each

pag

e

–LFU

:re

pla

cepag

ew

ith

smal

lest

count

–M

FU

:re

pla

cehig

hes

tco

unt

bec

ause

low

count⇒

mos

tre

cently

brou

ght

in.

•Pag

eB

uffer

ing

Alg

orithm

s:

–ke

epa

min

.num

ber

ofvi

ctim

sin

afr

eepool

–new

pag

ere

adin

bef

ore

writing

out

vict

im.

•(P

seudo)

MR

U:

–co

nsider

acce

ssof

e.g.

larg

ear

ray.

–pag

eto

repla

ceis

one

applic

atio

nhas

just

finished

with,i.e.

mos

tre

cently

use

d.

–e.

g.tr

ack

pag

efa

ults

and

look

for

sequen

ces.

–disca

rdth

ekth

invi

ctim

sequen

ce.

•A

pplic

atio

n-s

pec

ific:

–st

optr

ying

tose

cond

gues

sw

hat

’sgo

ing

on.

–pr

ovid

ehook

for

app.to

sugg

est

repla

cem

ent.

–m

ust

be

care

fulw

ith

den

ialof

serv

ice.

..

Oper

atin

gSys

tem

s—

Pag

eR

epla

cem

ent

Alg

orithm

s62

Perform

ance

Com

par

ison

FIF

O

CLO

CK

LR

U

OP

T

Page Faults per 1000 References

5

10

15

20

25

30

35

40

45 0

56

78

910

Num

ber

of P

age F

ram

es A

vaila

ble

11

12

13

14

15

Gra

ph

plo

tspag

e-fa

ult

rate

agai

nst

num

ber

ofphys

ical

fram

esfo

ra

pse

udo-

loca

lre

fere

nce

string.

•w

ant

tom

inim

ise

area

under

curv

e

•FIF

Oca

nex

hib

itB

elad

y’s

anom

aly

(althou

ghit

does

n’t

inth

isca

se)

•ge

ttin

gfr

ame

allo

cation

righ

thas

maj

orim

pac

t...

Oper

atin

gSys

tem

s—

Pag

eR

epla

cem

ent

Alg

orithm

s63

Fra

me

Alloca

tion

•A

cert

ain

frac

tion

ofphys

ical

mem

ory

isre

serv

edper

-pro

cess

and

for

core

oper

atin

gsy

stem

code

and

dat

a.

•N

eed

anallocationpolicy

todet

erm

ine

how

todistr

ibute

the

rem

ainin

gfr

ames

.

•O

bje

ctiv

es:

–Fai

rnes

s(o

rpr

opor

tion

alfa

irnes

s)?

∗e.

g.div

idem

fram

esbet

wee

nn

proce

sses

asm/n

,w

ith

any

rem

ainder

stay

ing

inth

efr

eepool

∗e.

g.div

ide

fram

esin

prop

ortion

tosize

ofpr

oce

ss(i.e

.num

ber

ofpag

esuse

d)

–M

inim

ize

syst

em-w

ide

pag

e-fa

ult

rate

?(e

.g.al

loca

teal

lm

emor

yto

few

proce

sses

)

–M

axim

ize

leve

lof

multip

rogr

amm

ing?

(e.g

.al

loca

tem

inm

emor

yto

man

ypr

oce

sses

)

•M

ost

pag

ere

pla

cem

ent

schem

esar

eglobal:

allpag

esco

nsider

edfo

rre

pla

cem

ent.

⇒al

loca

tion

pol

icy

implic

itly

enfo

rced

during

pag

e-in

:

–al

loca

tion

succ

eeds

iffpol

icy

agre

es

–‘fre

efr

ames

’of

ten

inuse⇒

stea

lth

em!

Oper

atin

gSys

tem

s—

Fra

me

Allo

cation

64

The

Risk

ofT

hra

shin

g

CPU utilisation

De

gre

e o

f M

ult

ipro

gra

mm

ing

thra

sh

ing

•A

sm

ore

and

mor

epr

oce

sses

ente

rth

esy

stem

(multi-pr

ogra

mm

ing

leve

l(M

PL)

incr

ease

s),th

efr

ames

-per

-pro

cess

valu

eca

nge

tve

rysm

all.

•A

tso

me

poi

nt

we

hit

aw

all:

–a

proce

ssnee

ds

mor

efr

ames

,so

stea

lsth

em

–but

the

other

proce

sses

nee

dth

ose

pag

es,so

they

fault

tobr

ing

them

bac

kin

–num

ber

of

runnab

lepr

oce

sses

plu

nges

•To

avoi

dth

rash

ing

we

must

give

proce

sses

asm

any

fram

esas

they

“nee

d”

•If

we

can’t

,w

enee

dto

reduce

the

MP

L:bett

er

page-r

epla

cem

ent

won’t

help

!

Oper

atin

gSys

tem

s—

Fra

me

Allo

cation

65

Loca

lity

ofRefe

rence

0x10000

0x20000

0x30000

0x40000

0x50000

0x60000

0x70000

0x80000

0x90000

0xa0000

0xb0000

0xc0000

010000

20000

30000

40000

50000

60000

70000

80000

Miss address

Mis

s n

um

ber

Ex

ten

ded

Mal

loc

Init

ial

Mal

loc

I/O

Bu

ffer

s

Use

r d

ata/

bss

Use

r co

de

Use

r S

tack

VM

wo

rksp

ace

Ker

nel

dat

a/b

ss

Ker

nel

co

de

Pars

eO

pti

mis

eO

utp

ut

Kern

el

Init

move

ima

ge

clea

r

bss

Tim

er

IRQ

sco

nn

ect

or

da

em

on

Loca

lity

ofre

fere

nce

:in

ash

ort

tim

ein

terv

al,th

elo

cation

sre

fere

nce

dby

apr

oce

sste

nd

tobe

gro

uped

into

afe

wre

gion

sin

its

addre

sssp

ace.

•pr

oce

dure

bei

ng

exec

ute

d

•...su

b-p

roce

dure

s

•...dat

aac

cess

•...st

ack

variab

les

Note

:hav

elo

calit

yin

bot

hsp

ace

and

tim

e.

Oper

atin

gSys

tem

s—

Fra

me

Allo

cation

66

Avo

idin

gT

hra

shin

gW

eca

nuse

the

loca

lity

ofre

fere

nce

prin

ciple

tohel

pdet

erm

ine

how

man

yfr

ames

apr

oce

ssnee

ds:

•defi

ne

the

Wor

king

Set

(Den

nin

g,19

67)

–se

tof

pag

esth

ata

proce

ssnee

ds

tobe

residen

t“t

he

sam

etim

e”to

mak

ean

y(r

easo

nab

le)

prog

ress

–va

ries

bet

wee

npr

oce

sses

and

during

exec

ution

–as

sum

epr

oce

ssm

ove

sth

rough

phases:

∗in

each

phas

e,ge

t(s

pat

ial)

loca

lity

ofre

fere

nce

∗fr

omtim

eto

tim

ege

tphase

shift

•O

Sca

ntr

yto

prev

ent

thra

shin

gby

ensu

ring

suffi

cien

tpag

esfo

rcu

rren

tphas

e:

–sa

mple

pag

ere

fere

nce

bits

ever

ye.

g.

10m

s

–if

apag

eis

“in

use

”,sa

yit’s

inth

ew

orki

ng

set

–su

mw

orki

ng

set

size

sto

get

tota

ldem

andD

–ifD

>m

we

are

indan

ger

ofth

rash

ing⇒

susp

end

apr

oce

ss

•A

lter

nat

ivel

yuse

pag

efa

ult

freq

uen

cy(P

FF):

–m

onitor

per

-pro

cess

pag

efa

ult

rate

–if

too

hig

h,al

loca

tem

ore

fram

esto

proce

ss

Oper

atin

gSys

tem

s—

Fra

me

Allo

cation

67

Segm

enta

tion

pro

ced

ure

sta

ck

ma

in()

sym

bo

ls

sys

lib

rary

sta

ck

sys

lib

rary

pro

ced

ure

sym

bo

ls

ma

in()

Lim

itB

ase

01

2

3

4

0 1 2 3 4

1000

200

5000

200

300

0

200

5200

5300

5600

5700

5900

6900

0

5900

200

5700

5300

Lo

gic

al

Ad

dre

ss

Sp

ace

Ph

ysic

al

Mem

ory

Seg

men

t

Tab

le

•W

hen

progr

amm

ing,

ause

rpr

efer

sto

view

mem

ory

asa

set

of“o

bje

cts”

ofva

riou

ssize

s,w

ith

no

par

ticu

lar

order

ing

•Seg

men

tation

suppor

tsth

isuse

r-vi

ewof

mem

ory

—lo

gica

lad

dre

sssp

ace

isa

colle

ctio

nof

(typ

ical

lydisjo

int)

segm

ents

.

–Seg

men

tshav

ea

nam

e(o

ra

num

ber

)an

da

lengt

h.

–Log

ical

addre

sses

spec

ify

segm

ent

and

offse

t.

•C

ontr

ast

with

pag

ing

wher

euse

ris

unaw

are

ofm

emor

yst

ruct

ure

(one

big

linea

rvi

rtual

addre

sssp

ace,

allm

anag

edtr

ansp

aren

tly

byO

S).

Oper

atin

gSys

tem

s—

Seg

men

tation

68

Imple

menting

Segm

ents

•M

ainta

ina

segm

ent

table

for

each

proce

ss:

Segment

Access

Base

Size

Others!

•If

prog

ram

has

ave

ryla

rge

num

ber

ofse

gmen

tsth

enth

eta

ble

iske

pt

inm

emor

y,poi

nte

dto

byST

bas

ere

gist

erST

BR

•A

lso

nee

da

ST

lengt

hre

gist

erST

LR

since

num

ber

ofse

gsuse

dby

diff

eren

tpr

ogra

ms

will

diff

erw

idel

y

•T

he

table

ispar

tof

the

proce

ssco

nte

xtan

dhen

ceis

chan

ged

onea

chpr

oce

sssw

itch

.

Alg

orithm

:

1.P

rogr

ampr

esen

tsad

dre

ss(s,d).

Chec

kth

ats<

ST

LR

.If

not

,fa

ult

2.O

bta

inta

ble

entr

yat

refe

rence

s+ST

BR

,a

tuple

offo

rm(b

s,l

s)

3.If0≤

d<

l sth

enth

isis

ava

lidad

dre

ssat

loca

tion

(bs,d),

else

fault

Oper

atin

gSys

tem

s—

Seg

men

tation

69

Shar

ing

and

Pro

tect

ion

•B

igad

vanta

geof

segm

enta

tion

isth

atpr

otec

tion

isper

segm

ent;

i.e.

corr

esponds

tolo

gica

lvi

ew(a

nd

prog

ram

mer

’svi

ew)

•P

rote

ctio

nbits

asso

ciat

edw

ith

each

ST

entr

ych

ecke

din

usu

alw

ay

–e.

g.in

stru

ctio

nse

gmen

ts(s

hou

ldbe

non

-sel

fm

odifyi

ng!

)ca

nbe

prot

ecte

dag

ainst

write

s

–e.

g.pla

ceea

char

ray

inow

nse

g⇒

arra

ylim

its

chec

ked

byh/w

•Seg

men

tation

also

faci

litat

essh

arin

gof

code/

dat

a

–ea

chpr

oce

sshas

its

own

ST

BR

/ST

LR

–sh

arin

gen

able

dw

hen

two

proce

sses

hav

eid

entica

len

trie

s

–fo

rdat

ase

gmen

tsca

nuse

copy

-on-w

rite

asper

pag

edca

se.

•Sev

eral

subtle

cave

ats

exist

with

segm

enta

tion

—e.

g.ju

mps

within

shar

edco

de.

Oper

atin

gSys

tem

s—

Seg

men

tation

70

Shar

ing

Segm

ents

Per-

pro

cess

Seg

men

tT

ab

les

Ph

ysic

al M

em

ory

Sh

are

d

A B

A B

Syste

mS

eg

men

tT

ab

le

[DA

NG

ER

OU

S]

[SA

FE

]

Shar

ing

segm

ents

:dan

gero

usly

(lhs)

and

safe

ly(r

hs)

•w

aste

ful(a

nd

dan

gero

us)

tost

ore

com

mon

info

rmat

ion

onsh

ared

segm

ent

inea

chpr

oce

ssse

gmen

tta

ble

–w

antcanonical

vers

ion

ofse

gmen

tin

fo

•as

sign

each

segm

ent

auniq

ue

Sys

tem

Seg

men

tN

um

ber

(SSN

)

•pr

oce

ssse

gmen

tta

ble

map

sfr

oma

Pro

cess

Seg

men

tN

um

ber

(PSN

)to

SSN

Oper

atin

gSys

tem

s—

Seg

men

tation

71

Ext

ern

alFra

gm

enta

tion

Retu

rns.

..

•Lon

gte

rmsc

hed

ule

rm

ust

find

spot

sin

mem

ory

for

allse

gm

ents

of

apr

ogra

m...

but

segs

are

ofva

riab

lesize⇒

lead

sto

frag

men

tation

.

•Tra

deo

ffbet

wee

nco

mpac

tion

/del

aydep

ends

onth

edistr

ibution

of

segm

ent

size

s...

–O

ne

extr

eme:

each

proce

ssge

tsex

actly

1se

gmen

t⇒

reduce

sto

variab

lesize

dpar

tition

s

–A

not

her

extr

eme:

each

byte

isa

“seg

men

t”,se

par

atel

yre

loca

ted

⇒quad

ruple

sm

emor

yuse

!

–Fix

edsize

smal

lse

gm

ents≡

pag

ing!

•In

gener

alw

ith

smal

lav

erag

ese

gmen

tsize

s,ex

tern

alfr

agm

enta

tion

issm

all(c

onsider

pac

king

smal

lsu

itca

ses

into

boot

ofca

r...)

Oper

atin

gSys

tem

s—

Seg

men

tation

72

Segm

enta

tion

vers

us

Pagin

g

logi

calvi

ewal

loca

tion

Seg

men

tation✔

✘Pag

ing

✘✔

⇒tr

yco

mbin

edsc

hem

e.

•E.g

.pag

edse

gmen

ts(M

ultic

s,O

S/2

)

–div

ide

each

segm

ents i

into

k=⌈l

i/2n⌉

pag

es,w

her

el i

isth

elim

it(len

gth)

ofth

ese

gmen

tan

d2n

isth

epag

esize

.

–hav

ese

per

ate

pag

eta

ble

for

ever

yse

gmen

t.

✘hig

hhar

dw

are

cost

/co

mple

xity

.

✘not

very

por

table

.

•E.g

.so

ftw

are

segm

ents

(mos

tm

oder

nO

Ss)

–co

nsider

pag

es[m

,...,m

+l]

tobe

a“s

egm

ent”

–O

Sm

ust

ensu

repr

otec

tion

/sh

arin

gke

pt

consist

ent

over

regi

on.

✘lo

ssin

gran

ula

rity

.

✔re

lative

lysim

ple

/por

table

.

Oper

atin

gSys

tem

s—

Seg

men

tation

73

Sum

mar

y(1

of2)

Old

syst

ems

direc

tly

acce

ssed

[phys

ical

]m

emor

y,w

hic

hca

use

dso

me

prob

lem

s,e.

g.

•C

ontigu

ous

allo

cation

:

–nee

dla

rge

lum

pof

mem

ory

for

proce

ss

–w

ith

tim

e,ge

t[e

xter

nal

]fr

agm

enta

tion

⇒re

quire

expen

sive

com

pac

tion

•A

ddre

ssbin

din

g(i.e

.dea

ling

withabsolute

addre

ssin

g):

–“int

x;

x=

5;”→

“movl

$0x5,

????”

–co

mpile

tim

e⇒

must

know

load

addre

ss.

–lo

adtim

e⇒

wor

kev

ery

tim

e.

–w

hat

abou

tsw

appin

g?

•Por

tabili

ty:

–how

much

mem

ory

shou

ldw

eas

sum

ea

“sta

ndar

d”

mac

hin

ew

illhav

e?

–w

hat

hap

pen

sif

ithas

less

?or

mor

e?

Turn

sou

tth

atw

eca

nav

oid

lots

ofpr

oble

ms

byse

par

atin

gco

nce

pts

oflo

gica

lor

virt

ual

addre

sses

and

phys

ical

addre

sses

.

Oper

atin

gSys

tem

s—

Virtu

alA

ddre

ssin

gSum

mar

y74

Sum

mar

y(2

of2) C

PU

Memory

MM

U

logical

address

physical

address

fault (to OS)

translation

Run

tim

em

appin

gfr

omlo

gica

lto

phys

ical

addre

sses

per

form

edby

spec

ialhar

dw

are

(the

MM

U).

Ifw

em

ake

this

map

pin

ga

per

proce

ssth

ing

then

:

•Eac

hpr

oce

sshas

own

addre

sssp

ace.

•A

lloca

tion

prob

lem

solv

ed(o

rat

leas

tsp

lit):

–vi

rtual

addre

ssal

loca

tion

easy

.

–al

loca

tephys

ical

mem

ory

‘beh

ind

the

scen

es’.

•A

ddre

ssbin

din

gso

lved

:

–bin

dto

logi

calad

dre

sses

atco

mpile

-tim

e.

–bin

dto

real

addre

sses

atlo

adtim

e/ru

ntim

e.

Moder

nop

erat

ing

syst

ems

use

pag

ing

har

dw

are

and

fake

out

segm

ents

inso

ftw

are.

Oper

atin

gSys

tem

s—

Virtu

alA

ddre

ssin

gSum

mar

y75

I/O

Har

dwar

e

•W

ide

variet

yof

‘dev

ices

’w

hic

hin

tera

ctw

ith

the

com

pute

rvi

aI/

O:

–H

um

anre

adab

le:

grap

hic

aldispla

ys,ke

yboa

rd,m

ouse

,pr

inte

rs

–M

achin

ere

adab

le:

disks

,ta

pes

,C

D,se

nso

rs

–C

omm

unic

atio

ns:

modem

s,net

wor

kin

terf

aces

•T

hey

diff

ersign

ifica

ntly

from

one

anot

her

with

rega

rdto

:

–D

ata

rate

–C

omple

xity

ofco

ntr

ol

–U

nit

of

tran

sfer

–D

irec

tion

oftr

ansf

er

–D

ata

repr

esen

tation

–Err

orhan

dlin

g

⇒har

dto

pres

ent

aunifor

mI/

Osy

stem

whic

hm

asks

allco

mple

xity

I/O

subsy

stem

isgenera

lly

the

‘mess

iest

’par

tofO

S.

Oper

atin

gSys

tem

s—

I/O

Subsy

stem

76

I/O

Subsy

stem

De

vic

e D

riv

er

La

ye

rD

evic

e

Dri

ver

Devic

e

Dri

ver

Devic

e

Dri

ver

Co

mm

on

I/O

Fu

nc

tio

ns

Keyb

oard

Hard

Dis

kN

etw

ork

De

vic

e L

ay

er

Vir

tua

l D

ev

ice

La

ye

r

H/W

Unpriv

Priv

I/O

Sc

he

du

lin

gI/

O B

uff

eri

ng

Ap

plicati

on

-I/O

In

terf

ace

•P

rogr

ams

acce

ssvi

rtual

dev

ices

:

–te

rmin

alst

ream

snot

term

inal

s

–w

indow

snot

fram

ebuffer

–ev

ent

stre

amnot

raw

mou

se

–file

snot

disk

blo

cks

–pr

inte

rsp

ool

ernot

par

alle

lpor

t

–tr

ansp

ort

prot

oco

lsnot

raw

ether

net

•O

Sdea

lsw

ith

proce

ssor

–dev

ice

inte

rfac

e:

–I/

Oin

stru

ctio

ns

vers

us

mem

ory

map

ped

–I/

Ohar

dw

are

type

(e.g

.10

’sof

serial

chip

s)

–pol

led

vers

us

inte

rrupt

drive

n

–pr

oce

ssor

inte

rrupt

mec

han

ism

Oper

atin

gSys

tem

s—

I/O

Subsy

stem

77

Polled

Mode

I/O

status

command

data (r/w)

device-busy (R/O)

command-ready (W/O)

error (R/O)

read (W/O)

write (W/O)

*

•C

onsider

asim

ple

dev

ice

with

thre

ere

gist

ers:

status,data

andcommand.

•(H

ost

can

read

and

write

thes

evi

abus)

•T

hen

pol

led

mode

oper

atio

nw

orks

asfo

llow

s:

–H

ost

repea

tedly

read

sdevicebusy

untilcl

ear.

–H

ost

sets

e.g.

write

bit

incommand

regi

ster

,an

dputs

dat

ain

todata

regi

ster

.

–H

ost

sets

commandready

bit

instatus

regi

ster

.

–D

evic

ese

escommandready

and

sets

devicebusy.

–D

evic

eper

form

sw

rite

oper

atio

n.

–D

evic

ecl

ears

commandready

&th

endevicebusy.

•W

hat

’sth

epr

oble

mher

e?

Oper

atin

gSys

tem

s—

I/O

Subsy

stem

78

Inte

rrupts

Revi

site

d

Rec

all:

tohan

dle

mism

atch

bet

wee

nC

PU

and

dev

ice

spee

ds,

proce

ssor

spr

ovid

ean

inte

rrupt

mec

han

ism

:

•at

end

ofea

chin

stru

ctio

n,pr

oce

ssor

chec

ksin

terr

upt

line(

s)fo

rpen

din

gin

terr

upt

•if

line

isas

sert

edth

enpr

oce

ssor

:

–sa

ves

prog

ram

counte

r,

–sa

ves

proce

ssor

stat

us,

–ch

ange

spr

oce

ssor

mode,

and

–ju

mp

toa

wel

lkn

own

addre

ss(o

rits

conte

nts

)

•af

ter

inte

rrupt-

han

dlin

gro

utine

isfinished

,ca

nuse

e.g.

therti

inst

ruct

ion

tore

sum

ew

her

ew

ele

ftoff

.

Som

em

ore

com

ple

xpr

oce

ssor

spr

ovid

e:

•m

ultip

lele

vels

ofin

terr

upts

•har

dw

are

vect

orin

gof

inte

rrupts

•m

ode

dep

enden

tre

gist

ers

Oper

atin

gSys

tem

s—

I/O

Subsy

stem

79

Inte

rrupt-

Drive

nI/

O

Can

split

imple

men

tation

into

low

-lev

elinterrupthandler

plu

sper

-dev

iceinterrupt

serviceroutine:

•in

terr

upt

han

dle

r(p

roce

ssor

-dep

enden

t)m

ay:

–sa

vem

ore

regi

ster

s–

esta

blis

ha

langu

age

envi

ronm

ent

(e.g

.a

Cru

n-t

ime

stac

k)–

dem

ultip

lex

inte

rrupt

inso

ftw

are.

–in

voke

appr

opriat

ein

terr

upt

serv

ice

routine

(ISR

)

•T

hen

inte

rrupt

serv

ice

routine

(dev

ice-

spec

ific

but

not

proce

ssor

-spec

ific)

will

:

1.fo

rpr

ogra

mm

edI/

Odev

ice:

–tr

ansf

erdat

a.–

clea

rin

terr

upt

(som

etim

esa

side

effec

tof

tx).

1.fo

rD

MA

dev

ice:

–ac

know

ledge

tran

sfer

.2.

reques

tan

other

tran

sfer

ifth

ere

are

any

mor

eI/

Ore

ques

tspen

din

gon

dev

ice.

3.sign

alan

yw

aiting

proce

sses

.4.

ente

rsc

hed

ule

ror

retu

rn.

Quest

ion:

who

issc

hed

ulin

gw

ho?

Oper

atin

gSys

tem

s—

I/O

Subsy

stem

80

Devi

ceCla

sses

Hom

ogen

isin

gdev

ice

AP

Ico

mple

tely

not

pos

sible

⇒O

Sge

ner

ally

split

sdev

ices

into

fourclasses

:

1.B

lock

dev

ices

(e.g

.disk

drive

s,C

D):

•co

mm

ands

incl

uderead,write,seek

•ra

wI/

Oor

file

-sys

tem

acce

ss

•m

emor

y-m

apped

file

acce

sspos

sible

2.C

har

acte

rdev

ices

(e.g

.ke

yboa

rds,

mic

e,se

rial

por

ts):

•co

mm

ands

incl

udeget,put

•lib

raries

laye

red

onto

pto

allo

wlin

eed

itin

g

3.N

etw

ork

Dev

ices

•va

ryin

gen

ough

from

blo

ckan

dch

arac

ter

tohav

eow

nin

terf

ace

•U

nix

and

Win

dow

s/N

Tuse

socket

inte

rfac

e

4.M

isce

llaneo

us

(e.g

.cl

ock

san

dtim

ers)

•pr

ovid

ecu

rren

ttim

e,el

apse

dtim

e,tim

er

•ioctl

(on

UN

IX)

cove

rsodd

aspec

tsof

I/O

such

ascl

ock

san

dtim

ers.

Oper

atin

gSys

tem

s—

I/O

Subsy

stem

81

I/O

Buffering

•B

uffer

ing:

OS

stor

es(its

own

copy

of)

dat

ain

mem

ory

while

tran

sfer

ring

toor

from

dev

ices

–to

cope

with

dev

ice

spee

dm

ism

atch

–to

cope

with

dev

ice

tran

sfer

size

mism

atch

–to

mai

nta

in“c

opy

sem

antics

”

•O

Sca

nuse

variou

ski

nds

of

buffer

ing:

1.singl

ebuffer

ing

—O

Sas

sign

sa

syst

embuffer

toth

euse

rre

ques

t

2.dou

ble

buffer

ing

—pr

oce

ssco

nsu

mes

from

one

buffer

while

syst

emfills

the

nex

t

3.ci

rcula

rbuffer

s—

mos

tuse

fulfo

rburs

tyI/

O

•M

any

aspec

tsof

buffer

ing

dic

tate

dby

dev

ice

type:

–ch

arac

ter

dev

ices⇒

line

prob

ably

suffi

cien

t.

–net

wor

kdev

ices⇒

burs

ty(t

ime

&sp

ace)

.

–blo

ckdev

ices⇒

lots

offixe

dsize

tran

sfer

s.

–(las

tusu

ally

maj

oruse

rof

buffer

mem

ory)

Oper

atin

gSys

tem

s—

I/O

Subsy

stem

82

Blo

ckin

gv.N

onblo

ckin

gI/

O

Fro

mth

epr

ogra

mm

er’s

poi

nt

ofvi

ew,I/

Osy

stem

calls

exhib

iton

eof

thre

eki

nds

ofbeh

avio

ur:

1.B

lock

ing:

proce

sssu

spen

ded

untilI/

Oco

mple

ted

•ea

syto

use

and

under

stan

d.

•in

suffi

cien

tfo

rso

me

nee

ds.

2.N

onblo

ckin

g:

I/O

call

retu

rns

asm

uch

asav

aila

ble

•re

turn

sal

mos

tim

med

iate

lyw

ith

count

ofby

tes

read

orw

ritt

en(p

ossibly

0).

•ca

nbe

use

dby

e.g.

use

rin

terf

ace

code.

•es

sential

lyap

plic

atio

n-lev

el“p

olle

dI/

O”.

3.A

synch

ronou

s:pr

oce

ssco

ntinues

toru

nw

hile

I/O

exec

ute

s

•I/

Osu

bsy

stem

explic

itly

sign

als

proce

ssw

hen

its

I/O

reques

thas

com

ple

ted.

•m

ost

flex

ible

(and

pot

ential

lyeffi

cien

t).

•...but

also

mos

tdiffi

cult

touse

.

Mos

tsy

stem

spr

ovid

ebot

hblo

ckin

gan

dnon

-blo

ckin

gI/

Oin

terf

aces

;m

oder

nsy

stem

s(e

.g.N

T,Lin

ux)

also

suppor

tas

ynch

ronou

sI/

O,but

use

din

freq

uen

tly.

Oper

atin

gSys

tem

s—

I/O

Subsy

stem

83

Oth

er

I/O

Issu

es

•C

achin

g:fa

stm

emor

yhol

din

gco

pyof

dat

a

–ca

nw

ork

with

bot

hre

ads

and

write

s

–ke

yto

I/O

per

form

ance

•Sch

edulin

g:

–e.

g.or

der

ing

I/O

reques

tsvi

aper

-dev

ice

queu

e

–so

me

oper

atin

gsy

stem

str

yfa

irnes

s...

•Spool

ing:

queu

eou

tput

for

adev

ice

–use

fulfo

r“s

ingl

euse

r”dev

ices

whic

hca

nse

rve

only

one

reques

tat

atim

e(e

.g.

prin

ter)

•D

evic

ere

serv

atio

n:

–sy

stem

calls

for

acquirin

gor

rele

asin

gex

clusive

acce

ssto

adev

ice

(car

eful!)

•Err

orhan

dlin

g:

–e.

g.re

cove

rfr

omdisk

read

,dev

ice

unav

aila

ble

,tr

ansien

tw

rite

failu

res,

etc.

–m

ost

I/O

syst

emca

llsre

turn

aner

ror

num

ber

orco

de

when

anI/

Ore

ques

tfa

ils

–sy

stem

erro

rlo

gshol

dpr

oble

mre

por

ts.

Oper

atin

gSys

tem

s—

I/O

Subsy

stem

84

I/O

and

Perform

ance

•I/

Ois

am

ajor

fact

orin

over

allsy

stem

per

form

ance

–dem

ands

CP

Uto

exec

ute

dev

ice

drive

r,ke

rnel

I/O

code,

etc.

–co

nte

xtsw

itch

esdue

toin

terr

upts

–dat

aco

pyin

g,buffer

ing,et

c

–(n

etw

ork

traffi

ces

pec

ially

stre

ssfu

l)

•Im

prov

ing

per

form

ance

:

–re

duce

num

ber

ofco

nte

xtsw

itch

es

–re

duce

dat

aco

pyin

g

–re

duce

#in

terr

upts

byusing

larg

etr

ansf

ers,

smar

tco

ntr

olle

rs,

adap

tive

pol

ling

(e.g

.Lin

ux

NA

PI)

–use

DM

Aw

her

epos

sible

–bal

ance

CP

U,m

emor

y,bus

and

I/O

for

bes

tth

roughput.

Impro

vin

gI/

Operform

ance

isa

majo

rre

main

ing

OS

challenge

Oper

atin

gSys

tem

s—

I/O

Subsy

stem

85

File

Managem

ent

Dir

ec

tory

Se

rvic

e

Sto

rag

e S

erv

ice

Dis

k H

an

dle

r

text

nam

eu

ser

file

-id

info

rmati

on

req

ueste

d

fro

m f

ile

user

space

I/O

subsyste

m

filin

g s

yste

m

Fili

ng

syst

ems

hav

etw

om

ain

com

pon

ents

:

1.D

irec

tory

Ser

vice

•m

aps

from

nam

esto

file

iden

tifier

s.

•han

dle

sac

cess

&ex

iste

nce

contr

ol

2.Sto

rage

Ser

vice

•pr

ovid

esm

echan

ism

tost

ore

dat

aon

disk

•in

cludes

mea

ns

toim

ple

men

tdirec

tory

serv

ice

Oper

atin

gSys

tem

s—

Fili

ng

Sys

tem

s86

File

Conce

pt

What

isa

file

?

•B

asic

abst

ract

ion

for

non

-vol

atile

stor

age.

•Typ

ical

lyco

mpr

ises

asingl

eco

ntigu

ous

logi

calad

dre

sssp

ace.

•In

tern

alst

ruct

ure

:

1.N

one

(e.g

.se

quen

ceof

wor

ds,

byte

s)

2.Sim

ple

reco

rdst

ruct

ure

s

–lin

es–

fixe

dle

ngt

h–

variab

lele

ngt

h

3.C

omple

xst

ruct

ure

s

–fo

rmat

ted

docu

men

t–

relo

cata

ble

obje

ctfile

•C

ansim

ula

te2,

3w

ith

byte

sequen

ceby

inse

rtin

gap

prop

riat

eco

ntr

olch

arac

ters

.

•A

lla

ques

tion

ofw

ho

dec

ides

:

–op

erat

ing

syst

em

–pr

ogra

m(m

er).

Oper

atin

gSys

tem

s—

File

san

dFile

Met

a-dat

a87

Nam

ing

Files

File

susu

ally

hav

eat

leas

ttw

oki

nds

of‘n

ame’

:

1.sy

stem

file

iden

tifier

(SFID

):

•(t

ypic

ally

)a

uniq

ue

inte

ger

valu

eas

soci

ated

with

agi

ven

file

•SFID

sar

eth

enam

esuse

dw

ithin

the

filin

gsy

stem

itse

lf

2.hum

an-r

eadab

lenam

e,e.

g.hello.java

•w

hat

use

rslik

eto

use

•m

appin

gfr

omhum

annam

eto

SFID

ishel

din

adirectory

,e.

g.

Name

SFID

hello.java

23812

Makefile

12353

README

9742

•direc

tories

also

non

-vol

atile⇒

must

be

stor

edon

disk

alon

gw

ith

file

s.

3.Fre

quen

tly

also

get

use

rfile

iden

tifier

(UFID

)

•use

dto

iden

tify

open

file

s(s

eela

ter)

Oper

atin

gSys

tem

s—

File

san

dFile

Met

a-dat

a88

File

Meta

-data

Typ

e (

file

or

dire

cto

ry)

Lo

ca

tio

n o

n D

isk

Siz

e in

byte

s

Tim

e o

f cre

atio

n

Acce

ss p

erm

issio

ns

File C

on

tro

l B

lock

Meta

data

Tab

le(o

n d

isk)

f(SFID)

SFID

As

wel

las

thei

rco

nte

nts

and

thei

rnam

e(s)

,file

sca

nhav

eoth

erat

trib

ute

s,e.

g.

•Loca

tion

:poi

nte

rto

file

loca

tion

ondev

ice

•Siz

e:cu

rren

tfile

size

•Typ

e:nee

ded

ifsy

stem

suppor

tsdiff

eren

tty

pes

•P

rote

ctio

n:

contr

ols

who

can

read

,w

rite

,et

c.

•T

ime,

dat

e,an

duse

rid

entifica

tion

:fo

rpr

otec

tion

,se

curity

and

usa

ge

mon

itor

ing.

Tog

ether

this

info

rmat

ion

isca

lled

meta

-data

.It

isco

nta

ined

ina

file

contr

olblo

ck.

Oper

atin

gSys

tem

s—

File

san

dFile

Met

a-dat

a89

Direct

ory

Nam

eSpace

(I)

What

are

the

requirem

ents

for

our

nam

esp

ace?

•Effi

cien

cy:

loca

ting

afile

quic

kly.

•N

amin

g:use

rco

nve

nie

nce

–al

low

two

(or

mor

ege

ner

ally

N)

use

rsto

hav

eth

esa

me

nam

efo

rdiff

eren

tfile

s

–al

low

one

file

hav

ese

vera

ldiff

eren

tnam

es

•G

roupin

g:lo

gica

lgr

oupin

gof

file

sby

prop

erties

(e.g

.al

lJa

vapr

ogra

ms,

allga

mes

)

First

atte

mpts

:

•Sin

gle-

leve

l:on

edirec

tory

shar

edbet

wee

nal

luse

rs

⇒nam

ing

prob

lem

⇒gr

oupin

gpr

oble

m

•T

wo-

leve

ldirec

tory

:on

edirec

tory

per

use

r

–ac

cess

viapathname

(e.g

.bob:hello.java)

–ca

nhav

esa

me

file

nam

efo

rdiff

eren

tuse

r

–but

still

no

grou

pin

gca

pab

ility

.

Oper

atin

gSys

tem

s—

Direc

tories

90

Direct

ory

Nam

eSpace

(II)

An

nB

ob

Ya

o

java

mail

A

BC

GH

IJ

sen

t

FE

D

•G

etm

ore

flex

ibili

tyw

ith

age

ner

alhie

rarc

hy.

–direc

tories

hol

dfile

sor

[furt

her

]direc

tories

–cr

eate

/del

ete

file

sre

lative

toa

give

ndirec

tory

•H

um

annam

eis

full

pat

hnam

e,but

can

get

long:

e.g.

/usr

/gro

ups/

X11

R5/

src/

mit/s

erve

r/os

/4.2

bsd

/utils.c

–off

erre

lative

nam

ing

–lo

gin

direc

tory

–cu

rren

tw

orki

ng

direc

tory

•W

hat

does

itm

ean

todel

ete

a[s

ub]-direc

tory

?

Oper

atin

gSys

tem

s—

Direc

tories

91

Direct

ory

Nam

eSpace

(III)

An

nB

ob

Yao

java

mail

A

BC

DE

F

GH

IJ

sen

t

•H

iera

rchy

good,but

still

only

one

nam

eper

file

.

⇒ex

tend

todirec

ted

acyc

licgr

aph

(DA

G)

stru

cture

:

–al

low

shar

edsu

bdirec

tories

and

file

s.

–ca

nhav

em

ultip

leal

iase

sfo

rth

esa

me

thin

g

•Pro

ble

m:

dan

glin

gre

fere

nce

s

•Sol

ution

s:

–bac

k-re

fere

nce

s(b

ut

require

variab

lesize

reco

rds)

;or

–re

fere

nce

counts

.

•Pro

ble

m:

cycl

es...

Oper

atin

gSys

tem

s—

Direc

tories

92

Direct

ory

Imple

menta

tion

/Ann/mail/B

Ann

Bob

Yao

Name

SFID

1034

179

7182

mail

A

Name

SFID

2165

5797

sent

B C

Name

SFID

434

2459

25

D

D

D Y Y Y

Y

Y N

N N

•D

irec

tories

are

non-v

ola

tile⇒

stor

eas

“file

s”on

disk,

each

with

own

SFID

.

•M

ust

be

diff

eren

tty

pes

offile

(for

trav

ersa

l)

•Exp

licit

direc

tory

oper

atio

ns

incl

ude:

–cr

eate

direc

tory

–del

ete

direc

tory

–lis

tco

nte

nts

–se

lect

curr

ent

wor

king

direc

tory

–in

sert

anen

try

for

afile

(a“l

ink”

)

Oper

atin

gSys

tem

s—

Direc

tories

93

File

Opera

tions

(I) U

FID

SFID

File Control Block (Copy)

1 2 3 4

23421

3250

10532

7122

location on disk, size,...

" "

" "

" "

•O

pen

ing

afile

:UFID

=open(<pathname>)

1.direc

tory

serv

ice

recu

rsiv

ely

sear

ches

for

com

pon

ents

of<pathname>

2.if

allgo

esw

ell,

even

tual

lyge

tSFID

offile

.3.

copy

file

contr

olblo

ckin

tom

emor

y.4.

crea

tenew

UFID

and

retu

rnto

calle

r.

•C

reat

ea

new

file

:UFID

=create(<pathname>)

•O

nce

hav

eUFID

can

read

,w

rite

,et

c.

–va

riou

sm

odes

(see

nex

tslid

e)

•C

losing

afile

:status

=close(UFID)

1.co

py[n

ew]file

contr

olblo

ckbac

kto

disk.

2.in

valid

ateUFID

Oper

atin

gSys

tem

s—

File

syst

emIn

terf

ace

94

File

Opera

tions

(II)

cu

rre

nt

file

po

sit

ion

en

d o

f fi

les

tart

of

file

alr

ea

dy

ac

ce

ss

ed

to b

e r

ea

d

•A

ssoci

ate

acu

rsor

orfile

pos

itio

nw

ith

each

open

file

(viz

.U

FID

)

–in

itia

lised

atopen

tim

eto

refe

rto

star

tof

file

.

•B

asic

oper

atio

ns:

read

next

orwrite

next,

e.g.

–read(UFID,

buf,

nbytes),or

read(UFID,

buf,

nrecords)

•Seq

uen

tial

Acc

ess:

abov

e,plu

srewind(UFID).

•D

irec

tA

cces

s:read

Nor

write

N

–al

low

“ran

dom

”ac

cess

toan

ypar

tof

file

.

–ca

nim

ple

men

tw

ithseek(UFID,

pos)

•O

ther

form

sof

dat

aac

cess

pos

sible

,e.

g.

–ap

pen

d-o

nly

(may

be

fast

er)

–in

dex

edse

quen

tial

acce

ssm

ode

(ISA

M)

Oper

atin

gSys

tem

s—

File

syst

emIn

terf

ace

95

Oth

er

Filin

gSys

tem

Issu

es

•A

cces

sC

ontr

ol:

file

owner

/cre

ator

shou

ldbe

able

toco

ntr

olw

hat

can

be

don

e,an

dby

whom

.

–nor

mal

lya

funct

ion

ofdirec

tory

serv

ice⇒

chec

ksdon

eat

file

open

tim

e

–va

riou

sty

pes

ofac

cess

,e.

g.

∗re

ad,w

rite

,ex

ecute

,(a

ppen

d?)

,∗

del

ete,

list,

renam

e

–m

ore

adva

nce

dsc

hem

espos

sible

(see

late

r)

•Exi

sten

ceC

ontr

ol:

what

ifa

use

rdel

etes

afile

?

–pr

obab

lyw

ant

toke

epfile

inex

iste

nce

while

ther

eis

ava

lidpat

hnam

ere

fere

nci

ng

it

–plu

sch

eck

entire

FS

per

iodic

ally

for

garb

age

–ex

iste

nce

contr

olca

nal

sobe

afa

ctor

when

afile

isre

nam

ed/m

oved

.

•C

oncu

rren

cyC

ontr

ol:

nee

dso

me

form

oflocking

tohan

dle

sim

ultan

eous

acce

ss

–m

aybe

man

dat

ory

orad

viso

ry

–lo

cks

may

be

shar

edor

excl

usive

–gr

anula

rity

may

be

file

orsu

bse

t

Oper

atin

gSys

tem

s—

File

syst

emIn

terf

ace

96

Pro

tect

ion

Req

uire

prot

ection

agai

nst

unau

thor

ised

:

•re

leas

eof

info

rmat

ion

–re

adin

gor

leak

ing

dat

a–

viol

atin

gpr

ivac

yle

gislat

ion

–using

prop

riet

ary

soft

war

e–

cove

rtch

annel

s

•m

odifi

cation

of

info

rmat

ion

–ca

nal

so“s

abot

age”

bydel

etin

gor

hid

ing

info

rmat

ion

•den

ialof

serv

ice

–ca

using

acr

ash

–ca

using

hig

hlo

ad(e

.g.

proce

sses

orpac

kets

)–

chan

ging

acce

ssrights

Also

wish

topr

otec

tag

ainst

the

effec

tsof

erro

rs:

•isol

ate

for

deb

ugg

ing

•isol

ate

for

dam

age

contr

ol

Inge

ner

al,pr

otec

tion

mec

han

ism

sim

pos

eco

ntr

ols

onac

cess

bysubjects

(e.g

.use

rs)

onobjects

(e.g

.file

sor

dev

ices

oroth

erpr

oce

sses

).

Oper

atin

gSys

tem

s—

Pro

tect

ion

97

Pro

tect

ion

and

Shar

ing

Ifw

ehav

ea

singl

euse

rm

achin

ew

ith

no

net

wor

kco

nnec

tion

ina

lock

edro

omth

enpr

otec

tion

isea

sy...but

inpr

actice

we

wan

tto

:

•sh

are

faci

litie

s(f

orec

onom

icre

ason

s)

•sh

are

and

exch

ange

dat

a(a

pplic

atio

nre

quirem

ent)

Som

em

echan

ism

sw

ehav

eal

read

yco

me

acro

ss:

•use

ran

dsu

per

viso

rle

vels

–usu

ally

one

ofea

ch

–co

uld

hav

ese

vera

l(e

.g.M

ULT

ICS

rings

)

•m

emor

ym

anag

emen

thar

dw

are

–re

loca

tion

har

dw

are

–bou

nds

chec

king

–se

par

ate

addre

sssp

aces

•file

s

–ac

cess

contr

ollis

t

–gr

oups

etc

Oper

atin

gSys

tem

s—

Pro

tect

ion

98

Design

ofPro

tect

ion

Sys

tem

•Som

eot

her

prot

ection

mec

han

ism

s:

–lo

ckth

eco

mpute

rro

om(p

reve

nt

peo

ple

from

tam

per

ing

with

the

har

dw

are)

–re

strict

acce

ssto

syst

emso

ftw

are

–de-

skill

syst

ems

oper

atin

gst

aff

–ke

epdes

igner

saw

ayfr

omfinal

syst

em!

–use

pas

swor

ds

(in

gener

alch

alle

nge

/res

pon

se)

–use

encr

yption

–le

gislat

e

•re

f:Sal

tzer

+Sch

roed

erP

roc.

IEEE

Sep

t75

–des

ign

shou

ldbe

public

–def

ault

shou

ldbe

no

acce

ss

–ch

eck

for

curr

ent

auth

ority

–gi

veea

chpr

oce

ssm

inim

um

pos

sible

auth

ority

–m

echan

ism

ssh

ould

be

sim

ple

,unifor

man

dbuilt

into

low

est

laye

rs

–sh

ould

be

psy

chol

ogic

ally

acce

pta

ble

–co

stof

circ

um

vention

shou

ldbe

hig

h

–m

inim

ize

shar

edac

cess

Oper

atin

gSys

tem

s—

Pro

tect

ion

99

Auth

entica

tion

ofU

ser

toSys

tem

(1)

Pas

swor

ds

curr

ently

wid

ely

use

d:

•id

eally

wan

ta

long

sequen

ceof

random

char

acte

rsissu

edby

syst

em,but

use

rw

ould

write

itdow

n

•if

allo

wuse

rse

lect

ion,th

eyw

illuse

dic

tion

ary

wor

ds,

car

regi

stra

tion

,th

eir

nam

e,th

eir

dat

eof

birth

,th

eir

pet

’snam

e,th

eir

pet

’sdat

eof

birth

,et

c.

•bes

tbet

prob

ably

isto

enco

ura

geth

euse

ofan

algo

rith

mto

rem

ember

pas

swor

d

•ot

her

top

tips:

–don

’tre

flec

ton

term

inal

,or

over

prin

t

–ad

ddel

ayaf

ter

faile

dat

tem

pt

–use

encr

yption

iflin

esu

spec

t

•w

hat

abou

tse

curity

ofpas

swor

dfile

?

–on

lyac

cess

ible

tolo

gin

prog

ram

(CA

P,T

ITA

N)

–hol

dsc

ram

ble

d,e.

g.U

NIX

∗on

lynee

dto

write

prote

ctfile

∗nee

dsc

ram

blin

gto

wor

kw

ithou

tpas

swor

d(e

.g.use

a“o

ne

way

”fu

nct

ion)

Oper

atin

gSys

tem

s—

Pro

tect

ion

100

Auth

entica

tion

ofU

ser

toSys

tem

(2)

E.g

.pas

swor

ds

inU

NIX

:

•sim

ple

for

use

rto

rem

ember

arachnid

•se

nsible

use

rap

plie

san

algo

rith

m

!r!chn#d

•pas

swor

dis

DES-e

ncr

ypte

d25

tim

esusing

a2-

byte

per

-use

r‘s

alt’

topr

oduce

a11

byte

string

•sa

ltfo

llow

edby

thes

e11

byte

sar

eth

enst

ored

IML.DVMcz6Sh2

Rea

llyre

quire

unfo

rgea

ble

evid

ence

ofid

entity

that

syst

emca

nch

eck:

•en

han

ced

pas

swor

d:

chal

lenge

-res

pon

se.

•id

card

inse

rted

into

slot

•finge

rprint,

voic

eprint,

face

reco

gnitio

n

•sm

art

card

s

Oper

atin

gSys

tem

s—

Pro

tect

ion

101

Auth

entica

tion

ofSys

tem

toU

ser

Use

rw

ants

toav

oid

•ta

lkin

gto

the

wro

ng

com

pute

r

•or

talk

ing

toth

erigh

tco

mpute

r,but

not

the

login

prog

ram

:-)

Par

tial

solu

tion

inol

dday

sfo

rdirec

tly

wired

term

inal

s:

•m

ake

logi

nch

arac

ter

sam

eas

term

inal

atte

ntion

,or

•al

way

sdo

ate

rmin

alat

tention

bef

ore

tryi

ng

logi

n

But,

today

PC

suse

das

term

inal

s⇒

•lo

calso

ftw

are

may

hav

ebee

nch

ange

d

•so

carr

yyo

ur

own

copy

ofth

ete

rmin

alpr

ogra

m

•but

har

dw

are

/firm

war

ein

public

mac

hin

em

ayhav

ebee

nm

odifi

ed

Anyw

ay,st

illhav

eth

epr

oble

mof

com

ms

lines

:

•w

iret

appin

gis

easy

•w

orks

tation

can

ofte

nse

eal

lpac

kets

onnet

wor

k

⇒m

ust

use

encr

yption

ofso

me

kind,an

dtr

ust

encr

yption

dev

ice

(e.g

.a

smar

tca

rd)

Oper

atin

gSys

tem

s—

Pro

tect

ion

102

Mutu

alsu

spic

ion

•W

enee

dto

enco

ura

ge

lots

and

lots

ofsu

spic

ion:

–sy

stem

ofuse

r

–use

rsof

each

oth

er

–use

rof

syst

em

•C

alle

dpr

ogra

ms

shou

ldbe

susp

icio

us

ofca

ller

(e.g

.O

Sca

llsal

way

snee

dto

chec

kpar

amet

ers)

•C

alle

rsh

ould

be

susp

icio

us

ofca

lled

prog

ram

•e.

g.Tro

jan

hor

se:

–a

‘use

ful’

look

ing

prog

ram

—a

gam

eper

hap

s

–w

hen

calle

dby

use

r(in

man

ysy

stem

s)in

her

its

allof

the

use

r’s

priv

ilege

s

–it

can

then

copy

file

s,m

odify

file

s,ch

ange

pas

swor

d,se

nd

mai

l,et

c...

–e.

g.M

ultic

sed

itor

troj

anhor

se,co

pie

dfile

sas

wel

las

edited

.

•e.

g.V

irus:

–oft

enst

arts

offas

Tro

jan

hor

se

–se

lf-r

eplic

atin

g(e

.g.IL

OV

EY

OU

,C

ode

Red

,Stu

xnet

)

Oper

atin

gSys

tem

s—

Pro

tect

ion

103

Acc

ess

matr

ix

Acc

ess

mat

rix

isa

mat

rix

ofsu

bje

cts

agai

nst

obje

cts.

Subje

ct(o

rpr

inci

pal

)m

ight

be:

•use

rse.

g.by

uid

•ex

ecuting

proce

ssin

apr

ote

ctio

ndom

ain

•se

tsof

use

rsor

proce

sses

Obje

cts

are

thin

gslik

e:

•file

s

•dev

ices

•dom

ains

/pr

oce

sses

•m

essa

gepor

ts(in

mic

roke

rnel

s)

Mat

rix

isla

rge

and

spar

se⇒

don

’tw

ant

tost

ore

ital

l.

Tw

oco

mm

on

repr

esen

tation

s:

1.by

obje

ct:

stor

elis

tof

subje

cts

and

righ

tsw

ith

each

obje

ct⇒

accesscontrollist

2.by

subje

ct:

stor

elis

tof

obje

cts

and

righ

tsw

ith

each

subje

ct⇒

capabilities

Oper

atin

gSys

tem

s—

Pro

tect

ion

104

Acc

ess

Contr

olLists

Oft

enuse

din

stor

age

syst

ems:

•sy

stem

nam

ing

schem

epr

ovid

esfo

rA

CL

tobe

inse

rted

innam

ing

pat

h,e.

g.file

s

•if

AC

Ls

stor

edon

disk

but

chec

kis

mad

ein

soft

war

e⇒

must

only

use

onlo

wduty

cycl

e(o

rper

form

ance

will

suffer

)

•fo

rhig

her

duty

cycl

em

ust

cach

ere

sults

ofch

eck

•e.

g.M

ultic

s:op

enfile

=m

emor

yse

gmen

t.O

nfirs

tre

fere

nce

tose

gmen

t:

1.in

terr

upt

(seg

men

tfa

ult)

2.ch

eck

AC

L

3.se

tup

segm

ent

des

crip

tor

inse

gmen

tta

ble

•m

ost

syst

ems

chec

kA

CL

–w

hen

file

open

edfo

rre

ador

write

–w

hen

code

file

isto

be

exec

ute

d

•ac

cess

contr

olby

prog

ram

allo

ws

arbitra

rypol

icie

s(s

ee.

e.g.

Unix

late

r)

•so

met

imes

even

this

isn’t

enou

gh(e

.g.M

AC

)

Oper

atin

gSys

tem

s—

Pro

tect

ion

105

Capabilitie

s

Cap

abili

ties

asso

ciat

edw

ith

active

subje

cts,

so:

•st

ore

inad

dre

sssp

ace

ofsu

bje

ct

•m

ust

mak

esu

resu

bje

ctca

n’t

forg

eca

pab

ilities

•ea

sily

acce

ssib

leto

har

dw

are

•ca

nbe

use

dw

ith

hig

hduty

cycl

ee.

g.as

par

tof

addre

ssin

ghar

dw

are

–P

less

eyP

P25

0

–C

AP

I,II,III

–IB

Msy

stem

/38

–In

teliA

PX

432

•hav

esp

ecia

lm

achin

ein

stru

ctio

ns

tom

odify

(res

tric

t)ca

pab

ilities

•su

ppor

tpas

sing

ofca

pab

ilities

onpr

oce

dure

(pro

gram

)ca

ll

Can

also

use

software

capab

ilities

:

•ch

ecke

dby

encr

yption

•nic

efo

rdistr

ibute

dsy

stem

s

Oper

atin

gSys

tem

s—

Pro

tect

ion

106

Pass

word

Capabilitie

s

•C

apab

ilities

nic

efo

rdistr

ibute

dsy

stem

sbut:

–m

essy

for

applic

atio

n,an

d

–re

voca

tion

istr

icky

.

•C

ould

use

tim

eouts

(e.g

.A

moeb

a).

•A

lter

nat

ivel

y:co

mbin

epas

swor

ds

and

capab

ilities

.

•Sto

reA

CL

with

obje

ct,but

key

iton

capab

ility

(not

implic

itco

nce

pt

of

“princi

pal

”fr

omO

S).

•A

dva

nta

ges

:

–re

voca

tion

pos

sible

–m

ultip

le“r

oles

”av

aila

ble

.

•D

isad

vanta

ges:

–st

illm

essy

(use

‘im

plic

it’ca

che?

).

Oper

atin

gSys

tem

s—

Pro

tect

ion

107

Cove

rtch

annels

Info

rmat

ion

leak

age

byside-

effec

ts:

lots

offu

n!

At

the

har

dw

are

leve

l:

•w

ire

tappin

g

•m

onitor

sign

als

inm

achin

e

•m

odifi

cation

tohar

dw

are

•el

ectr

omag

net

icra

dia

tion

ofdev

ices

By

soft

war

e:

•le

aka

bit

stre

amas

:

file

exists

pag

efa

ult

com

pute

aw

hile

1no

file

no

pag

efa

ult

slee

pfo

ra

while

0

•sy

stem

may

prov

ide

stat

istics

e.g.

TEN

EX

pas

swor

dcr

acke

rusing

syst

empr

ovid

edco

unt

ofpag

efa

ults

Inge

ner

al,guar

din

gag

ainst

cove

rtch

annel

sis

proh

ibitiv

ely

expen

sive

.

(only

usu

ally

aco

nsider

atio

nfo

rm

ilita

ryty

pes

)

Oper

atin

gSys

tem

s—

Pro

tect

ion

108

Unix

:In

troduct

ion

•U

nix

firs

tdev

elop

edin

1969

atB

ellLab

s(T

hom

pso

n&

Ritch

ie)

•O

rigin

ally

writt

enin

PD

P-7

asm

,but

then

(197

3)re

writt

enin

the

‘new

’hig

h-lev

ella

ngu

ageC

⇒ea

syto

por

t,al

ter,

read

,et

c.

•6th

editio

n(“

V6”

)w

asw

idel

yav

aila

ble

(197

6).

–so

urc

eav

ail⇒

peo

ple

could

write

new

tool

s.

–nic

efe

ature

sof

other

OSes

rolle

din

prom

ptly.

•B

y19

78,V

7av

aila

ble

(for

bot

hth

e16

-bit

PD

P-1

1an

dth

enew

32-b

itVA

X-1

1).

•Sin

ceth

en,tw

om

ain

fam

ilies

:

–AT

&T

:“S

yste

mV

”,cu

rren

tly

SV

R4.

–B

erke

ley:

“BSD

”,cu

rren

tly

4.3B

SD

/4.4

BSD

.

•Sta

ndar

disat

ion

effor

ts(e

.g.

PO

SIX

,X

/OP

EN

)to

hom

ogen

ise.

•B

est

know

n“U

NIX

”to

day

ispr

obab

lylinux,

but

also

get

Fre

eBSD

,N

etB

SD

,an

d(c

omm

erci

ally

)Sola

ris,

OSF/1

,IR

IX,an

dTru

64.

Unix

Cas

eStu

dy—

Intr

oduct

ion

109

Unix

Fam

ily

Tre

e(S

implified)

Sys

tem

V

SV

R2

SV

R3

SV

R4

4.2

BS

D

4.3

BS

D

4.3

BS

D/T

ah

oe

4.3

BS

D/R

eno

4.4

BS

D

Eig

hth

Ed

itio

n

Nin

th E

dit

ion

Ten

th E

dit

ion

Ma

ch

OS

F/1

Su

nO

S 4

So

lari

s

So

lari

s 2

Su

nO

S

Su

nO

S 3

Fir

st E

dit

ion

Fif

th E

dit

ion

Six

th E

dit

ion

Sev

enth

Ed

itio

n

3B

SD

4.0

BS

D

4.1

BS

D

Sys

tem

III

32

V

19

74

19

75

19

77

19

83

19

84

19

85

19

86

19

87

19

88

19

89

19

90

19

91

19

92

19

93

19

69

19

73

19

76

19

78

19

79

19

80

19

81

19

82

Unix

Cas

eStu

dy—

Intr

oduct

ion

110

Design

Featu

res

Ritch

iean

dT

hom

pso

nw

riting

inC

AC

M,Ju

ly74

,id

entified

the

follo

win

g(n

ew)

feat

ure

sof

UN

IX:

1.A

hie

rarc

hic

alfile

syst

emin

corp

orat

ing

dem

ounta

ble

volu

mes

.

2.C

ompat

ible

file

,dev

ice

and

inte

r-pr

oce

ssI/

O.

3.T

he

abili

tyto

initia

teas

ynch

ronou

spr

oce

sses

.

4.Sys

tem

com

man

dla

ngu

age

sele

ctab

leon

aper

-use

rbas

is.

5.O

ver

100

subsy

stem

sin

cludin

ga

doz

enla

ngu

ages

.

6.A

hig

hdeg

ree

ofpor

tabili

ty.

Fea

ture

sw

hic

hw

ere

not

incl

uded

:

•re

altim

e

•m

ultip

roce

ssor

suppor

t

Fix

ing

the

abov

eis

pret

tyhar

d.

Unix

Cas

eStu

dy—

Ove

rvie

w11

1

Str

uct

ura

lO

verv

iew

Syste

m C

all

Inte

rfa

ce

Ap

plic

atio

n

(Pro

ce

ss)

Ap

plic

atio

n

(Pro

ce

ss)

Ap

plic

atio

n

(Pro

ce

ss)

Ke

rne

l

Use

r

Ha

rdw

are

Pro

cess

Man

ag

em

en

tM

em

ory

Man

ag

em

en

t Blo

ck I

/OC

ha

r I/

O

File S

yste

m

Devic

e D

river

Devic

e D

river

Devic

e D

river

Devic

e D

river

•C

lear

separ

atio

nbet

wee

nuse

ran

dke

rnel

por

tion

s.

•P

roce

sses

are

unit

ofsc

hed

ulin

gan

dpr

otec

tion

.

•A

llI/

Olo

oks

like

oper

atio

ns

onfile

s.

Unix

Cas

eStu

dy—

Ove

rvie

w11

2

File

Abst

ract

ion

•A

file

isan

unst

ruct

ure

dse

quen

ceof

byte

s.

•R

epre

sente

din

use

r-sp

ace

byafile

descriptor

(fd)

•O

per

atio

ns

onfile

sar

e:

–fd

=open

(pathnam

e,mode)

–fd

=cr

eat(pathnam

e,mode)

–by

tes

=re

ad(fd,buffer

,nbytes)

–co

unt

=w

rite

(fd,buffer

,nbytes)

–re

ply

=se

ek(fd,off

set,whence

)

–re

ply

=cl

ose

(fd)

•D

evic

esre

pres

ente

dby

spec

ialfile

s:

–su

ppor

tab

ove

oper

atio

ns,

alth

ough

per

hap

sw

ith

biz

arre

sem

antics

.

–al

sohav

eioctl’s

:al

low

acce

ssto

dev

ice-

spec

ific

funct

ional

ity.

•H

iera

rchic

alst

ruct

ure

suppor

ted

bydirec

tory

file

s.

Unix

Cas

eStu

dy—

File

san

dth

eFile

syst

em11

3

Direct

ory

Hie

rarc

hy

/

etc/

bin/

usr/

dev/

home/

steve/

unix.ps

index.html

jean/

hda

hdb

tty

•D

irec

tories

map

nam

esto

file

s(a

nd

direc

tories

).

•H

ave

distingu

ished

root

direc

tory

calle

d’/

’

•Fully

qual

ified

pat

hnam

es⇒

per

form

trav

ersa

lfr

omro

ot.

•Eve

rydirec

tory

has

’.’

and

’..’

entr

ies:

refe

rto

self

and

par

ent

resp

ective

ly.

•Shor

tcut:

curr

ent

wor

king

direc

tory

( cw

d).

•In

additio

nsh

ellpr

ovid

esac

cess

tohom

edirec

tory

as~usernam

e(e

.g.~st

eve/

)

Unix

Cas

eStu

dy—

File

san

dth

eFile

syst

em11

4

Aside:

Pass

word

File

•/etc/passwd

hol

ds

list

ofpas

swor

den

trie

s.

•Eac

hen

try

rough

lyof

the

form

:

user-nam

e:encrypted-passw

d:hom

e-directory

:shell

•U

seon

e-w

ayfu

nct

ion

toen

cryp

tpas

swor

ds.

–i.e.

afu

nct

ion

whic

his

easy

toco

mpute

inon

edirec

tion,but

has

ahar

dto

com

pute

inve

rse

(e.g

.per

son

tophon

e-num

ber

look

up).

•To

logi

n:

1.G

etuse

rnam

e

2.G

etpas

swor

d

3.Encr

ypt

pas

swor

d

4.C

hec

kag

ainst

vers

ion

in/etc/password

5.If

ok,in

stan

tiat

elo

gin

shel

l.

•P

ublic

lyre

adab

lesince

lots

ofuse

fulin

foth

ere.

•Pro

ble

m:

off-lin

eat

tack

.

•Solu

tion:

shad

owpas

swor

ds

(/etc/shadow)

Unix

Cas

eStu

dy—

File

san

dth

eFile

syst

em11

5

File

Sys

tem

Imple

menta

tion

typ

em

od

e

tim

es

tam

ps

(x

3)

dir

ec

t b

loc

ks

(x

12

)

sin

gle

in

dir

ec

t

do

ub

le i

nd

ire

ct

trip

le i

nd

ire

ct

dir

ec

t

blo

ck

s

(51

2)

da

ta

da

ta

da

ta

da

ta

da

ta

da

ta

to b

lock w

ith

51

2

sin

gle

in

dire

ct

en

trie

s

to b

lock w

ith

51

2

do

ub

le in

dire

ct

en

trie

s

us

eri

dg

rou

pid

siz

en

blo

ck

s

nli

nk

sfl

ag

s

•In

kern

el,a

file

isre

pres

ente

dby

adat

ast

ruct

ure

calle

dan

index

-node

ori-node.

•H

olds

file

met

a-dat

a:

a)O

wner

,per

mission

s,re

fere

nce

count,

etc.

b)

Loca

tion

ondisk

ofac

tual

dat

a(fi

leco

nte

nts

).

•Q

uest

ion:

Wher

eis

the

file

nam

eke

pt?

Unix

Cas

eStu

dy—

File

san

dth

eFile

syst

em11

6

Direct

ories

and

Lin

ks

ho

me

/

ste

ve

/je

an

/

/

do

c/

. ..

unix.ps

index.html

214

78

385

56

Fil

en

am

eI-

No

de

misc

47

. ..

unix.ps

hello.txt

2

78

107

13

Fil

en

am

eI-

No

de

mis

c/

ind

ex

.htm

lu

nix

.ps

he

llo

.tx

t

bin

/

•D

irec

tory

isa

file

whic

hm

aps

file

nam

esto

i-nodes

.

•A

nin

stan

ceof

afile

ina

direc

tory

isa

(har

d)

link.

•(t

his

isw

hy

hav

ere

fere

nce

count

ini-node)

.

•D

irec

tories

can

hav

eat

mos

t1

(rea

l)lin

k.W

hy?

•A

lso

get

soft

-or

sym

bol

ic-lin

ks:

a‘n

orm

al’file

whic

hco

nta

ins

afile

nam

e.

Unix

Cas

eStu

dy—

File

san

dth

eFile

syst

em11

7

On-D

isk

Str

uct

ure

s

Super-Block

Boot-Block

Ino

de

Ta

ble

Da

ta

Blo

ck

s

Super-Block

Ino

de

Ta

ble

Da

ta

Blo

ck

s

Part

itio

n 1

Part

itio

n 2

Ha

rd D

isk

01

2i

i+1

jj+

1j+

2l

l+1

m

•A

disk

ism

ade

up

ofa

boot

blo

ckfo

llow

edby

one

orm

ore

par

tition

s.

•(a

par

tition

isa

contigu

ous

range

ofN

fixe

d-s

ize

blo

cks

ofsize

kfo

rso

meN

,k).

•A

Unix

file

-sys

tem

resides

within

apar

tition

.

•T

he

file

-sys

tem

super

blo

ckco

nta

ins

info

such

as:

–num

ber

ofblo

cks

infile

-sys

tem

–num

ber

offr

eeblo

cks

infile

-sys

tem

–st

art

ofth

efr

ee-b

lock

list

–st

art

ofth

efr

ee-inode

list.

–va

riou

sbook

keep

ing

info

rmat

ion.

Unix

Cas

eStu

dy—

File

san

dth

eFile

syst

em11

8

Mounting

File-S

yste

ms

/

etc/

bin/

usr/

dev/

home/

hda1hda2hdb1

steve/

jean/

/Mount

Point

Root File-System

File-System

on /dev/hda2

•Entire

file

-sys

tem

sca

nbe

mou

nte

don

anex

isting

direc

tory

inan

alre

ady

mou

nte

dfile

syst

em.

•A

tve

ryst

art,

only‘/’

exists⇒

nee

dto

mou

nt

aro

otfile

-sys

tem

.

•Subse

quen

tly

can

mou

nt

other

file

-sys

tem

s,e.

g.mount("/dev/hda2",

"/home",

options)

•P

rovi

des

aunifi

ednam

e-sp

ace:

e.g.

acce

ss/home/steve/

direc

tly.

•C

annot

hav

ehar

dlin

ksac

ross

mou

nt

poi

nts

:w

hy?

•W

hat

abou

tso

ftlin

ks?

Unix

Cas

eStu

dy—

File

san

dth

eFile

syst

em11

9

In-M

em

ory

Table

s

Process A

Process B

0 1 2 3 4 N

11 3 25

17 1 6

0 1 2 3 4 N

2 2762 5 17

32

047

1

17

135

78

pro

ce

ss-s

pe

cific

file

ta

ble

s

syste

m-w

ide

op

en

file

ta

ble

Inode 78

acitve

in

od

e t

ab

le

•R

ecal

lpr

oce

ssse

esfile

sas

file

des

crip

tors

•In

imple

men

tation

thes

ear

eju

stin

dic

esin

toa

proce

ss-s

pec

ific

open

file

table

.

•Entr

ies

poin

tto

syst

em-w

ide

open

file

table

.W

hy?

•T

hes

ein

turn

poi

nt

to(in

mem

ory)

inode

table

.

Unix

Cas

eStu

dy—

File

san

dth

eFile

syst

em12

0

Acc

ess

Contr

ol

Ow

ner

Gro

up

Wo

rld

RW

ER

WE

RW

E

= 0640

Ow

ner

Gro

up

Wo

rld

RW

ER

WE

RW

E

= 0755

•A

cces

sco

ntr

olin

form

atio

nhel

din

each

inode.

•T

hre

ebits

for

each

ofow

ner

,gr

oup

and

wor

ld:{

read

,w

rite

and

exec

ute}

•Q

uest

ion:

What

do

thes

em

ean

for

direc

tories

?

•In

additio

nhav

ese

tuid

and

setg

idbits:

–nor

mal

lypr

oce

sses

inher

itper

mission

sof

invo

king

use

r.

–se

tuid

/set

gid

allo

wth

euse

rto

“bec

ome”

som

eone

else

when

runnin

ga

par

ticu

lar

prog

ram

.

–e.

g.prof

owns

bot

hex

ecuta

bletest

(0711

and

setu

id),

andscore

file

(0600)

⇒an

yuse

rca

nru

nit.

⇒it

can

updat

escore

file

.⇒

but

use

rsca

n’t

chea

t.

•Q

uest

ion:

and

what

dothese

mea

nfo

rdirec

tories

?

Unix

Cas

eStu

dy—

File

san

dth

eFile

syst

em12

1

Consist

ency

Issu

es

•To

del

ete

afile

,use

theunlink

syst

emca

ll.

•Fro

mth

esh

ell,

this

isrm

<filename>

•P

roce

dure

is:

1.ch

eck

ifuse

rhas

suffi

cien

tper

mission

son

the

file

(must

hav

ew

rite

acce

ss).

2.ch

eck

ifuse

rhas

suffi

cien

tper

mission

son

the

direc

tory

(must

hav

ew

rite

acce

ss).

3.if

ok,re

mov

een

try

from

direc

tory

.

4.D

ecre

men

tre

fere

nce

count

onin

ode.

5.if

now

zero

:

a.fr

eedat

ablo

cks.

b.

free

inode.

•If

the

syst

emcr

ashes

:m

ust

chec

ken

tire

file

-sys

tem

:

–ch

eck

ifan

yblo

ckunre

fere

nce

d.

–ch

eck

ifan

yblo

ckdou

ble

refe

rence

d.

•(W

e’ll

see

mor

eon

this

late

r)

Unix

Cas

eStu

dy—

File

san

dth

eFile

syst

em12

2

Unix

File-S

yste

m:

Sum

mar

y

•File

sar

eunst

ruct

ure

dby

test

ream

s.

•Eve

ryth

ing

isa

file

:‘n

orm

al’file

s,direc

tories

,sy

mbol

iclin

ks,sp

ecia

lfile

s.

•H

iera

rchy

built

from

root

(‘/’

).

•U

nifi

ednam

e-sp

ace

(multip

lefile

-sys

tem

sm

aybe

mou

nte

don

any

leaf

direc

tory

).

•Low

-lev

elim

ple

men

tation

bas

edar

ound

inodes

.

•D

isk

conta

ins

list

ofin

odes

(alo

ng

with,of

cours

e,ac

tual

dat

ablo

cks)

.

•P

roce

sses

see

file

des

crip

tors

:sm

allin

tege

rsw

hic

hm

apto

syst

emfile

table

.

•Per

mission

sfo

row

ner

,gr

oup

and

ever

yone

else

.

•Set

uid

/set

gid

allo

wfo

rm

ore

flex

ible

contr

ol.

•C

are

nee

ded

toen

sure

consist

ency

.

Unix

Cas

eStu

dy—

File

san

dth

eFile

syst

em12

3

Unix

Pro

cess

es

Unix

Ker

nel

Ad

dre

ss S

pace

per

Pro

cess

Tex

t S

egm

ent

Da

ta S

egm

ent

Sta

ck S

egm

ent

Fre

eS

pace

gro

ws

do

wn

wa

rd a

s

fun

ctio

ns

are

ca

lled

gro

ws

up

wa

rds

as

mo

re

mem

ory

all

oca

ted

Ker

nel

Addre

ss S

pace

(sh

are

d b

y all

)

•R

ecal

l:a

proce

ssis

apr

ogra

min

exec

ution

.

•H

ave

thre

ese

gmen

ts:text,data

andstack.

•U

nix

proce

sses

are

hea

vyw

eigh

t.

Unix

Cas

eStu

dy—

Pro

cess

es12

4

Unix

Pro

cess

Dynam

ics execve

exit

fork

wait

pa

ren

tp

roce

ss

pro

gra

m e

xec

ute

s

chil

dp

roce

sszo

mb

iep

roce

ss

pa

ren

t p

roce

ss

(po

ten

tia

lly)

co

nti

nu

es

•P

roce

ssre

pres

ente

dby

aprocess

id(p

id)

•H

iera

rchic

alsc

hem

e:par

ents

crea

tech

ildre

n.

•Fou

rbas

icpr

imitiv

es:

–pid

=fo

rk()

–re

ply

=exe

cve(pathnam

e,argv

,envp

)

–exi

t(status)

–pid

=wait

(status)

•fo

rk()

nea

rlyalways

follo

wed

byexe

c()

⇒vf

ork

()an

d/o

rC

OW

.

Unix

Cas

eStu

dy—

Pro

cess

es12

5

Sta

rtofD

ay

•K

ernel

(/vmunix)

load

edfr

omdisk

(how

?)an

dex

ecution

star

ts.

•R

oot

file

-sys

tem

mou

nte

d.

•P

roce

ss1

(/etc/init)

han

d-c

raft

ed.

•in

itre

ads

file/etc/inittab

and

for

each

entr

y:

1.op

ens

term

inal

spec

ialfile

(e.g

./dev/tty0)

2.duplic

ates

the

resu

ltin

gfd

twic

e.

3.fo

rks

an/etc/tty

proce

ss.

•ea

chtt

ypr

oce

ssnex

t:

1.in

itia

lises

the

term

inal

2.ou

tputs

the

string

“login:”

&w

aits

for

input

3.ex

ecve

()’s/bin/login

•lo

gin

then

:

1.ou

tputs

“password:”

&w

aits

for

input

2.en

cryp

tspas

swor

dan

dch

ecks

itag

ainst

/etc/passwd.

3.if

ok,se

tsuid

&gi

d,an

dex

ecve

()’s

shel

l.

•Pat

riar

chin

itre

surr

ects

/etc/tty

onex

it.

Unix

Cas

eStu

dy—

Pro

cess

es12

6

The

Shell

execve

chil

dp

roce

ss

pro

gra

m

exec

ute

sfg

?

rep

eat

ad

infi

nit

um

yes

no

fork

read

get

co

mm

an

d l

ine

issu

e p

rom

pt

write

exit

wait

zom

bie

pro

cess

•T

he

shel

lju

sta

proce

sslik

eev

eryt

hin

gel

se.

•U

ses

pat

h(=

list

ofdirec

tories

tose

arch

)fo

rco

nve

nie

nce

.

•C

onve

ntion

ally

‘&’sp

ecifi

esru

nin

bac

kgro

und.

•Par

sing

stag

e(o

mitte

d)

can

do

lots

...

Unix

Cas

eStu

dy—

Pro

cess

es12

7

Shell

Exa

mple

s

#pwd

/home/steve

#ls

-F

IRAM.micro.ps

gnome_sizes

prog-nc.ps

Mail/

ica.tgz

rafe/

OSDI99_self_paging.ps.gz

lectures/

rio107/

TeX/

linbot-1.0/

src/

adag.pdf

manual.ps

store.ps.gz

docs/

past-papers/

wolfson/

emacs-lisp/

pbosch/

xeno_prop/

fs.html

pepsi_logo.tif

#cd

src/

#pwd

/home/steve/src

#ls

-F

cdq/

emacs-20.3.tar.gz

misc/

read_mem.c

emacs-20.3/

ispell/

read_mem*

rio007.tgz

#wc

read_mem.c

95

225

2262

read_mem.c

#ls

-lF

r*

-rwxrwxr-x

1steve

user

34956

Mar

21

1999

read_mem*

-rw-rw-r--

1steve

user

2262

Mar

21

1999

read_mem.c

-rw-------

1steve

user

28953

Aug

27

17:40

rio007.tgz

#ls

-l

/usr/bin/X11/xterm

-rwxr-xr-x

2root

system

164328

Sep

24

18:21

/usr/bin/X11/xterm*

•P

rom

pt

is‘#

’.

•U

seman

tofind

out

abou

tco

mm

ands.

•U

ser

frie

ndly

?

Unix

Cas

eStu

dy—

Pro

cess

es12

8

Sta

ndar

dI/

O

•Eve

rypr

oce

sshas

thre

efd

son

crea

tion

:

–st

din

:w

her

eto

read

input

from

.

–st

dout:

wher

eto

send

outp

ut.

–st

derr

:w

her

eto

send

dia

gnos

tics

.

•N

orm

ally

inher

ited

from

par

ent,

but

shel

lal

low

sre

direc

tion

to/f

rom

afile

,e.

g.:

–ls

>listing.txt

–ls

>&listing.txt

–sh

<commands.sh.

•A

ctual

file

not

alw

ays

appr

opriat

e;e.

g.co

nsider

:

ls

>temp.txt;

wc

<temp.txt

>results

•P

ipel

ine

isbet

ter

(e.g

.ls

|wc

>results)

•M

ost

Unix

com

man

ds

are

filter

s,i.e.

read

from

stdin

and

outp

ut

tost

dout⇒

can

build

alm

ost

arbitra

rily

com

ple

xco

mm

and

lines

.

•R

edirec

tion

can

cause

som

ebuffer

ing

subtlet

ies.

Unix

Cas

eStu

dy—

Pro

cess

es12

9

Pip

es

Pro

cess B

Pro

cess A

read(fd, buf, n)

write(fd, buf, n)

old

da

ta

ne

w d

ata

fre

e s

pa

ce

•O

ne

ofth

ebas

icU

nix

IPC

schem

es.

•Log

ical

lyco

nsist

sof

apai

rof

fds,

one

for

each

‘end’of

the

pip

e.

•e.

g.re

ply

=pip

e(

int

fds[

2])

•C

once

pt

of

“full”

and

“em

pty

”pip

es.

•O

nly

allo

ws

com

munic

atio

nbet

wee

npr

oce

sses

with

aco

mm

onan

cest

or(w

hy?

).

•N

amed

pip

esad

dre

ssth

is...

Unix

Cas

eStu

dy—

Inte

rpro

cess

Com

munic

atio

n13

0

Sig

nals

•P

roble

m:

pip

esnee

dpla

nnin

g⇒

use

sign

als.

•Sim

ilar

toa

(sof

twar

e)in

terr

upt.

•Exa

mple

s:

–SIGINT

:use

rhit

Ctr

l-C

.

–SIGSEGV

:pr

ogra

mer

ror.

–SIGCHLD

:a

dea

thin

the

fam

ily...

–SIGTERM

:...or

clos

erto

hom

e.

•U

nix

allo

ws

proce

sses

toca

tch

sign

als.

•e.

g.Jo

bco

ntr

ol:

–SIGTTIN,SIGTTOU

sent

tobg

proce

sses

–SIGCONT

turn

sbg

tofg

.

–SIGSTOP

does

the

reve

rse.

•C

annot

catc

hSIGKILL

(hen

cekill

-9)

•Sig

nal

sca

nal

sobe

use

dfo

rtim

ers,

win

dow

resize

,pr

oce

sstr

acin

g,...

Unix

Cas

eStu

dy—

Inte

rpro

cess

Com

munic

atio

n13

1

I/O

Imple

menta

tion

Ha

rdw

are

Devic

e D

river

Devic

e D

river

Devic

e D

river

Devic

e D

river

Ge

ne

ric F

ile S

yste

m L

aye

r

Buffer

Cache

Raw

Blo

ck I/O

Raw

Chara

cte

r I/O

Cooked

Chara

cte

r I/O

Ke

rne

l

Ke

rne

l

Use

r

•R

ecal

l:

–ev

eryt

hin

gac

cess

edvi

ath

efile

syst

em.

–tw

obr

oad

cate

gories

:blo

ckan

dch

ar.

•Low

-lev

elst

uff

gory

and

mac

hin

edep

enden

t⇒

ignor

e.

•C

har

acte

rI/

Ois

low

rate

but

com

ple

x⇒

mos

tco

de

inth

e“c

ook

ed”

inte

rfac

e.

•B

lock

I/O

sim

ple

rbut

per

form

ance

mat

ters⇒

emphas

ison

the

buffer

cach

e.

Unix

Cas

eStu

dy—

I/O

Subsy

stem

132

The

Buffer

Cach

e

•B

asic

idea

:ke

epco

pyof

som

epar

tsof

disk

inm

emor

yfo

rsp

eed.

•O

nre

addo:

1.Loca

tere

leva

nt

blo

cks

(fro

min

ode)

2.C

hec

kif

inbuffer

cach

e.

3.If

not

,re

adfr

omdisk

into

mem

ory.

4.R

eturn

dat

afr

ombuffer

cach

e.

•O

nw

rite

dosame

firs

tth

ree,

and

then

updat

eve

rsio

nin

cach

e,not

ondisk.

•“T

ypic

ally

”pr

even

ts85%

ofim

plie

ddisk

tran

sfer

s.

•Q

uest

ion:

when

does

dat

aac

tual

lyhit

disk?

•A

nsw

er:

callsync

ever

y30

seco

nds

toflush

dirty

buffer

sto

disk.

•C

anca

che

met

adat

ato

o—

prob

lem

s?

Unix

Cas

eStu

dy—

I/O

Subsy

stem

133

Unix

Pro

cess

Sch

eduling

•P

rior

itie

s0–

127;

use

rpr

oce

sses≥

PUSER

=50

.

•R

ound

robin

within

prio

rities

,quan

tum

100m

s.

•P

rior

itie

sar

ebas

edon

usa

gean

dnic

eva

lue,

i.e.

Pj(i)=

Base

j+

CPUj(i−

1)

4+

2×

nice j

give

sth

epr

iority

ofpr

oce

ssj

atth

ebeg

innin

gof

inte

rval

iw

her

e:

CPUj(i)=

2×

loadj

(2×

loadj)+

1CPUj(i−

1)+

nice j

andnice j

isa

(par

tial

ly)

use

rco

ntr

olla

ble

adju

stm

ent

par

amet

er∈[−

20,20]

.

•loadj

isth

esa

mple

dav

erag

ele

ngt

hof

the

run

queu

ein

whic

hpr

oce

ssj

resides

,ov

erth

ela

stm

inute

ofop

erat

ion

•so

ife.

g.lo

adis

1⇒

appr

oxim

atel

y90

%of

1se

conds

CP

Uusa

gew

illbe

“for

gott

en”

within

5se

conds.

Unix

Cas

eStu

dy—

Pro

cess

Sch

edulin

g13

4

Unix

Pro

cess

Sta

tes

fork()

ru

rkp

rb

z sl

c

sch

edu

le

wa

keu

p

slee

p

inte

rru

pt

exit

sysc

all

retu

rnre

turn

pre

emp

t

sam

est

ate

ru=

runnin

g(u

ser-

mode)

rk=

runnin

g(k

ernel

-mode)

z=

zom

bie

p=

pre-

empte

d

sl=

slee

pin

grb

=ru

nnab

le

c=

crea

ted

•N

ote

:ab

ove

issim

plifi

ed—

see

CS

sect

ion

23.1

4fo

rdet

aile

ddes

crip

tion

sof

allst

ates

/tra

nsition

s.

Unix

Cas

eStu

dy—

Pro

cess

Sch

edulin

g13

5

Sum

mar

y

•M

ain

Unix

feat

ure

sar

e:

–file

abst

ract

ion

∗a

file

isan

unst

ruct

ure

dse

quen

ceof

byte

s∗

(not

real

lytr

ue

for

dev

ice

and

direc

tory

file

s)

–hie

rarc

hic

alnam

espac

e

∗direc

ted

acyc

licgr

aph

(if

excl

ude

soft

links

)∗

can

recu

rsiv

ely

mou

nt

file

syst

ems

–hea

vy-w

eigh

tpr

oce

sses

–IP

C:pip

es&

sign

als

–I/

O:blo

ckan

dch

arac

ter

–dyn

amic

prio

rity

sched

ulin

g

∗bas

epr

iority

leve

lfo

ral

lpr

oce

sses

∗pr

iority

islo

wer

edif

proce

ssge

tsto

run

∗ov

ertim

e,th

epas

tis

forg

otte

n

•B

ut

Unix

V7

had

inflex

ible

IPC

,in

effici

ent

mem

ory

man

agem

ent,

and

poor

kern

elco

ncu

rren

cy.

•Lat

erve

rsio

ns

addre

ssth

ese

issu

es.

Unix

Cas

eStu

dy—

Sum

mar

y13

6

Win

dow

sN

T:H

isto

ry

Aft

erO

S/2

,M

Sdec

ide

they

nee

d“N

ewT

echnol

ogy”

:

•19

88:

Dav

eC

utler

recr

uited

from

DEC

.

•19

89:

team

(∼10

peo

ple

)st

arts

wor

kon

anew

OS

(mic

ro-k

ernel

arch

itec

ture

)

•Ju

ly19

93:

firs

tve

rsio

n(3

.1)

intr

oduce

d

•(n

ame

com

pat

ible

with

win

dow

s3.

1)

Blo

ated

and

suck

ful⇒

•N

T3.

5re

leas

edin

Sep

tem

ber

1994

:m

ainly

size

and

per

form

ance

optim

isat

ions.

•Fol

low

edin

May

1995

byN

T3.

51

(suppor

tfo

rth

ePow

erP

C,an

dm

ore

per

form

ance

twea

ks)

•Ju

ly19

96:

NT

4.0

–new

(win

dow

s95

)lo

ok’n

feel

–so

me

des

ktop

use

rsbut

mos

tly

limited

tose

rver

s

–fo

rper

form

ance

reas

ons,

variou

sfu

nct

ions

push

edbac

kin

toke

rnel

(mos

tnot

ably

grap

hic

sre

nder

ing

funct

ions)

–ongo

ing

upgra

des

via

serv

ice

pac

ks

NT

Cas

eStu

dy—

Intr

oduct

ion

&O

verv

iew

137

Win

dow

sN

T:Evo

lution

•Feb

2000

:N

T5.

0ak

aW

indow

s20

00

–bor

row

sfr

omw

indow

s98

look

’nfe

el

–bot

hse

rver

and

wor

ksta

tion

vers

ions,

latt

erof

whic

hst

arts

toge

tw

ider

use

–big

push

tofinal

lyki

llD

OS/W

in9x

fam

ily(b

ut

fails

due

toin

tern

alpol

itic

king)

•W

indow

sX

P(N

T5.

1)

launch

edO

ctob

er20

01

–hom

ean

dpr

ofes

sion

al⇒

final

lyki

llsw

in9x

.

–va

riou

s“e

ditio

ns”

( med

iace

nte

r,64

-bit)

&se

rvic

epac

ks(S

P1,

SP

2,SP

3)

•Ser

ver

product

Win

dow

sSer

ver

2003

(NT

5.2)

rele

ased

2003

–bas

ical

lyth

esa

me

modulo

regi

stry

twea

ks,su

ppor

tco

ntr

act

and

ofco

urs

eco

st

–a

ple

thor

aof

editio

ns.

..

•W

indow

sV

ista

(NT

6.0)

limped

onto

the

scen

eQ

420

06

–new

Aero

UI,

new

WinFX

AP

I

–m

issing

Lon

ghor

nbits

likeWinFS,Msh

•W

indow

sSer

ver

2008

(also

bas

edon

NT

6.0,but

good)

landed

Feb

200

8

•W

indow

s7

(NT

6.1

for

now

)re

leas

edO

ctob

er200

9...

NT

Cas

eStu

dy—

Intr

oduct

ion

&O

verv

iew

138

NT

Design

Princi

ple

s

Key

goal

sfo

rth

esy

stem

wer

e:

•por

tabili

ty

•se

curity

•P

OSIX

com

plia

nce

•m

ultip

roce

ssor

suppor

t

•ex

tensibili

ty

•in

tern

atio

nal

suppor

t

•co

mpat

ibili

tyw

ith

MS-D

OS/W

indow

sap

plic

atio

ns

This

led

toth

edev

elop

men

tof

asy

stem

whic

hw

as:

•w

ritt

enin

hig

h-lev

ella

ngu

ages

(Can

dC

++

)

•bas

edar

ound

am

icro

-ker

nel

,an

d

•co

nst

ruct

edin

ala

yere

d/m

odula

rfa

shio

n.

NT

Cas

eStu

dy—

Intr

oduct

ion

&O

verv

iew

139

Str

uct

ura

lO

verv

iew

OS

/2

Su

bsyte

m

OS

/2A

pp

lic

ati

on

s

Win

32

Ap

pli

ca

tio

ns

Ke

rne

l M

od

e

Use

r M

od

e

Ha

rdw

are

Na

tiv

e N

T I

nte

rfa

ce

(S

yte

m C

all

s)

Ob

ject

Man

ag

er

Pro

cess

Man

ag

er

VM

Man

ag

er

I/O

Man

ag

er

Win

32

Su

bs

yte

m

PO

SIX

Su

bsyte

m

Se

cu

rity

Su

bsyte

m

MS

-DO

SA

pp

lic

ati

on

s

Po

six

Ap

pli

ca

tio

ns

Win

16

Ap

pli

ca

tio

ns

Lo

go

nP

roc

es

s

MS

-DO

S

Su

bsyte

m

Win

16

Su

bsyte

m

ER

NE

LK

EV

ICE

DH

ard

wa

re A

bstr

actio

n L

aye

r (H

AL

)R

IVE

RS

D

File S

yste

mD

rivers

Cach

eM

an

ag

er

Secu

rity

Man

ag

er

LP

CF

acilit

y

XE

CU

TIV

EE

•K

ernel

Mode:

HA

L,K

ernel

,&

Exe

cutive

•U

ser

Mode:

envi

ronm

enta

lsu

bsy

stem

s,pr

otec

tion

subsy

stem

NT

Cas

eStu

dy—

Intr

oduct

ion

&O

verv

iew

140

HA

L

•Lay

erof

soft

war

e(HAL.DLL)

whic

hhid

esdet

ails

ofunder

lyin

ghar

dw

are

•e.

g.lo

w-lev

elin

terr

upt

mec

han

ism

s,D

MA

contr

olle

rs,m

ultip

roce

ssor

com

munic

atio

nm

echan

ism

s

•Sev

eral

HA

Ls

exist

with

sam

ein

terf

ace

but

diff

eren

tim

ple

men

tation

(oft

enve

ndor

-spec

ific,

e.g.

for

larg

ecc

-NU

MA

mac

hin

es)

Kern

el

•Fou

ndat

ion

for

the

exec

utive

and

the

subsy

stem

s

•Exe

cution

isnev

erpr

eem

pte

d.

•Fou

rm

ain

resp

onsibili

ties

:

1.C

PU

sched

ulin

g2.

inte

rrupt

and

exce

ption

han

dlin

g3.

low

-lev

elpr

oce

ssor

synch

ronisat

ion

4.re

cove

ryaf

ter

apow

erfa

ilure

•K

ernel

isob

ject

ed-o

rien

ted;al

lob

ject

sar

eei

ther

dispat

cher

obje

cts

(act

ive

orte

mpor

alth

ings)

orco

ntr

olob

ject

s(e

very

thin

gel

se)

NT

Cas

eStu

dy—

Low

-lev

elFunct

ions

141

Pro

cess

es

and

Thre

ads

NT

split

sth

e“v

irtu

alpr

oce

ssor

”in

totw

opar

ts:

1.A

pro

cess

isth

eunit

ofre

sourc

eow

ner

ship

.Eac

hpr

oce

sshas

:

•a

secu

rity

toke

n,

•a

virt

ual

addre

sssp

ace,

•a

set

ofre

sourc

es( o

bje

cthan

dle

s),an

d

•on

eor

mor

eth

read

s.

2.A

thre

ad

are

the

unit

ofdispat

chin

g.Eac

hth

read

has

:

•a

sched

ulin

gst

ate

(rea

dy,

runnin

g,et

c.),

•ot

her

sched

ulin

gpar

amet

ers

(prior

ity,

etc)

,

•a

conte

xtslot

,an

d

•(g

ener

ally

)an

asso

ciat

edpr

oce

ss.

Thre

ads

are:

•co

-oper

ativ

e:al

lth

read

sin

apr

oce

sssh

are

addre

sssp

ace

&ob

ject

han

dle

s.

•lig

htw

eigh

t:re

quire

less

wor

kto

crea

te/d

elet

eth

anpr

oce

sses

(mai

nly

due

tosh

ared

virt

ual

addre

sssp

ace)

.

NT

Cas

eStu

dy—

Low

-lev

elFunct

ions

142

CPU

Sch

eduling

•H

ybrid

stat

ic/d

ynam

icpr

iority

sched

ulin

g:

–P

rior

itie

s16

–31:

“rea

ltim

e”(s

tatic

prio

rity

).

–P

rior

itie

s1–

15:

“var

iable

”(d

ynam

ic)

prio

rity

.

–(p

rior

ity

0is

rese

rved

for

zero

pag

eth

read

)

•D

efau

ltquan

tum

2tick

s(∼

20m

s)on

Wor

ksta

tion

,12

tick

s(∼

120m

s)on

Ser

ver.

•T

hre

ads

hav

ebas

ean

dcu

rren

t(≥

bas

e)pr

iorities

.

–O

nre

turn

from

I/O

,cu

rren

tpr

iority

isboos

ted

bydrive

r-sp

ecifi

cam

ount.

–Subse

quen

tly,

curr

ent

prio

rity

dec

ays

by1

afte

rea

chco

mple

ted

quan

tum

.

–A

lso

get

boost

for

GU

Ith

read

saw

aiting

input:

curr

ent

prio

rity

boos

ted

to14

for

one

quan

tum

(but

quan

tum

also

dou

ble

d)

–Yes

,th

isis

true.

•O

nW

orks

tation

also

get

quan

tum

stre

tchin

g:

–“.

..per

form

ance

boos

tfo

rth

efo

regr

ound

applic

atio

n”

(win

dow

with

focu

s)

–fg

thre

adge

tsdou

ble

ortr

iple

quan

tum

.

•If

no

runnab

leth

read

,dispat

ch‘idle

’th

read

(whic

hex

ecute

sD

PC

s).

NT

Cas

eStu

dy—

Low

-lev

elFunct

ions

143

Obje

ctM

anager

Te

mp

ora

ry/P

erm

an

en

t

Ob

ject

Head

er

Ty

pe

Ob

jec

t

Ty

pe

Na

me

Co

mm

on

In

fo.

Open

Close

Delete

Parse

Security

Query Name

Ob

jec

t N

am

eO

bje

ct

Dir

ec

tory

Se

cu

rity

De

sc

rip

tor

Op

en

Ha

nd

le C

ou

nt

Re

fere

nc

e C

ou

nt

Ty

pe

Ob

jec

t P

oin

ter

Qu

ota

Ch

arg

es

Op

en

Ha

nd

les

Lis

t

Ob

ject

Bo

dy

Ob

jec

t-S

pe

cfi

c D

ata

(pe

rha

ps

in

clu

din

ga

ke

rne

l o

bje

ct)

Me

tho

ds

:

Pro

ce

ss

1P

roc

es

s2

Pro

ce

ss

3

•Eve

ryre

sourc

ein

NT

isre

pres

ente

dby

anob

ject

•T

he

Obje

ctM

anag

er(p

art

ofth

eExe

cutive

)is

resp

onsible

for:

–cr

eating

obje

cts

and

obje

cthan

dle

s

–per

form

ing

secu

rity

chec

ks

–tr

acki

ng

whic

hpr

oce

sses

are

using

each

obje

ct

•Typ

ical

oper

atio

n:

–handle

=open(objectname,

accessmode)

–result

=service(handle,

arguments)

NT

Cas

eStu

dy—

Exe

cutive

Funct

ions

144

Obje

ctN

am

esp

ace

\

??\

device\

BaseNamedObjects\

driver\ doc\

exams.tex

A:

C:

COM1:

Harddisk0\

Serial0\

Floppy0\

Partition1\

Partition2\

winnt\

temp\

•R

ecal

l:ob

ject

s(o

ption

ally

)hav

ea

nam

e

•O

bje

ctM

ange

rm

anag

esa

hie

rarc

hic

alnam

espac

e:

–sh

ared

bet

wee

nal

lpr

oce

sses⇒

shar

ing

–im

ple

men

ted

via

direc

tory

obje

cts

–ea

chob

ject

prot

ecte

dby

anac

cess

contr

ollis

t.

–nam

ing

dom

ains

(usingparse)

mea

nfile

-sys

tem

nam

espac

esca

nbe

inte

grat

ed

•A

lso

get

sym

bol

iclin

kob

ject

s:al

low

multip

lenam

es(a

liase

s)fo

rth

esa

me

obje

ct.

•M

odifi

edvi

ewpr

esen

ted

atA

PIle

vel.

..

NT

Cas

eStu

dy—

Exe

cutive

Funct

ions

145

Pro

cess

Manager

•P

rovi

des

serv

ices

for

crea

ting,

del

etin

g,an

dusing

thre

ads

and

proce

sses

.

•V

ery

flex

ible

:

–no

built

inco

nce

pt

ofpar

ent/

child

rela

tion

ship

sor

proce

sshie

rarc

hie

s

–pr

oce

sses

and

thre

ads

trea

ted

orth

ogon

ally.

⇒ca

nsu

ppor

tPos

ix,O

S/2

and

Win

32m

odel

s.

Virtu

alM

em

ory

Manager

•N

Tem

plo

yspag

edvi

rtual

mem

ory

man

agem

ent

•T

he

VM

Mpr

ovid

espr

oce

sses

with

serv

ices

to:

–al

loca

tean

dfr

eevi

rtual

mem

ory

–m

odify

per

-pag

epr

otec

tion

s

•C

anal

sosh

are

por

tion

sof

mem

ory:

–use

sect

ion

obje

cts

(≈so

ftw

are

segm

ents

)

–se

ctio

nob

ject

sar

eei

ther

bas

ed(s

pec

ific

bas

ead

dre

ss)

ornon

-bas

ed(fl

oating)

–al

souse

dfo

rm

emor

y-m

apped

file

s

NT

Cas

eStu

dy—

Exe

cutive

Funct

ions

146

Secu

rity

Refe

rence

Manager

•N

T’s

obje

ct-o

rien

ted

nat

ure

enab

les

aunifor

mm

echan

ism

for

runtim

eac

cess

and

audit

chec

ks

–ev

eryt

ime

apr

oce

ssop

ens

han

dle

toan

obje

ct,ch

eck

proce

ss’s

secu

rity

toke

nan

dob

ject

’sA

CL

–co

mpar

ew

ith

Unix

(file

-sys

tem

,net

wor

king,

win

dow

syst

em,sh

ared

mem

ory)

Loca

lPro

cedure

Call

Faci

lity

•LP

C(o

rIP

C)

pas

ses

reques

tsan

dre

sults

bet

wee

ncl

ient

and

serv

erpr

oce

sses

within

asingl

em

achin

e.

•U

sed

tore

ques

tse

rvic

esfr

om

the

variou

sN

Ten

viro

nm

enta

lsu

bsy

stem

s.

•T

hre

eva

rian

tsof

LP

Cch

annel

s:

1.sm

allm

essa

ges

(≤25

6by

tes)

:co

pym

essa

ges

bet

wee

npr

oce

sses

2.ze

roco

py:

avoi

dco

pyin

gla

rge

mes

sage

sby

poi

nting

toa

shar

edm

emor

yse

ctio

nob

ject

crea

ted

for

the

chan

nel

.

3.quic

kLP

C:use

dby

the

grap

hic

aldispla

ypor

tion

sof

the

Win

32su

bsy

stem

.

NT

Cas

eStu

dy—

Exe

cutive

Funct

ions

147

I/O

Manager

I/O

Man

ag

er

File

Syste

mD

river

Inte

rmed

iate

Dri

ver

Devic

eD

river

HA

L

I/O

Re

qu

es

ts

•T

he

I/O

Man

ager

isre

sponsible

for:

–file

syst

ems

–ca

che

man

agem

ent

–dev

ice

drive

rs

•B

asic

model

isas

ynch

ronous:

–ea

chI/

Oop

erat

ion

explic

itly

split

into

are

ques

tan

da

resp

onse

–an

I/O

Req

ues

tPac

ket

(IR

P)

use

dto

hol

dpar

amet

ers,

resu

lts,

etc.

•File

-sys

tem

&dev

ice

drive

rsar

est

acka

ble

...

NT

Cas

eStu

dy—

Exe

cutive

Funct

ions

148

Cach

eM

anager

•C

ache

Man

ager

cach

es“vi

rtual

blo

cks”

:

–vi

z.ke

eps

trac

kof

cach

e“l

ines

”as

offse

tsw

ithin

afile

rath

erth

ana

volu

me.

–disk

layo

ut

&vo

lum

eco

nce

pt

abst

ract

edaw

ay.

⇒no

tran

slat

ion

required

for

cach

ehit.

⇒ca

nge

tm

ore

inte

llige

nt

pref

etch

ing

•C

omple

tely

unifi

edca

che:

–ca

che

“lin

es”

allliv

ein

the

virt

ual

addre

sssp

ace.

–dec

ouple

sphys

ical

&vi

rtual

cach

esy

stem

s:e.

g.

∗vi

rtual

lyca

che

in25

6Kblo

cks,

∗phys

ical

lycl

ust

erup

to64

K.

–N

Tvi

rtual

mem

ory

man

ager

resp

onsible

for

actu

ally

doi

ng

the

I/O

.

–so

lots

ofFS

cach

ew

hen

VM

syst

emlig

htly

load

ed,lit

tle

when

syst

emth

rash

ing

•N

Tal

sopr

ovid

esso

me

use

rco

ntr

ol:

–if

spec

ifytemporary

attr

ibw

hen

crea

ting

file⇒

dat

aw

illnev

erbe

flush

edto

disk

unle

ssab

solu

tely

nec

essa

ry.

–if

spec

ifywritethrough

attr

ibw

hen

open

ing

afile⇒

allw

rite

sw

illsy

nch

ronou

sly

com

ple

te.

NT

Cas

eStu

dy—

Exe

cutive

Funct

ions

149

File

Sys

tem

s:FAT

16

A

B

8 74

EOF

Free

Free

Free

3

n-2

Free

EOF

Disk Info

0 1 2 6 7 8 9

n-1

File N

am

e (

8 B

yte

s)

AD

RV

SH

Exte

nsio

n (

3 B

yte

s)

Reserv

ed

(10 B

yte

s)

Tim

e (

2 B

yte

s)

Da

te (

2 B

yte

s)

Fir

st

Clu

ste

r (2

By

tes

)

File S

ize (

4 B

yte

s)

Att

rib

ute

Bit

s

A:

Arc

hiv

e

D:

Dir

ec

tory

V:

Vo

lum

e L

ab

el

S:

Sy

ste

m

H:

Hid

de

n

R:

Re

ad

-On

ly

•A

file

isa

linke

dlis

tof

clust

ers

(=a

set

of2n

contigu

ous

disk

blo

cks,n≥

0)

•Eac

hen

try

inth

eFAT

conta

ins

eith

er:

–th

ein

dex

ofan

other

entr

yw

ithin

theFAT,or

–a

spec

ialva

lueEOF

mea

nin

g“e

nd

offile

”,or

–a

spec

ialva

lueFree

mea

nin

g“f

ree”

.

•D

irec

tory

entr

ies

conta

inin

dex

into

theFAT

•FAT16

could

only

han

dle

par

tition

sup

to(2

16×

c)by

tes⇒

max

2Gb

par

tition

with

32K

clust

ers

(and

big

clust

ersize

isbad)

NT

Cas

eStu

dy—

Mic

roso

ftFile

Sys

tem

s15

0

File

Sys

tem

s:FAT

32

•O

bvi

ous

exte

nsion

:in

stea

dof

using

2by

tes

per

entr

y,FAT32

use

s4

byte

s

⇒ca

nsu

ppor

te.

g.8G

bpar

tition

with

4Kcl

ust

ers

•Furt

her

enhan

cem

ents

withFAT32

incl

ude:

–ca

nlo

cate

the

root

direc

tory

anyw

her

eon

the

par

tition

(inFAT16,th

ero

otdirec

tory

had

toim

med

iate

lyfo

llow

theFAT(s

)).

–ca

nuse

the

bac

kup

copy

ofth

eFAT

inst

ead

ofth

edef

ault

(mor

efa

ult

tole

rant)

–im

prov

edsu

ppor

tfo

rdem

and

pag

edex

ecuta

ble

s(c

onsider

the

4Kdef

ault

clust

ersize

...).

•VFAT

onto

pof

FAT32

adds

long

nam

esu

ppor

tan

din

tern

atio

nal

izat

ion:

–nam

esnow

unic

ode

strings

ofup

to25

6ch

arac

ters

.

–w

ant

toke

epsa

me

direc

tory

entr

yst

ruct

ure

for

com

pat

ibili

tyw

ith

e.g.

DO

S

⇒use

multiple

direc

tory

entr

ies

toco

nta

insu

cces

sive

par

tsof

nam

e.

–ab

use

Vat

trib

ute

toav

oid

listing

thes

e

Still

pret

typr

imitiv

e...

NT

Cas

eStu

dy—

Mic

roso

ftFile

Sys

tem

s15

1

File-S

yste

ms:

NT

FS

File

Re

co

rdM

aste

r F

ile

Tab

le (

MF

T)

0 1 2 3 4 5 6 7

16

17

$Mft

$MftMirr

$LogFile

$Volume

$AttrDef

\ $Bitmap

$BadClus

user

file

/directo

ry

user

file

/directo

ry

15

Sta

nd

ard

In

form

ati

on

Filen

am

e

Data

...

•Fundam

enta

lst

ruct

ure

ofN

TFS

isa

volu

me:

–bas

edon

alo

gica

ldisk

par

tition

–m

ayocc

upy

apor

tion

of

adisk,

and

entire

disk,

orsp

anac

ross

seve

raldisks

.

•N

TFS

stor

esal

lfile

reco

rds

ina

spec

ialfile

calle

dth

eM

aste

rFile

Tab

le(M

FT

).

•T

he

MFT

isin

dex

edby

afile

refe

rence

:a

64-b

ituniq

ue

iden

tifier

for

afile

•A

file

itse

lfis

ast

ruct

ure

dob

ject

consist

ing

ofse

tof

attr

ibute

/val

ue

pai

rsof

variab

lele

ngt

h...

NT

Cas

eStu

dy—

Mic

roso

ftFile

Sys

tem

s15

2

NT

FS:Reco

very

•To

aid

reco

very

,al

lfile

syst

emdat

ast

ruct

ure

updat

esar

eper

form

edin

side

tran

sact

ions:

–bef

ore

adat

ast

ruct

ure

isal

tere

d,th

etr

ansa

ctio

nw

rite

sa

log

reco

rdth

atco

nta

ins

redo

and

undo

info

rmat

ion.

–af

ter

the

dat

ast

ruct

ure

has

bee

nch

ange

d,a

com

mit

reco

rdis

writt

ento

the

log

tosign

ify

that

the

tran

sact

ion

succ

eeded

.

–af

ter

acr

ash,th

efile

syst

emca

nbe

rest

ored

toa

consist

ent

stat

eby

proce

ssin

gth

elo

gre

cord

s.

•D

oes

not

guar

ante

eth

atal

lth

euse

rfile

dat

aca

nbe

reco

vere

daf

ter

acr

ash

—ju

stth

atm

etad

ata

file

sw

illre

flec

tso

me

prio

rco

nsist

ent

stat

e.

•T

he

log

isst

ored

inth

eth

ird

met

adat

afile

atth

ebeg

innin

gof

the

volu

me

($Logfile)

–in

fact

,N

Thas

age

ner

iclo

gfile

serv

ice

⇒co

uld

inpr

inci

ple

be

use

dby

e.g.

dat

abas

e

•O

vera

llm

akes

for

far

quic

ker

reco

very

afte

rcr

ash

•(m

oder

nU

nix

fs[e

xt3,xf

s]use

sim

ilar

schem

e)

NT

Cas

eStu

dy—

Mic

roso

ftFile

Sys

tem

s15

3

NT

FS:Fault

Tole

rance

Part

itio

n A

1

Ha

rd D

isk

AH

ard

Dis

k B

Part

itio

n A

2

Part

itio

n A

3

Part

itio

n B

1

Part

itio

n B

2

•FtDisk

drive

ral

low

sm

ultip

lepar

tition

sbe

com

bin

edin

toa

logi

calvo

lum

e:

–e.

g.lo

gic

ally

conca

tenat

em

ultip

ledisks

tofo

rma

larg

elo

gica

lvo

lum

e

–bas

edon

the

conce

pt

ofR

AID

=R

edundan

tA

rray

ofIn

expen

sive

Disks

:

–e.

g.R

AID

leve

l0:

inte

rlea

vem

ultip

lepar

tition

sro

und-r

obin

tofo

rma

stripe

set:

∗lo

gica

lblo

ck0→

blo

ck0

ofpar

tition

A2,

logi

calblo

ck1→

blo

ck0

ofpar

tition

B2,

logi

calblo

ck2→

blo

ck1

ofpar

tition

A2,

etc

–e.

g.R

AID

leve

l1

incr

ease

sro

bust

nes

sby

using

am

irro

rse

t:tw

oeq

ual

lysize

dpar

titions

ontw

odisks

with

iden

tica

ldat

aco

nte

nts

.

–(o

ther

mor

eco

mple

xR

AID

leve

lsal

soex

ist)

•FtDisk

can

also

han

dle

sect

orsp

arin

gw

her

eth

eunder

lyin

gSC

SIdisk

suppor

tsit

•(if

not

,N

TFS

suppor

tscl

ust

erre

map

pin

gin

soft

war

e)

NT

Cas

eStu

dy—

Mic

roso

ftFile

Sys

tem

s15

4

NT

FS:O

ther

Featu

res

•Sec

urity

:

–se

curity

der

ived

from

the

NT

obje

ctm

odel

.–

each

file

obje

cthas

ase

curity

des

crip

tor

attr

ibute

stor

edin

its

MFT

reco

rd.

–th

isat

rrib

ute

hol

ds

the

acce

ssto

ken

offile

owner

plu

san

acce

ssco

ntr

ollis

t

•C

ompr

ession

:

–N

TFS

can

div

ide

afile

’sdat

ain

toco

mpr

ession

units

(set

sof

16co

ntigu

ous

clust

ers

inth

efile

)–

NT

FS

also

has

suppor

tfo

rsp

arse

file

s∗

clust

ers

with

allze

ros

not

actu

ally

allo

cate

dor

stor

edon

disk.

∗in

stea

d,gap

sar

ele

ftin

the

sequen

ces

ofV

CN

ske

pt

inth

efile

reco

rd∗

when

read

ing

afile

,ga

ps

cause

NT

FS

toze

ro-fi

llth

atpor

tion

of

the

calle

r’s

buffer

.

•Encr

yption

:

–U

sesy

mm

etric

key

toen

cryp

tfile

s;file

attr

ibute

hol

ds

this

key

encr

ypte

dw

ith

use

rpublic

key

–N

otre

ally

that

use

ful:

priv

ate

key

pret

tyea

syto

obta

in;an

dad

min

istr

ator

can

bypas

sen

tire

thin

gan

yhow

.

NT

Cas

eStu

dy—

Mic

roso

ftFile

Sys

tem

s15

5

Envi

ronm

enta

lSubsy

stem

s

•U

ser-

mode

proce

sses

laye

red

over

the

nat

ive

NT

exec

utive

serv

ices

toen

able

NT

toru

npr

ogr

ams

dev

elop

edfo

rot

her

oper

atin

gsy

stem

s.

•N

Tuse

sth

eW

in32

subsy

stem

asth

em

ain

oper

atin

gen

viro

nm

ent

–W

in32

isuse

dto

star

tal

lpr

oce

sses

.

–A

lso

prov

ides

allth

eke

yboa

rd,m

ouse

and

grap

hic

aldispla

yca

pab

ilities

.

•M

S-D

OS

envi

ronm

ent

ispr

ovid

edby

aW

in32

applic

atio

nca

lled

the

virt

ual

dos

mac

hin

e(V

DM

),a

use

r-m

ode

proce

ssth

atis

pag

edan

ddispat

ched

like

any

other

NT

thre

ad.

–U

ses

virt

ual

8086

mode,

sonot

100%

com

pat

ible

•16

-Bit

Win

dow

sEnvi

ronm

ent:

–P

rovi

ded

bya

VD

Mth

atin

corp

orat

esW

indow

son

Win

dow

s

–P

rovi

des

the

Win

dow

s3.1

kern

elro

utines

and

stub

routings

for

win

dow

man

ager

and

GD

Ifu

nct

ions.

•T

he

PO

SIX

subsy

stem

isdes

igned

toru

nP

OSIX

applic

atio

ns

follo

win

gth

eP

OSIX

.1st

andar

dw

hic

his

bas

edon

the

UN

IXm

odel

.

NT

Cas

eStu

dy—

Use

rM

ode

Com

pon

ents

156

Sum

mar

y

•M

ain

Win

dow

sN

Tfe

ature

sar

e:

–la

yere

d/m

odula

rar

chitec

ture

:

–ge

ner

icuse

ofob

ject

sth

rough

out

–m

ulti-th

read

edpr

oce

sses

–m

ultip

roce

ssor

suppor

t

–as

ynch

ronou

sI/

Osu

bsy

stem

–N

TFS

filin

gsy

stem

(vas

tly

super

ior

toFA

T32

)

–pr

eem

ptive

prio

rity

-bas

edsc

hed

ulin

g

•D

esig

nes

sential

lymore

advanced

than

Unix

.

•Im

ple

men

tation

oflo

wer

leve

ls(H

AL,ke

rnel

&ex

ecutive

)ac

tual

lyra

ther

dec

ent.

•B

ut:

has

histo

rica

llybee

ncr

ipple

dby

–al

mos

tex

clusive

use

ofW

in32

AP

I

–le

gacy

dev

ice

drive

rs(e

.g.V

XD

s)

–la

ckof

dem

and

for

“adva

nce

d”

feat

ure

s

–“fe

ature

inte

ract

ion”,

aka

huge

swat

hes

ofco

mple

xpoor

lyim

ple

men

ted

use

r-sp

ace

code

writt

enby

idio

ts

•C

ontinues

toev

olve

...

NT

Cas

eStu

dy—

Sum

mar

y15

7

Cours

eRevi

ew

•Par

tI:

Oper

atin

gSys

tem

Funct

ions

–O

Sst

ruct

ure

s:re

quired

h/w

suppor

t,ke

rnel

vs.µ-k

ernel

–P

roce

sses

:st

ates

,st

ruct

ure

s,sc

hed

ulin

g

–M

emor

y:vi

rtual

addre

sses

,sh

arin

g,pr

otec

tion

–I/

Osu

bsy

tem

:pol

ling/

inte

rrupts

,buffer

ing.

–Fili

ng:

direc

tories

,m

eta-

dat

a,file

oper

atio

ns.

–P

rote

ctio

n:

auth

entica

tion

,ac

cess

contr

ol

•Par

tII:C

ase

Stu

die

s

–U

nix

:file

abst

ract

ion,co

mm

and

‘ext

ensibili

ty’

–W

indow

sN

T:la

yering,

obje

cts,

asyn

ch.I/

O.

NT

Cas

eStu

dy—

Sum

mar

y15

8

—em

pty

—

NT

Cas

eStu

dy—

Sum

mar

y15

9

—em

pty

—

NT

Cas

eStu

dy—

Sum

mar

y16

0

Glo

ssar

yand

Acr

onym

s:A

–H

AG

PA

dva

nce

dG

raphic

sPor

tA

LU

Arith

met

ic/L

ogic

Unit

API

Applic

atio

nP

rogr

amm

ing

Inte

rfac

eA

RM

a32

-bit

RIS

Cm

icro

proce

ssor

ASCII

Am

eric

anSta

ndar

dC

ode

for

Info

rmat

ion

Inte

rchan

geB

SD

Ber

kele

ySof

twar

eD

istr

ibution

(Unix

varian

t)B

UB

ranch

Unit

CA

MC

onte

nt

Addre

ssab

leM

emor

yCO

WC

opy-

on-W

rite

CPU

Cen

tral

Pro

cess

ing

Unit

DAG

Direc

ted

Acy

clic

Gra

ph

DM

AD

irec

tM

emor

yA

cces

sD

OS

1.

apr

imitiv

eO

S(M

icro

soft

);2.

Den

ialof

Ser

vice

DRA

MD

ynam

icR

AM

FCFS

First

-Com

e-First

-Ser

ved

(see

also

FIF

O)

FIF

OFirst

-In-F

irst

-Out

(see

also

FC

FS)

Fork

crea

tea

new

copy

ofa

proce

ssFra

me

chunk

ofphys

ical

mem

ory

(alsopage

frame)

HA

LH

ardw

are

Abst

ract

ion

Lay

er

NT

Cas

eStu

dy—

Glo

ssar

y16

1

Glo

ssar

yand

Acr

onym

s:I–

N

I/O

Input/

Outp

ut

(alsoIO

)IA

32

Inte

l’s

32-b

itpr

oce

ssor

arch

itec

ture

IA64

Inte

l’s

64-b

itpr

oce

ssor

arch

itec

ture

IDE

Inte

grat

edD

rive

Ele

ctro

nic

s(d

isk

inte

rfac

e)IP

CIn

ter-

Pro

cess

Com

munic

atio

nIR

PI/

OR

eques

tPac

ket

IRQ

Inte

rrupt

ReQ

ues

tIS

A1.

Indust

rySta

ndar

dA

rchitec

ture

(bus)

;2.

Inst

ruct

ion

Set

Arc

hitec

ture

Inte

rrupt

asign

alfr

omhar

dw

are

toth

eC

PU

IOCT

La

syst

emca

llto

contr

olan

I/O

dev

ice

LPC

Loca

lP

roce

dure

Cal

lM

AU

Mem

ory

Acc

ess

Unit

MFT

Multip

leFix

edTas

ks(I

BM

OS)

MIP

S1.

Mill

ions

ofIn

stru

ctio

ns

per

Sec

ond;2.

a32

-bit

RIS

Cpr

oce

ssor

MM

UM

emor

yM

anag

emen

tU

nit

MFT

Multip

leFix

edTas

ks(I

BM

OS)

MV

TM

ultip

leV

aria

ble

Tas

ks(I

BM

OS)

NT

New

Tec

hnol

ogy

(Mic

roso

ftO

SFam

ily)

NT

FS

NT

File

Sys

tem

NT

Cas

eStu

dy—

Glo

ssar

y16

2

Glo

ssar

yand

Acr

onym

s:O

–SM

OS

Oper

atin

gSys

tem

OS/2

aP

Cop

erat

ing

syst

em(I

BM

&M

icro

soft

)PC

1.

Pro

gram

Cou

nte

r;2.

Per

sonal

Com

pute

rPCB

1.

Pro

cess

Con

trol

Blo

ck;2.

Printe

dC

ircu

itB

oard

PCI

Per

ipher

alC

om

pon

ent

Inte

rfac

ePIC

Pro

gram

mab

leIn

terr

upt

Con

trol

ler

PT

BR

Pag

eTab

leB

ase

Reg

iste

rPT

EPag

eTab

leEntr

yPage

fixe

dsize

chunk

ofvi

rtual

mem

ory

Poll

[rep

eate

dly

]det

erm

ine

the

stat

us

ofPosix

Por

table

OS

Inte

rfac

efo

rU

nix

RA

MR

andom

Acc

ess

Mem

ory

RO

MR

ead-O

nly

Mem

ory

SCSI

Sm

allC

ompute

rSys

tem

Inte

rfac

eSFID

Sys

tem

File

IDShell

prog

ram

allo

win

guse

r-co

mpute

rin

tera

ctio

nSig

nal

even

tdel

iver

edfr

omO

Sto

apr

oce

ssSJF

Shor

test

Job

First

SM

PSym

met

ric

Multi-P

roce

ssor

NT

Cas

eStu

dy—

Glo

ssar

y16

3

Glo

ssar

yand

Acr

onym

s:SR–X

SRA

MSta

tic

RA

MSRT

FShor

test

Rem

ainin

gT

ime

First

ST

BR

Seg

men

tTab

leB

ase

Reg

iste

rST

LR

Seg

men

tTab

leLen

gth

Reg

iste

rSys

tem

Va

varian

tof

Unix

TCB

1.

Thre

adC

ontr

olB

lock

;2.

Tru

sted

Com

puting

Bas

eT

LB

Tra

nslat

ion

Look

asid

eB

uffer

UCS

Univ

ersa

lC

har

acte

rSet

UFID

Use

rFile

IDU

TF-8

UC

STra

nsf

orm

atio

nFor

mat

8U

nix

the

firs

tke

rnel

-bas

edO

SVA

SV

irtu

alA

ddre

ssSpac

eV

LSI

Ver

yLar

geSca

leIn

tegr

atio

nV

M1.

Virtu

alM

emor

y;2.

Virtu

alM

achin

eV

MS

Virtu

alM

emor

ySys

tem

(Dig

ital

OS)

VX

DV

irtu

alD

evic

eD

rive

rW

in32

AP

Ipr

ovid

edby

moder

nW

indow

sO

Ses

XP

are

cent

OS

from

Mic

roso

ftx86

Inte

lfa

milt

yof

32-b

itC

ISC

proce

ssor

s

NT

Cas

eStu

dy—

Glo

ssar

y16

4