CPE 442 vm.2Introduction To Computer Architecture
Outline of Today’s Lecture
° Recap of Memory Hierarchy & Introduction to Cache (10 min)
° Virtual Memory (5 min)
° Page Tables and TLB (25 min)
° Protection (20 min)
° Impact of Memory Hierarchy (5 min)
CPE 442 vm.3Introduction To Computer Architecture
Review: The Principle of Locality
° The Principle of Locality:• Program access a relatively small portion of the address space at
any instant of time.• Example: 90% of time in 10% of the code
Address Space0 2
Probabilityof reference
CPE 442 vm.4Introduction To Computer Architecture
Review: The Need to Make a Decision for replacement!
° Direct Mapped Cache:• Each memory location can only mapped to 1 cache location• No need to make any decision :-)
- Current item replaced the previous item in that cache location
° N-way Set Associative Cache:• Each memory location have a choice of N cache locations
° Fully Associative Cache:• Each memory location can be placed in ANY cache location
° Cache miss in a N-way Set Associative or Fully Associative Cache:• Bring in new block from memory• Throw out a cache block to make room for the new block• Damn! We need to make a decision which block to throw out!
CPE 442 vm.5Introduction To Computer Architecture
Review: DECStation 3100 16K words cache with one word per block
Address (showing bit positions)
16 14 Byte�offset
Valid Tag Data
Hit Data
16 32
16K�entries
16 bits 32 bits
31 30 17 16 15 5 4 3 2 1 0
CPE 442 vm.6Introduction To Computer Architecture
Review: 64K Direct mapped cache with block size of 4-words
Address (showing bit positions)
16 12 Byte�offset
V Tag Data
Hit Data
16 32
4K�entries
16 bits 128 bits
Mux
32 32 32
2
32
Block offsetIndex
Tag
31 16 15 4 32 1 0
CPE 442 vm.7Introduction To Computer Architecture
Review: block 12 from main memory and its mapping toto a block frame in direct mapped, set assoc., and fullyassoc. cache designs
1�
2Tag
Data
Block # 0 1 2 3 4 5 6 7
Search
Direct mapped
1�
2Tag
Data
Set # 0 1 2 3
Search
Set associative
1�
2Tag
Data
Search
Fully associativeDirect Mapped Set Associative Fully Associative
CPE 442 vm.8Introduction To Computer Architecture
Review:direct-mapped, set assoc., fully assoc.
Tag Data Tag Data Tag Data Tag Data Tag Data Tag Data Tag Data Tag Data
Eight-way set associative (fully associative)
Tag Data Tag Data Tag Data Tag Data
Four-way set associative
Set
0
1
Tag Data
One-way set associative�(direct mapped)
Block
0
7
1
2
3
4
5
6
Tag Data
Two-way set associative
Set
0
1
2
3
Tag Data
CPE 442 vm.9Introduction To Computer Architecture
Review:4-way setassociative
Address
22 8
V TagIndex
012
253254255
Data V Tag Data V Tag Data V Tag Data
3222
4-to-1 multiplexor
Hit Data
123891011123031 0
CPE 442 vm.10Introduction To Computer Architecture
Review Summary:
° The Principle of Locality:• Program access a relatively small portion of the address space at
any instant of time.- Temporal Locality: Locality in Time- Spatial Locality: Locality in Space
° Three Major Categories of Cache Misses:• Compulsory Misses: sad facts of life. Example: cold start misses.• Capacity Misses: increase cache size• Conflict Misses: increase cache size and/or associativity.
Nightmare Scenario: ping pong effect!
° Write Policy:• Write Through: need a write buffer. Nightmare: WB saturation• Write Back: control can be complex
CPE 442 vm.11Introduction To Computer Architecture
Review: Levels of the Memory Hierarchy
CPU Registers100s Bytes<10s ns
CacheK Bytes10-100 ns$.01-.001/bit
Main MemoryM Bytes100ns-1us$.01-.001
DiskG Bytesms10 - 10 cents-3 -4
CapacityAccess TimeCost
Tapeinfinitesec-min10-6
Registers
Cache
Memory
Disk
Tape
Instr. Operands
Blocks
Pages
Files
StagingXfer Unit
prog./compiler1-8 bytes
cache cntl8-128 bytes
OS512-4K bytes
user/operatorMbytes
Upper Level
Lower Level
faster
Larger
CPE 442 vm.12Introduction To Computer Architecture
Outline of Today’s Lecture
° Recap of Memory Hierarchy & Introduction to Cache (10 min)
° Virtual Memory° Page Tables and TLB (25 min)
° Protection (20 min)
° Impact of Memory Hierarchy (5 min)
CPE 442 vm.13Introduction To Computer Architecture
Virtual Memory
Provides illusion of very large memory– sum of the memory of many jobs greater than physical memory– address space of each job larger than physical memory
Allows available (fast and expensive) physical memory to be very well utilized
Simplifies memory management (main reason today)
Exploits memory hierarchy to keep average access time low.
Involves at least two storage levels: main and secondary
Virtual Address -- address used by the programmer
Virtual Address Space -- collection of such addresses
Memory Address -- address of word in physical memoryalso known as “physical address” or “real address”
CPE 442 vm.14Introduction To Computer Architecture
Basic Issues in VM System Designsize of information blocks that are transferred from
secondary to main storage
block of information brought into M, and M is full, then some regionof M must be released to make room for the new block -->replacement policy
which region of M is to hold the new block --> placement policy
missing item fetched from secondary memory only on the occurrenceof a fault --> fetch/load policy
Paging Organization
virtual and physical address space partitioned into blocks of equal size
page frames
pages
pagesreg
cachemem disk
frame
CPE 442 vm.15Introduction To Computer Architecture
Address MapV = {0, 1, . . . , n - 1} virtual address spaceM = {0, 1, . . . , m - 1} physical address space
MAP: V --> M U {0} address mapping function
n >> m
MAP(a) = a' if data at virtual address a is present in physical address a' and a' in M
= 0 if data at virtual address a is not present in M
Processor
Name Space V
Addr TransMechanism
faulthandler
MainMemory
SecondaryMemory
a
aa'
0
missing item fault
physical address OS performsthis transfer
CPE 442 vm.16Introduction To Computer Architecture
Outline of Today’s Lecture
° Recap of Memory Hierarchy & Introduction to Cache (10 min)
° Virtual Memory (5 min)
° Page Tables and TLB (25 min)
° Protection (20 min)
° Impact of Memory Hierarchy (5 min)
CPE 442 vm.17Introduction To Computer Architecture
Paging Organization
frame 01
7
01024
7168
P.A.
PhysicalMemory
1K1K
1K
AddrTransMAP
page 01
31
1K1K
1K
01024
31744
unit of mapping
also unit oftransfer fromvirtual tophysical memory
Virtual MemoryAddress Mapping
VA page no. disp10
Page Table
indexintopagetable
Page TableBase Reg
V AccessRights PA +
table locatedin physicalmemory
physicalmemoryaddress
actually, concatenation is more likely
3 2 1 011 10 9 815 14 13 1231 30 29 28 27
Page offsetVirtual page number
Virtual address
3 2 1 011 10 9 815 14 13 1229 28 27
Page offsetPhysical page number
Physical address
Translation
CPE 442 vm.19Introduction To Computer Architecture
Address Mapping AlgorithmIf V = 1
then page is in main memory at frame address stored in tableelse address located page in secondary memory
Access RightsR = Read-only, R/W = read/write, X = execute only
If kind of access not compatible with specified access rights,then protection_violation_fault
If valid bit not set then page fault
Protection Fault: access rights violation; causes trap to hardware,microcode, or software fault handler
Page Fault: page not resident in physical memory, also causes a trap;usually accompanied by a context switch: current processsuspended while page is fetched from secondary storage
32
CPE 442 vm.20Introduction To Computer Architecture
Fragmentation & Relocation
Fragmentation is when areas of memory space become unavailable forsome reason
Relocation: move program or data to a new region of the addressspace (possibly fixing all the pointers)
Internal Fragmentation:program is not an integral # of pages, part of the last page frame is"wasted" (obviously less of an issue as physical memories get larger)
0 1 k-1. . .occupied
External Fragmentation: Space left between blocks.
CPE 442 vm.21Introduction To Computer Architecture
Optimal Page SizeChoose page that minimizes fragmentation
large page size => internal fragmentation more severeBUT lower page size increases the # of pages / name space
=> larger page tables
In general, the trend is towards larger page sizes because
Most machines at 4K byte pages today, with page sizes likely toincrease
-- memories get larger as the price of RAM drops
-- the gap between processor speed and disk speed grow wider
-- programmers desire larger virtual address spaces
CPE 442 vm.22Introduction To Computer Architecture
Fragmentation (cont.)Table Fragmentation occurs when page tables become very large
because of large virtual address spaces; direct mapped pagetables could take up sizable chunk of memory
XX Page Number Disp21 9
00 P0 region of user process01 P1 region of user process10 system name space
EX: VAX Architecture
NOTE: this implies that page table could require up to 2 ^21 entries, each on the order of 4 bytes long (8 M Bytes)
Alternatives:(1) Hardware associative mapping: Only keep in the page table entries for pages in Main memory. Use Associative search
requires one entry per page frame (O(|M|)) rather than per page (O(|N|))
Present Access Page# Phy Addr
associative lookuppn the page number field
page# dispPage Table
CPE 442 vm.23Introduction To Computer Architecture
(2) 2-level page table
.
.
.
Seg 0
Seg 1
Seg255
4 bytes
256 P0
P255
4 bytes
1 K
.
.
.
PA
PA
D0
D1023
PA
PA ...
Root Page TablesData Pages
4 K
Second Level Page Table
2 2 2 28 8 10 12 238x x x =
Allocated inUser Virtual
Space
1 Mbyte, but allocatedin system virtual addr
space256K bytes inphysical memory
CPE 442 vm.24Introduction To Computer Architecture
Page Replacement AlgorithmsJust like cache block replacement!
Least Recently Used:-- selects the least recently used page for replacement
-- requires knowledge about past references, more difficult to implement(thread thru page table entries from most recently referenced to leastrecently referenced; when a page is referenced it is placed at the headof the list; the end of the list is the page to replace)
-- good performance, recognizes principle of locality
CPE 442 vm.25Introduction To Computer Architecture
Page Replacement (Continued)Not Recently Used:Associated with each page is a reference flag such that
ref flag = 1 if the page has been referenced in recent past= 0 otherwise
-- if replacement is necessary, choose any page frame such that itsreference bit is 0. This is a page that has not been referenced in therecent past
-- clock implementation of NRU:
1 01 000
page table entrypagetableentry
refbit
last replaced pointer (lrp)if replacement is to take place,advance lrp to next entry (modtable size) until one with a 0 bitis found; this is the target forreplacement; As a side effect,all examined PTE's have theirreference bits set to zero.
1 0
An optimization is to search for the a page that is both not recently referenced AND not dirty.
CPE 442 vm.26Introduction To Computer Architecture
Demand Paging and Prefetching PagesFetch Policy
when is the page brought into memory?if pages are loaded solely in response to page faults, then the
policy is demand paging
An alternative is prefetching:anticipate future references and load such pages before their
actual use
+ reduces page transfer overhead
- removes pages already in page frames, which could adverselyaffect the page fault rate
- predicting future references usually difficult
Most systems implement demand paging without prepaging
CPE 442 vm.27Introduction To Computer Architecture
Virtual Address and a Cache
CPU Trans-lation Cache Main
Memory
VA PA miss
hitdata
It takes an extra memory access to translate VA to PA
This makes cache access very expensive, and this is the "innermostloop" that you want to go as fast as possible
ASIDE: Why access cache with PA at all? VA caches have a problem!synonym problem: two different virtual addresses map to same physical address => two different cache entries holding data forthe same physical address!
for update: must update all cache entries with samephysical address or memory becomes inconsistent
determining this requires significant hardware, essentially anassociative lookup on the physical address tags to see if you have multiple hits
CPE 442 vm.28Introduction To Computer Architecture
TLBs (Translation Look aside Buffer) A way to speed up translation is to use a special cache of recently
used page table entries -- this has many names, but the mostfrequently used is Translation Lookaside Buffer or TLB
Virtual Address Physical Address Dirty Ref Valid Access
TLB access time comparable to, though shorter than, cache access time(still much less than main memory access time)
CPE 442 vm.29Introduction To Computer Architecture
TLB: acts as a specialized cache for address translation
Valid
1
1
1
1
0
1
1
0
1
1
0
1
Page table
Physical page�addressValid
TLB
1
1
1
1
0
1
TagVirtual page�
number
Physical page�or disk address
Physical memory
Disk storage
CPE 442 vm.30Introduction To Computer Architecture
Translation Look-Aside BuffersJust like any other cache, the TLB can be organized as fully associative,
set associative, or direct mapped
TLBs are usually small, typically not more than 128 - 256 entries even onhigh end machines. This permits fully associativelookup on these machines. Most mid-range machines use smalln-way set associative organizations.
CPU TLBLookup Cache Main
Memory
VA PA miss
hit
data
Trans-lation
hit
miss
20 tt1/2 t
Translationwith a TLB
CPE 442 vm.31Introduction To Computer Architecture
Reducing Translation Time
Machines with TLBs go one step further to reduce # cycles/cache access
They overlap the cache access with the TLB access
Works because high order bits of the VA are used to look in the TLBwhile low order bits are used as index into cache
CPE 442 vm.32Introduction To Computer Architecture
TLB look up and cache access
Valid Tag Data
Page offset
Page offset
Virtual page number
Virtual address
Physical page numberValid
1220
20
16 14
Cache index
32
Cache
DataCache hit
2
Byte�offset
Dirty Tag
TLB hit
Physical page number
Physical address tag
TLB
�
Physical address
31 30 29 15 14 13 12 11 10 9 8 3 2 1 0
CPE 442 vm.33Introduction To Computer Architecture
DCEStation 3100 Write Through Write Not-Allocate Cache, sequence of events for read and write access
Yes
Deliver data�to the CPU
Write?
Try to read data�from cache
Write data into cache,�update the tag, and put�
the data and the address�into the write buffer
Cache hit?Cache miss stall
TLB hit?
TLB access
Virtual address
TLB miss�exception
No
YesNo
YesNo
Write access�bit on?
�
YesNo
Write protection�exception
Physical address
CPE 442 vm.34Introduction To Computer Architecture
Overlapped Cache & TLB Access
TLB Cache
10 200
4 bytes
index 1 K
page # disp20 12
assoclookup32
PA Hit/Miss PA Data Hit/
Miss
=
IF cache hit AND (cache tag = PA) then deliver data to CPUELSE IF [cache miss OR (cache tag = PA)] and TLB hit THEN
access memory with the PA from the TLBELSE do standard VA translation
CPE 442 vm.35Introduction To Computer Architecture
Problems With Overlapped TLB AccessOverlapped access only works as long as the address bits used to
index into the cache do not change as the result of VA translation
This usually limits things to small caches, large page sizes, or highn-way set associative caches if you want a large cache
Example: suppose everything the same except that the cache isincreased to 8 K bytes instead of 4 K:
11 200
virt page # disp20 12
cache index
This bit is changedby VA translation, butis needed for cachelookup
Solutions:go to 8K byte page sizesgo to 2 way set associative cache (would allow you to continue to
use a 10 bit index)
1K4 4
102 way set assoc cache
CPE 442 vm.36Introduction To Computer Architecture
More on Selecting a Page Size
° Reasons for larger page size• Page table size is inversely proportional to the page size; therefore
memory saved .• Transferring larger pages to or from secondary storage, possibly
over a network, is more efficient• Number of TLB entries are restricted by clock cycle time, so a
larger page size maps more memory thereby reducing TLB misses.
° Reasons for a smaller page size• don’t waste storage; data must be contiguous within page• quicker process start for small processes?
° Hybrid solution: multiple page sizesAlpha: 8KB, 64KB, 512 KB, 4 MB pages
CPE 442 vm.37Introduction To Computer Architecture
Outline of Today’s Lecture
° Recap of Memory Hierarchy & Introduction to Cache (10 min)
° Virtual Memory (5 min)
° Page Tables and TLB (25 min)
° Segmentation° Impact of Memory Hierarchy (5 min)
CPE 442 vm.38Introduction To Computer Architecture
Segmentation (see x86)Alternative to paging (often combined with paging)
Segments allocated for each program module; may be different sizessegment is unit of transfer between physical memory and disk
Present Access Length Phy Addrseg # disp
SegmentTable
segment lengthaccess rightsAddr=start addr of
segment
+physical addr
Presence Bit
BR
Faults:missing segment (Present = 0)overflow (Displacement exceeds segment length)protection violation (access incompatible with segment protection)
Segment-based addressing sometimes used to implement capabilities, i.e., hardware support for sophisticated protection mechanisms
CPE 442 vm.39Introduction To Computer Architecture
Segment Based AddressingThree Serious Drawbacks:
(1) storage allocation with variable sized blocks (best fit vs. first fit vs. buddy system)
(2) external fragmentation: physical memory allocated in such a fashion that all remaining pieces are too small to be allocated to any segment. Solved be expensive run-time memory compaction.
(3) Non-linear address matching pointer arithmetic in C?
The best of both worlds: paged segmentation schemesseg # page # displacementvirtual address
CPE 442 vm.40Introduction To Computer Architecture
Outline of Today’s Lecture
° Recap of Memory Hierarchy & Introduction to Cache (10 min)
° Virtual Memory (5 min)
° Page Tables and TLB (25 min)
° Protection (20 min)
° Examples° Impact of Memory Hierarchy (5 min)
CPE 442 vm.41Introduction To Computer Architecture
Alpha VM Mapping
° Alpha 21064 TLB: 32 entry fully associative
° “64-bit” address divided into 3 segments• seg0 (bit 63=0) user code/heap• seg1 (bit 63 = 1, 62 = 1) user stack• kseg (bit 63 = 1, 62 = 0)
kernel segment for OS
° 3 level page table
CPE 442 vm.42Introduction To Computer Architecture
Alpha 21064
° Separate Instr & Data TLB & Caches
° TLBs fully associative
° Caches 8KB direct mapped
° Pre fetch Buffer for Is
° Write Buffer for data
° 2 MB L2 cache, direct mapped
° 256 bit path to main memory, 4 64-bit modules
V Data<1>
D<1> <10> <256>
=?
(65,536blocks)
<10>
Tag Index
<19>
Main Memory
Tag
Victim Buffer
Write Buffer
Blockoffset
Index
<8> <5>
1
1
2
2
3
5
5
6
7
89
10
11 12
12
12
13
14
15
16
17
17
18
18
19
19
19
20
21
22
23
23
23
24
25
26
27
28
28
4:1 MUX
Page-frame address <30>
Instruction <64> Data In <64>Data Out <64>
V Physical address<1> <21>
R<2>
W<2>
Tag<30>
<21>
<64>
<64>
<29><29>
<64>
……
…
(high-order 21 bits of physical address)
12:1 MUX
Pageoffset<13>
Blockoffset
Index
<8> <5>
Data Page-frame address <30>
V Physical address<1> <21>
R<2>
W<2>
Tag<30>
<21>
……
…
(high-order 21 bits of physical address)
32:1 MUX
Pageoffset<13>
ITLB
ICACHE
L2CACHE
DCACHE
DTLB
PC
CPU
Alpha AXP 21064
=?
Instruction Prefetch Stream Buffer
Tag <29> Data <256>=?
Tag <29> Data <256>
MagneticDisk
Valid Data<1> <21> <64>
=?
2
4
5
912
(256blocks)
Tag Valid Data<1> <21> <64>
=?
(256blocks)
Tag
Delayed Write Buffer
CPE 442 vm.47Introduction To Computer Architecture
Outline of Today’s Lecture
° Recap of Memory Hierarchy & Introduction to Cache (10 min)
° Virtual Memory (5 min)
° Page Tables and TLB (25 min)
° Protection (20 min)
° Summary
CPE 442 vm.48Introduction To Computer Architecture
Conclusion
° Virtual Memory invented as another level of the hierarchy
° Controversial at the time: can SW automatically manage 64KB across many programs?
° DRAM growth removed the controversy
° Today VM allows many processes to share single memory without having to swap all processes to disk, protection more important
° Address translation using page tables
° (Multi-level) page tables to map virtual address to physical address
° TLBs are important for fast translation
° TLB misses are significant in performance