48
CS 162: Operating Systems and Systems Programming Lecture 19: Address Translation, Paging, Protection July 29, 2019 Instructor: Jack Kolb https://cs162.eecs.berkeley.edu
CS 162: Operating Systems and Systems Programming
Lecture 19: Address Translation, Paging, ProtectionJuly 29, 2019Instructor: Jack Kolbhttps://cs162.eecs.berkeley.edu
Logistics
• Project 2 Due Tonight 11:59 PM
•HW3 Released, Due on August 9
•Mid-Summer Course Survey:https://tinyurl.com/cs162midsummer• Still time to fill it out for participation credit
• Thanks for the feedback!
Recall: Address Translation
• Program sees and uses virtual addresses• Kernel-controlled mapping to convert from a
virtual address to physical address• Implemented by hardware
• The preferred mechanism for memory multiplexing
Recall: Segmentation
• Manage each region of address space as an independent unit• One process uses multiple contiguous allocations of
memory, called segments
1
3
2
4
user view ofmemory space
14
2
3
physical memory space
1
2
Segmentation Implementation
• Segment map stored in processor• Segment # extracted from virtual address
• Base added to generate physical address• Bound checked, error if out of range
Base0 Limit0 VBase1 Limit1 VBase2 Limit2 VBase3 Limit3 NBase4 Limit4 VBase5 Limit5 NBase6 Limit6 NBase7 Limit7 V
OffsetSeg #VirtualAddress
Base2 Limit2 V
+ PhysicalAddress
> Erroroffset
Segmentation Implementation
• Valid Bit: Did we actually allocate this region of mem?• Access error if No• Can’t have a negative bound
Base0 Limit0 VBase1 Limit1 VBase2 Limit2 VBase3 Limit3 NBase4 Limit4 VBase5 Limit5 NBase6 Limit6 NBase7 Limit7 V
OffsetSeg #VirtualAddress
Base2 Limit2 V
+ PhysicalAddress
> Erroroffset
Check Valid
AccessError
Segmentation Steps
1. CPU references a virtual address in an instruction2. Split the address into two parts:• Segment Number (top bits): Which segment?• Offset (bottom bits): How far into the segment?
3. Look up entry in segment table matching seg. #• If invalid, report an error• Otherwise, we get Base and Bound for the segment• If Offset > Bound, report an error
4. Physical Address = Base + Offset
Segmentation vs Base & Bound
• Internal Fragmentation: Much Better
• External Fragmentation: Still a Problem• But segments are typically smaller
• Enables Sharing: Entries in two different segment tables refer to same physical address range• Two different processes can use this range
How can we do better?
• Allocate memory in fixed-size chunks: pages• All allocations are full pages• All holes are multiples of a page size
• Remember blocks in file systems? Same concept• Expanding memory footprint: allocate new page• No need to relocate existing allocations• No struggle to find free region of proper size
PagingProcess sees contiguous virtual address space• Transparently divided into
virtual pages
Physical Memory is Divided Into Chunks Called Frames
What’s the catch?
• More bookkeeping: Map between virtual pages and physical page frames
• Page size represents a tradeoff• Number of pages vs. Internal Fragmentation
• Less locality: Sequential access in virtual space is not necessarily sequential in physical space• Jumping across page boundaries
Physical Address
Offset
Implementing Paging
Page Table: Process-Specific map for bookkeeping• Tells us in which physical page frame our virtual page is stored• Virtual page number is index into table• Table itself resides in physical memory• Trigger fault if page access is not permitted
OffsetVirtualPage #Virtual Address:
Access Error
>PageTableSize
PageTablePtr page #0
page #2page #3page #4page #5
V,R
page #1 V,R
V,R,W
V,R,W
N
V,R,W
page #1 V,R
Check Perm
AccessError
PhysicalFrame #
Physical Address
Offset
Paging: Address Mapping
• Offset: copied from virtual addr to physical addr• Example: 4K pages require a 12-bit offset
• Virtual page # is all remaining bits• 32-bit address space with 4K Pages: 220 = 1 Million pages
• Physical page # replaces virtual page # to form physical addr• May be different length (fewer physical frames than virtual pages)
OffsetVirtualPage #Virtual Address:
Access Error
>PageTableSize
PageTablePtr page #0
page #2page #3page #4page #5
V,R
page #1 V,R
V,R,W
V,R,W
N
V,R,W
page #1 V,R
Check Perm
AccessError
PhysicalFrame #
Address Translation: Overhead
• Base and Bound: Two Registers• Segmentation: Segment Table• One table entry per segment• Few segments, so typically small• Potentially store in registers
• Paging: Page Table• Allocate physical memory to store table• Extra memory access per operation
• Once to translate address, second to do actual work• Tables can get large (1 million entries on 32-bit machine)
PageTablePtrB page #0page #1page #2page #3
page #5
V,RNV,R,WN
page #4 V,RV,R,W
page #4 V,R
What about Sharing?OffsetVirtual
Page #Virtual Address
(Process A):
PageTablePtrA page #0page #1
page #3page #4page #5
V,RV,R
page #2 V,R,WV,R,WNV,R,W
OffsetVirtualPage #
Virtual Address(Process B):
SharedPage
This physical pageappears in address
space of both processes
page #2 V,R,W
1111 1111stack
heap
code
data
Virtual memory view
0000 0000
0100 0000
1000 0000
1100 0000
page # offset
Physical memory view
data
code
heap
stack
0000 00000001 0000
0101 000
0111 000
1110 0000
11111 1110111110 1110011101 null 11100 null 11011 null11010 null11001 null11000 null10111 null10110 null10101 null10100 null10011 null10010 1000010001 0111110000 0111001111 null01110 null 01101 null01100 null01011 01101 01010 01100 01001 0101101000 0101000111 null00110 null00101 null 00100 null 00011 0010100010 0010000001 0001100000 00010
Page Table
1110 0000
What happens if stack grows to 1110 0000?
Paging: Allocation
Summary: Paging1111 1111
stack
heap
code
data
Virtual memory view
0000 0000
0100 0000
1000 0000
1100 0000
page # offset
11111 1110111110 1110011101 1011111100 1011011011 null11010 null11001 null11000 null10111 null10110 null10101 null10100 null10011 null10010 1000010001 0111110000 0111001111 null01110 null01101 null01100 null01011 01101 01010 01100 01001 0101101000 0101000111 null00110 null00101 null 00100 null 00011 0010100010 0010000001 0001100000 00010
Page Table
0000 00000001 0000
0101 000
0111 000
1110 00001110 0000
Physical memory view
data
code
heap
stack
stack
Allocate new pages where room!
Paging: Hardware Support
• Special Register to store page table pointer• Like base and bound, modified in kernel mode only• Change register value on context switch
• Page Table Lookup• Two physical memory accesses per virtual access• Later: making this faster• Modern systems do a lot of this in hardware
Thinking Back to File Systems
• Page table similar to inode array/MFT• Page number, like file number, gives us a table index with
the relevant information
• Both have to reserve space for bookkeeping• Chunk of physical memory for page tables• Set of disk blocks for inode array and indirect blocks
• Also some significant differences• 1 Page Table per process• 1 inode array per filesystem
Space Consumption
• Say a process has allocated 32 MB of space• How much physical memory does it require?• Assume each page table entry is 4 bytes in size
• Scenario 1: Pages are 4KB, machine offers 32-bit virtual addresses• Scenario 2: Pages are 4KB, machine offers 64-bit
virtual addresses
Space Consumption
• OS is paying bookkeeping cost for the entire virtual address range• But most processes only allocate and use a small
fraction of their address space!
• Thinking back to file systems• inodes add layers of indirection as needed for larger files
• Indirect pointer, doubly indirect pointer, …
• Apply tree structure to virtual memory:Multilevel Page Tables
PhysicalAddress:
OffsetPhysicalPage #
4KB
Two-Level Page Tables10 bits 10 bits 12 bits
Virtual Address:
OffsetVirtualP2 index
VirtualP1 index
4 bytes
PageTablePtr
4 bytes
• A tree of page tables• Each "node" has a fixed size• x86: 1024 4-byte entries (why?)
• Still just one register tochange on context switch
PhysicalAddress:
OffsetPhysicalPage #
4KB
Two-Level Page Tables10 bits 10 bits 12 bits
Virtual Address:
OffsetVirtualP2 index
VirtualP1 index
4 bytes
PageTablePtr
4 bytes
Big gaps in virtual address space don't need second level page tables
PhysicalAddress:
OffsetPhysicalPage #
4KB
Two-Level Page Tables10 bits 10 bits 12 bits
Virtual Address:
OffsetVirtualP2 index
VirtualP1 index
4 bytes
PageTablePtr
4 bytes
But we're also making two memory accesses just for translation
stack
Two-Level Paging Example1111 1111
stack
heap
code
data
Virtual memory view
0000 0000
0100 0000
1000 0000
1100 0000
page1 # offset
Physical memory view
data
code
heap
stack
0000 00000001 0000
0101 000
0111 000
1110 0000
page2 #
111110 null101 null100011 null010001 null000
11 11101 10 1110001 1011100 10110
11 01101 10 0110001 0101100 01010
11 00101 10 0010001 0001100 00010
11 null 10 1000001 0111100 01110
Page Tables(level 2)
Page Table(level 1)
1111 0000
stack
Summary: Two-Level Paging
stack
heap
code
data
Virtual memory view
1001 0000(0x90)
Physical memory view
data
code
heap
stack
0000 00000001 0000
1000 0000(0x80)
1110 0000
111110 null101 null100011 null010001 null000
11 11101 10 1110001 1011100 10110
11 01101 10 0110001 0101100 01010
11 00101 10 0010001 0001100 00010
11 null 10 1000001 0111100 01110
Page Tables(level 2)
Page Table(level 1)
Page Table Discussion
• Good Points:• Easy memory allocation (fixed size chunks)• Easy sharing among processes• No physical memory used for holes in virtual address sp.
• Bad Points:• Overhead:At least 1 pointer per 4K page• Page tables need to be contiguous (tricky allocation)
• Clever design (x86) can limit them to one page• Several memory accesses per translation
• Later: making this faster
Page Tables for Huge Address Spaces• Want full 64 bits of address space?• How many levels of indirection needed?
• Overhead: Extra space for tables in phys. mem.• And additional memory accesses per translation• 64-bit x86 does not do this• Virtual Addresses are 48 bits long• 4-level page table
OffsetVirtual P6 Index
Virtual P5 Index
Virtual P4 Index
Virtual P3 Index
Virtual P2 Index
12 bits10 bits10 bits10 bits10 bits10 bits2 bits
Alternative: Inverted Page Table
• Until Now: Size of page table proportional to size of virtual address space• New Structure: Size of page table proportional to
size of physical address space• Good if most of address space not in use• Particularly 64-bit address space
OffsetVirtualPage #
HashTable
OffsetPhysicalPage #
Alternative: Inverted Page Table
• Good• Overhead proportional to virtual addresses in use• Just one level of indirection
• Bad: Complexity• How do we deal with hash collisions?
OffsetVirtualPage #
HashTable
OffsetPhysicalPage #
Break
Combining Segments & Pages• One approach: Each segment has its own page table• Fragmentation/allocation advantages of pages
page #0page #1
page #3page #4page #5
V,R
V,R
page #2 V,R,W
V,R,W
N
V,R,W
Offset
Physical Address
Virtual Address:
OffsetVirtualPage #
VirtualSeg #
Base0 Limit0 VBase1 Limit1 VBase2 Limit2 VBase3 Limit3 NBase4 Limit4 VBase5 Limit5 NBase6 Limit6 NBase7 Limit7 V
Base2 Limit2 V
AccessError>
page #2 V,R,W
PhysicalPage #
Check Permissions
AccessError
Sharing a SegmentProcess A: OffsetVirtual
Page #VirtualSeg #
Base0 Limit0 VBase1 Limit1 VBase2 Limit2 VBase3 Limit3 NBase4 Limit4 VBase5 Limit5 NBase6 Limit6 NBase7 Limit7 V
Base2 Limit2 V
page #0page #1page #2page #3page #4page #5
V,R
V,R
V,R,W
V,R,W
N
V,R,W
Shared Segment
Base0 Limit0 VBase1 Limit1 VBase2 Limit2 VBase3 Limit3 NBase4 Limit4 VBase5 Limit5 NBase6 Limit6 NBase7 Limit7 V
Base2 Limit2 V
Process B: OffsetVirtualPage #
VirtualSeg #
Combining Segments & Pages
• Good:• Just need one page table per segment, even if shared• Easy memory allocation (no external fragmentation)• Easy sharing of segments, just change segment tbl. entry• Can still share at page granularity if necessary
• Bad• Overhead: 1 pointer per page• At least two extra lookups per memory access
Address Translation ComparisonAdvantages Disadvantages
SimpleSegmentation
Fast context switching: Segment mapping maintained by CPU
External fragmentation
Paging (single-level page)
No external fragmentation, fast easy allocation
Large table size ~ virtual memoryInternal fragmentation
Paged segmentation
Table size ~ # of pages in virtual memory, fast easy allocation
Multiple memory references per page access
Two-level pages
Inverted Table Table size ~ # of pages in physical memory
Hash function more complexNo cache locality of page table
Real World: 32-bit x86
• Has both segmentation and paging• Segmentation different from what we've described• Segment identified by instruction, not address
• Note: x86 actually offers multiple modes of memory operation, we'll talk about common one
Real Example: Intel x86 (Old Days)
Today: Register points to segment table in physical memory
80386 Special Registers
Intel x86
• Six segments: cs (code), ds (data), ss (stack),es, fs, gs (extras)• Instructions identify segment to use• mov [es:bx], ax• Some instructions have default segments, e.g. push andpop always refer to ss (stack)
• Underused in modern operating systems• In 64-bit x86, only fs and gs support enforcement
of base and bound
x86 Memory Model With Segmentation (32-bit)
Real Page Table Entries
• So far, we've said they contain a physical frame #• Or the frame # of a page table lower on the "tree"
• But we want some extra information• Present/valid bit: indicate unallocated regions of
physical address space• Protection bits, e.g. to set certain regions of
memory to read-only for sharing
32-bit x86 Page Table Entry
• 10/10/12 Split of Virtual Address• Top-level page tables called directories
• PFN: Physical Page number of page or next page table• P: Present Bit (Is page mapping valid?)
Page Table entry acts as a pointer
Page Frame Number(Physical Page Number)
Free(OS) 0 L D A
PC
DP
WT U W P
01234567811-931-12
32-bit x86 Page Table Entry
• 10/10/12 Split of Virtual Address• Top-level page tables called directories
• W: Is this page writable?• U: Can we access this page while in user mode?
Page Table entry acts as authority for permissions
Page Frame Number(Physical Page Number)
Free(OS) 0 L D A
PC
DP
WT U W P
01234567811-931-12
32-bit x86 Page Table Entry
• 10/10/12 Split of Virtual Address• Top-level page tables called directories
• A: Set by hardware when page is first accessed• D: Page marked dirty if it is ever modified
Page Table entry tells OS about usage patterns
Page Frame Number(Physical Page Number)
Free(OS) 0 L D A
PC
DP
WT U W P
01234567811-931-12
32-bit x86 Page Table Entry
• 10/10/12 Split of Virtual Address• Top-level page tables called directories
• PWT:Write-through cache behavior• PCD: Disable caching for this page (more later)
Pages for memory-mapped IO treated differently
Page Frame Number(Physical Page Number)
Free(OS) 0 L D A
PC
DP
WT U W P
01234567811-931-12
Paging Tricks
• What does it mean if a page table entry doesn't have the valid (present) bit set?• Region of address space is invalid or• Page is not loaded and ready yet
• When program accesses an invalid PTE, OS gets an exception (a page fault or protection fault)• Options• Terminate program (access was actually invalid)• Get page ready and restart instruction
Example Paging Tricks
1. Demand Paging: Swapping for pages• Keep only active pages in memory• Remember: not common on modern systems, except
perhaps when first loading program• Response: Load in page from disk, retry operation
2. Copy on Write (remember fork?)• Temporarily mark pages as read-only• Allocate new pages when OS receives protection fault
3. Zero-Fill on Demand• Slow to overwrite new pages with all zeros• Page starts as invalid, zero it out when it's actually used
Summary
• Segmentation: Variable-sized, contiguous regions• Segment Table small enough to live in registers• Table entry contains physical base address and bound
• Paging: Fixed size chunks• No external fragmentation, easy sharing, tricks• Bookkeeping Overhead (page tables)• Extra Memory Accesses (slows down translation)
• Instead of a flat array, might use:• Multi-Level Page Tables (tree structure)• Inverted Page Table (hash table stucture)
Summary
In 32-bit x86• Mix segmentation and paging• Two-level page tables (10/10/12 split)
In 64-bit x86• Segmentation more limited• Only 48-bit virtual addresses• 4-level page tables
