Post on 07-Apr-2018
transcript
8/3/2019 Evolution of the Windows Kernel Architecture
1/30
Dave Probert, Ph.D. - Windows Kernel ArchitectMicrosoft Windows Division
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
2/30
8/3/2019 Evolution of the Windows Kernel Architecture
3/30
UNIX vs NT Design
EnvironmentsEnvironment which influencedfundamental design decisions
UNIX [1969] Windows (NT) [1989]16-bit program address space
Kbytes of physical memory
Swapping system with memory mapping
Kbytes of disk, fixed disks
UniprocessorState-machine based I/O devices
Standalone interactive systems
Small number of friendly users
32-bit program address space
Mbytes of physical memory
Virtual memory
Mbytes of disk, removable disks
Multiprocessor (4-way)Micro-controller based I/O devices
Client/Server distributed computing
Large, diverse user populations
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
4/30
Effect on OS Design
NT vs UNIX
Although both Windows and Linux have adapted to changes in the environment,the original design environments (i.e. in 1989 and 1969) heavily influenced thedesign choices:
Unit of concurrency:
Process creation:
I/O:
Namespace root:
Security:
Threads vs processes
CreateProcess() vs fork()
Async vs sync
Virtual vs Filesystem
ACLs vs uid/gid
Addr space, uniproc
Addr space, swapping
Swapping, I/O devices
Removable storage
User populations
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
5/30
Todays Environment
[2009]64-bit addressesGBytes of physical memoryTBytes of rotational disk
New Storage hierarchies (SSDs)
Hypervisors, virtual processors
Multi-core/Many-coreHeterogeneous CPU architectures, Fixed function hardware
High-speed internet/intranet, Web Services
Media-rich applications
Single user, but vulnerable to hackers worldwide
Convergence: Smartphone / Netbook / Laptop / Desktop / TV / Web / Cloud
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
6/30
Windows Architecture
hardware interfaces (buses, I/O devices, interrupts,interval timers, DMA, memory cache control, etc., etc.)
System Service Dispatcher
Task Manager
Explorer
SvcHost.ExeWinMgt.Exe
SpoolSv.Exe
ServiceControl Mgr.
LSASS
Object
Mgr.
WindowsUSER,GDI
File
System
Cache
I/O Mgr
EnvironmentSubsystems
UserApplication
Subsystem DLLs
System Processes Services Applications
SystemThreads
UserMode
KernelMode
NTDLL.DLL
Device &File Sys.Drivers
WinLogon
SessionManager
Services.Exe POSIX
Windows DLLs
Plugand
PlayMgr.
Power
Mgr.
Security
Reference
Monitor
Virtual
Memory
Processes
&Threads
Local
Procedure
Call Graphics
Drivers
Kernel
Hardware Abstraction Layer (HAL)
(kernel mode callable interfaces)
C
onfigura-
tionMgr
(registry)
OS/2
Windows
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
7/30
Kernel-mode Architecture ofWindows
Copyright Microsoft Corporation
NT API stubs (wrap sysenter) -- system library (ntdll.dll)user
mode
kernel
mode
NTOS executive layer
Trap/Exception/Interrupt Dispatch
CPU mgmt: scheduling, synchr, ISRs/DPCs/APCs
DriversDevices, Filters,Volumes,Networking,
Graphics
Hardware Abstraction Layer (HAL): BIOS/chipset details
firmwar
e/hardwar
CPU, MMU, APIC, BIOS/ACPI, memory, devices
NTOSkernellayer
Caching Mgr
Security
Procs/Threads
VirtualMemory
IPC
glue
I/O
ObjectMgr
Registry
Co ri ht Microsoft Cor oration
8/3/2019 Evolution of the Windows Kernel Architecture
8/30
Kernel/Executive layers
Kernel layer ntos/ke ~ 5% of NTOSsource)
Abstracts the CPU
Threads, Asynchronous Procedure Calls (APCs) Interrupt Service Routines (ISRs)
Deferred Procedure Calls (DPCs aka SoftwareInterrupts)
Providers low-level synchronization Executive layer
OS Services running in a multithreadedenvironment
Full virtual memory, heap, handlesCopyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
9/30
NT (Native) API examples
NtCreateProcess (&ProcHandle, Access,SectionHandle, DebugPort, ExceptionPort,)
NtCreateThread (&ThreadHandle,ProcHandle, Access, ThreadContext,bCreateSuspended, )
NtAllocateVirtualMemory (ProcHandle,Addr, Size, Type, Protection, )
NtMapViewOfSection (SectionHandle,ProcHandle, Addr, Size, Protection, )
NtReadVirtualMemory (ProcHandle, Addr,Size, )
NtDuplicateObject (srcProcHandle,srcObjHandle, dstProcHandle, dstHandle,Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
10/30
Windows Vista Kernel
Changes Kernel changes mostly minorimprovements Algorithms, scalability, code maintainability CPU timing: Uses Time Stamp Counter (TSC) Interrupts not charged to threads
Timing and quanta are more accurate Communication ALPC: Advanced Lightweight Procedure Calls Kernel-mode RPC New TCP/IP stack (integrated IPv4 and IPv6)
I/O Remove a context switch from I/O CompletionPorts
I/O cancellation improvements Memory management Address space randomization (DLLs, stacks)
Kernel address space dynamically configured
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
11/30
Windows 7 Kernel Changes Miscellaneous kernel changes
MinWin
Change how Windows is built Lots of DLL refactoring API Sets (virtual DLLs)
Working-set management Runaway processes quickly start reusing own pages Break up kernel working-set into multiple working-setsSystem cache, paged pool, pageable system code
Security Better UAC, new account types, less BitLocker
blockers Energy efficiency Trigger-started background services
Core Parking Timer-coalescing, tick skipping
Major scalability improvements for largeserver apps Broke apart last two major kernel locks, >64p
Kernel support for ConcRT User-Mode Scheduling (UMS) Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
12/30
MinWin
MinWin is first step at creatingarchitectural partitions Can be built, booted and tested separately from the
rest of the system Higher layers can evolve independently
An engineering process improvement, not amicrokernel NT!
MinWin was defined as set ofcomponents required to boot and access
network Kernel, file system driver, TCP/IP stack, device
drivers, services No servicing, WMI, graphics, audio or shell, etc, etc,
etc
MinWin footprint:
8/3/2019 Evolution of the Windows Kernel Architecture
13/30
MinWin Layering
Shell,Graphics,Multimedia,Layered Services,
Applets,Etc.
Kernel,HAL,TCP/IP,File Systems,Drivers,Core System Services
MinWin
8/3/2019 Evolution of the Windows Kernel Architecture
14/30
Timer Coalescing
Secret of energy efficiency: Go idle and Stay idle Staying idle requires minimizing timer interrupts
Before, periodic timers had independent cycles evenwhen period was the same
New timer APIs permit timer coalescing
Application or driver specifies tolerable delay Timer system shifts timer firing
MarkRuss
8/3/2019 Evolution of the Windows Kernel Architecture
15/30
Broke apart the DispatcherLock
Scheduler Dispatcher lock hottest on server
workloads Lock protects all thread state changes (wait,
unwait) Very lock at >64x
Dispatcher lock broken up in Windows 7 / Server2008 R2 Each object protected by its own lock Many operations are lock-free
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
16/30
Removed PFN Lock Windows tracks the state of pages in physical
memory In use: in working sets:
Not assigned: on paging lists: freemodified,standby,
Before, all page state changes protected by globalPFN (Physical Frame Number) lock
As of Windows 7 the PFN lock is gone
Pages are now locked individually
Improves scalability for large memoryapplications
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
17/30
The Silicon Power WallThe situation: Power2 Clock frequency
Voltage Power2
Clock frequency and Voltage offset each other
Clock frequency inversely proportional to logic path length
Bad News: Power is about as low as it can go
Logic paths between clocked elements are pretty short
Good News: Moores Law continues (# transistors doubles ~22 months)
All that parallel computational theory is going into practice
Transistors going into more cores, not faster cores!
Software subject to Amdahls Law, not Moores Law(or Gustafsons Law
if my wife can find large enough datasets she cares about) 17
8/3/2019 Evolution of the Windows Kernel Architecture
18/30
Approaches to HW
parallelismHomogeneousMore big superscalar cores Extend with private (or shared) SIMD engines (SSE on steroids) (Maybe) not very energy efficient
A few more big, cores and lots of smaller, slower, coolercores
Use SIMD for performance Shutoff idle small cores for energy efficiency (but leakage?)
Lots of little fully programmable cores, all the same Nobody has ever gotten this to work more on this later
HeterogeneousProgrammable Accelerators (e.g. GPUs) Attach loosely-coupled, specialized (non-x86), energy-efficient
cores
Fixed-function Accelerators
Very energy-efficient, device-like computational units for very-specifictasks 18
8/3/2019 Evolution of the Windows Kernel Architecture
19/30
User Mode Scheduling (UMS) Improve support for efficient cooperative
multithreaded scheduling of small tasks (over-decomposition)Want to schedule tasks in user-mode
Use NT threads to simulate CPUs, multiplex tasks ontothese threads
When a task calls into the kernel and blocks, theCPU may get scheduled to a different appIf a single NT thread per CPU, when it blocks it blocks.
Could have extra threads, but then kernel and user-
mode are competing to schedule the CPU Tasks run arbitrary Win32 code (but only x64/IA64)
Assumes running on an NT thread (TEB, kernel thread)
Used by ConcRT (Visual Studio 2010s ConcurrencyRun-Time)
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
20/30
Windows 7 User-ModeScheduling
UMS breaks NT thread into two parts: UT: user-mode portion (TEB, ustack, registers) KT: kernel-mode portion (ETHREAD, kstack,
registers)
Three key properties: User-mode scheduler switches UTs w/o ring
crossing KT switch is lazy: at kernel entry (e.g. syscall,
pagefault)
CPU returned to user-mode scheduler when KTblocks
KT returns to user-mode by queuingcompletion
User-mode scheduler schedules corresponding UTCopyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
21/30
Normal NT Threading
kerne
luser
KT0 KT1 KT2
UT2UT1UT0
Kernel-mode
Scheduler NTOS executive
trap code
NT Thread is Kernel Thread (KT) and User Thread (UT)
UT/KT form a single logical thread representing NTthread in user or kernel
KT: ETHREAD, KSTACK, link to EPROCESSUT: TEB, USTACK
x86 core
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
22/30
User-Mode Scheduling (UMS)
kerne
luser
Thread Parking
KT0 KT1 KT2
UT Completion list
PrimaryThread
UT0UT1
UT0User-modeScheduler
trap code
NTOS executive
KT0 blocks
Only primary thread runs in user-mode
Trap code switches to parked KT
KT blocks primary returns to user-mode
KT unblocks & parks queue UTcompletion
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
23/30
UMS Based on NT threads
Each NT thread has user & kernel parts (UT & KT)When a thread becomes UMS, KT never returns to UT(Well, sort of)
Instead, theprimarythread calls the USched
USched
Switches between UTs, all in user-modeWhen a UT enters kernel and blocks, the primary
thread will hand CPU back to the USched declaring UTblockedWhen UT unblocks, kernel queues notification USched consumes notifications, marks UT runnable
Primary ThreadSelf-identified by entering kernel with wrong TEB So UTs can migrate between threads Affinities of primaries and KTs are orthogonal issues
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
24/30
UMS Thread Roles
Primary threads: represent CPUs, normal app threadsenter the USched world and become primaries,primaries also can be created by UScheds to allowparallel execution
Primaries represent concurrent execution
UMS threads (UT/KTs): allow blocking in the kernelwithout losing the CPU
UMS thread represent concurrent blocking inkernel
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
25/30
Thread Scheduling vs UMS
Core 2
Thread3
Non-running threads
Core 1
Thread4
Thread5
Thread1
Thread2
Thread6
Core 2Core 1
UserThrea
d2
Kernel
Thread
2
UserThrea
d1
Kernel
Thread
1
UserThrea
d3
Kernel
Thread3
UserThrea
d4
Kernel
Thread4
UserThrea
d5
Kernel
Thread5
UserThrea
d6
Kernel
Thread6
MarkRuss
8/3/2019 Evolution of the Windows Kernel Architecture
26/30
Win32 compat considerations
Why not Win32 fibers? TEB issues
Contains TLS and Win32-specific fields (incl LastError)
Fibers run on multiple threads, so TEB state doesnt
track Kernel thread issues
Visibility to TEB
I/O is queued to thread
Mutexes record thread owner Impersonation
Cross-thread operations expect to find threads and IDs
Win32 code has thread and affinity awareness
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
27/30
Futures: Master/Slave UMS?
remotekernel
Remotex86
Thread Parking
KT0 KT1 KT2
UT2UT1
RemoteScheduler
trap code
NTOS executiveKernel-mode
Scheduler
Syscall Completion QueueSyscall Request Queue
UT0
x86 core
UTs (can) run on accelerators or x86sKTs run on x86s, syscalls remoted/batched
Pagefaults are just like syscalls
Accelerator never loses the CPU (implicit primary)
Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
28/30
Operating Systems Futures
Many-core challenge
New driving force in software innovation:
Amdahls Law overtakes Moores Law as high-order bit
Heterogeneous cores?
OS Scalability Loosely coupled OS: mem + cpu + services? Energy efficiency
Shrink-wrap and Freeze-dry applications?
Hypervisor/Kernel/Runtime relationships Move kernel scheduling (cpu/memory) into run-
times? Move kernel resource management into
Hypervisor? Copyright Microsoft Corporation
8/3/2019 Evolution of the Windows Kernel Architecture
29/30
Windows Academic Program Windows Kernel Internals
Windows kernel in source (Windows Research Kernel
WRK) Windows kernel in PowerPoint (Curriculum Resource Kit
CRK)
Based on Windows Server 2008 Service Pack 1 Latest kernel at time of release
First kernel release with AMD64 support
Joint program between Windows Product Group andMS Academic Groups Program directed by Arkady Retik (Need a DVD? Have
questions?)Information available at
http://microsoft.com/WindowsAcademic OR compsci@microsoft.com
Microsoft Academic Contacts in Buenos Aires
Miguel Saez (masaez@microsoft.com) or Copyright Microsoft Corporation
http://microsoft.com/WindowsAcademicmailto:compsci@microsoft.commailto:masaez@microsoft.commailto:eglinsky@microsoft.commailto:masaez@microsoft.commailto:compsci@microsoft.comhttp://microsoft.com/WindowsAcademic8/3/2019 Evolution of the Windows Kernel Architecture
30/30
muchas gracias