Best Practices in Architecting and Implementing Windows Server Update Services (WSUS)

Greg ShieldsPartnerConcentrated TechnologyWSV302

Agenda

TopicsPart I: Architecting and Implementing WSUSPart II: Troubleshooting WSUSPart III: Tips and Tricks for Using WSUS

Architecting and Implementing WSUSpart 1

WSUS Product Vision

Simple, zero-cost solution for distributing Microsoft Updates content in a corporation

A “free” RTW add-on for Windows ServerSolution only distributes Microsoft Updates

Distributing 3rd party patches require purchasing advanced management tools such as SCE or Configuration Manager 2007

Provides a foundation for Update Management across Microsoft products: SCE, Configuration Manager 2007, MBSA, WU, SBS, Forefront …

Consistent scan resultsUnified client scan mechanism (WUA) irrespective of which server actually manages the updates

WSUS Momentum

Over 500,000 distinct WSUS servers synched with Microsoft Update last monthUsed by over 60% medium/large orgs and built into SBSWSUS 3 released April 30 2007

Huge improvements in performance, deployment options, reporting and UIEasy in-place upgrade from WSUS2

WSUS 3.0 SP1 released Feb 7, 2008WSUS 3.0 SP2 released Jan 26, 2009

WSUS Lifecycle/Roadmap

Support lifecycle

Next up: release WSUS3 SP2 RCRTM shortly after Windows Server 2008 R2 release

Version Support ends Comment

SUS 1.0 Not supported Crazy old now. Don’t use.

WSUS2 RTM Not supported Updates still flow

WSUS2 SP1 Not supported EOL is April 9 2009 (now) -two years after WSUS3 RTM

WSUS3 RTM Not supported One year after WSUS3 SP1

WSUS3 SP1 TBD One year after WSUS3 SP2

WSUS 3.0 SP1/SP2 Adds FeaturesWSUS 3 SP1 adds the following features:

Installs on Server 2008, integrated with Server Manager (after installing Server Manager update KB940518) API enhancements for advanced management toolsBug fixes

WSUS 3 SP2 will add:Installs on Server 2008 R2 betaSupports managing Win7 clientsSupport for BranchCacheAuto-approval rules with deadlinesBug fixes (DSS gets languages from USS, target groups sorted alphabetically, more robust setup upgrade)(RC) Compliance against approved updates

New Features in WSUS SP2demo

Elements of ArchitectureWhy Architecture?

Problems are usually results of improper architectureA correct architecture will drive a better design

Especially in situations of administrator distrust or insufficient bandwidth

Design your WSUS solution with the same goals as your AD solutionRoaming users should be dealt with separately

“Simple” ArchitectureSingle, well-connected site

WSUS Updates from MUClients update from WSUS

Single server can handle 25,000 clients50K clients with 2x front-end servers and big SQL back-end

Remote SQL configuration reduces server loadFront-end handles update sync loadBack-end handles reporting load

“Simple, with Groups” ArchitectureLargest use case in production todayDriving forces to move to Machine Groups:

Differing patching requirements or schedulesTest groupsServers vs. WorkstationsPolitics

Not necessarily used for load distribution

WSUS Chaining

Chaining involves downstream servers getting updates (and sometimes Group data) from upstream serversOptions for chaining

Distributed vs. Centralized model“Autonomous Mode” vs. “Replica Mode”

Chaining solves the problem of “mesh” or “fully independent” architectures

Wastes resources and bandwidthNot that some situations don’t mandate “mesh” or “fully independent” architectures!

“Centralized” Architecture

Downstream servers are replicas of primary serverLittle downstream control over servers

Downstream administrators drop machines into predefined groupsAll update approvals and schedule done at primary server

“Distributed” Architecture

Downstream servers obtain updates from primary server, except:

Update approvals do not flow down. Assigned at each site individuallyDownstream admins have greater control. Can create groups and assign approvals

Used for distribution rather than control of updates

Combinations of centralized anddistributed possible. Depends on

intra-IT trust model.

“Disconnected” Architecture

Many environments don’t have Internet connectivityTest/dev, government, classified, air gap environments

Data must be imported from “the outside”Any the previous architectures will work

Manual import process requiredGives CM/QA/Security the option to review updates prior to bringing “inside”

Sneakernet

“Disconnected” ArchitectureMatch advanced options between source and target

Express installation files & languages must matchBackup and restore updates from source to target

Back up C:\WSUS\WSUSContentRestore to the same location on the target server

Transfer update metadata from source to targetNavigate to C:\Program Files\Update Services\ToolsExport metadata using wsusutil.exe export {packageName} {logFile}Import with wsusutil.exe import {packageName} {logFile}packageName & logFile are unique names you choose

Database validation can take multiple hours to complete!

“Roaming” Architecture

Manages updates for external resources

WSUS servers distribute approval metadataClients download updates from Windows Update directlyExtra security for internet-facing WSUS server

Useful separate architecture for mostly off-net clients

Laptop WSUS

Laptops

“Roaming” Architecture

Four Steps to Internet-facing WSUS

Build server in DMZ and position behind ISA proxyLocate database on server not reachable from InternetEnable SSL for communicationsHost content on Microsoft Update

Laptop WSUS

Laptops

“High Availability” Architecture

WSUS 3.0 includes native support for high availabilityNLB Clusters connect multiple WSUS web servers via a single cluster IPSQL Cluster manages the databaseNo single point of failure

Critical: This design isuseful for availability, but does little for performance

Managing Branch Offices

Branch offices are typically managed through replica WSUS servers

Replica servers take all orders from the central serverSettings at the top flow downward, but take time

Alternatively, unify architecture through a single “central server”

Single server manages all clients across all officesDeploy ISA proxy in the branchEnable BITS peer-cachingUse delta files to reduce network traffic

10x more server disk space4x less client download

Upgrade DeploymentWSUS 3 SP1 setup supports in-place upgrade

One-way upgrade (no rollback)Can’t be done from WSUS 2 on Server 2000 or using SQL 2000

Alternative is migration upgrade:Install second serverIf original server is WSUS2 SP1:

Perform disconnected replica steps (wsusutil, ntbackup, wsusmigrate)Switch over client via policy

If original server is also WSUS3Configure new server to be a replica of the first and syncAfter sync, configure new server to be autonomous

Upgrade hierarchy from top down

Troubleshooting WSUSpart 2

Errors and Error Codes

Numerous WSUS error codes existA complete list of all WSUS error codes is available on-line at http://inetexplorer.mvps.org/archive/ windows_update_codes.htm

For example, 0x8DDD0018 occurs when one of these services is disabled

Automatic UpdatesBITSEvent Log

Errors and Error Codes II

0x80072EE2, 0x80072EFDThis issue occurs because the Windows Update client did not receive a timely response from the Windows Update Web site serverLikely a proxy configuration, personal firewall, or trusted hosts problem

Errors and Error Codes III

0x80246008, 0x8024402CCaused by BITS malfunctioning or corruptedDownload and extract the BITSAdmin tool from the Windows Support Tools CDBitsadmin /util /repairservice /forceIf that doesn’t work, try a BITS re-install

Though if you do a BITS re-install, clear out the %SystemRoot%\SoftwareDistribution folder and reboot when done

Its worth mentioning here that thereis no “backup” download process for WUA.

…like HTTP or FTP…

If BITS is non-functional, so is patching!

Errors and Error Codes IV0x80244019

This error is often caused when the Proxy server is not properly configured.Ensure that your Proxy server allows Anonymous access to these external addresses:

http://windowsupdate.microsoft.com http://*.windowsupdate.microsoft.com https://*.windowsupdate.microsoft.com http://*.update.microsoft.com https://*.update.microsoft.com http://*.windowsupdate.com http://download.windowsupdate.comhttp://download.microsoft.com http://*.download.windowsupdate.com http://wustat.windows.com http://ntservicepack.microsoft.com

Microsoft doesnot publish the IP’s

associated with theseFQDN’s.

So, if you do perimeternetwork security by IP

you’ve gotta’ stayon the ball with these!

WUA Client IssuesTo enable auto-updates, ensure:

Anonymous access granted to Self Update virtual directory on WSUS serverAuto-updates requires TCP/80 to function on WSUS server

Be aware of GP replication times90 to 120 minute GP refresh timing will impact speed of clients becoming visible in WSUS admin tool

Be aware of AU detection frequency timesWUA client set to check with server every 22 hours (minus offset).When WUA checks in is when it checks WUA versionNeed to do wuauclt /detectnow to force this to occur on-demand

WUA Client Issues IIKnown issue with imaged workstations:

If you image your workstations (and who doesn’t these days!), you must change SID

Sysinternals NewSID, Microsoft SysPrepNot doing this will prevent WUA from contacting WSUS

To fix this problem:Run one of the above tools to change the SIDHKLM\Software\Microsoft\Windows\ CurrentVersion\WindowsUpdateDelete PingID, SUSClientID, and AccountDomainSID valuesRestart wususerv serviceRun wuauclt /resetauthorization /detectnow

WUA Client Issues IIIDisabling the Automatic Updates Service or the BITS Service at any point in the past prevents it from starting properly when you need it!

Reset permissions on these services to re-enable functionality.Use the Service Control Resource Kit tool (sc.exe) to do this:

sc sdset bits "D:(A;;CCLCSWRPWPDTLOCRRC;;;SY)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BA)(A;;CCLCSWLOCRRC;;;AU)(A;;CCLCSWRPWPDTLOCRRC;;;PU)"sc sdset wuauserv "D:(A;;CCLCSWRPWPDTLOCRRC;;;SY)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BA)(A;;CCLCSWLOCRRC;;;AU)(A;;CCLCSWRPWPDTLOCRRC;;;PU)“

Every disabled client needs this!

Tips and Tricks for Using WSUSpart 3

Optimize Patch DistributionIn large, multi-site environments low bandwidth may cause problems for remote offices

Distributing updates to downstream servers is big problem

Potential solutions:Ensure downloading only the languages you needConfigure patch distribution to occur in the eveningsStagger patch distributions between tiered sitesExpress installation files can exacerbate this

The bandwidth savings in express installation files occurs from WSUS server to client, not between WSUS servers

Throttle BITS

Throttling BITSBITS can be throttled either on the WSUS server or additionally on all the clients

Alleviates network saturation during update distribution and during client installationBe aware that this does slow down update distributions!

Throttle BITS in Group Policy:Computer Configuration | Administrative Templates | Network | Background Intelligent Transfer ServiceTwo settings:

Maximum network bandwidth that BITS usesLimit by Kbps based on time of day or at all timesBe aware that Kbps is kiloBITS not kiloBYTES (divide by 8)

Timeout (in days) for inactive jobs

DNS Netmask Ordering

Non-centralized architectures can better route clients through DNS Netmask ordering

Microsoft DNS Round Robin will first provide an IP address in the same subnet as the requestorIf no IP exists in the same subnet, a random IP will be selected

All WSUS hosts must respond to the same FQDNDNS FQDN record is populated with IP addresses of all WSUS servers in the network

Server TuningRun cleanup and DB defrag every few months

Cleanup wizard is a new feature in WSUS 3Removes stale computers and updates

DB index defrag script available on ScriptCenterkeeps the server running fast

Look out:Take care to not remove computers that are still active (but having trouble contacting the server)

Populate from AD sample tool can helpIn a hierarchy, need to run cleanup on each WSUS server.

Clean computers from bottom-upClean updates from top-down (or between sync intervals)Can be automated through the API

Considerations for Updating ServersServers require more care than workstations…

A rebuild is usually not an acceptable solution for a failed patch installationOutage windows are shorter

But in some ways servers are easier…Data and system drives usually separatedHardware configuration is usually more stable or well-understoodService isolation and redundancy – in larger environments – limits exposure/riskPeople typically aren’t “surfing” on serversThe RAID 1 Undo Trick…

What About Reboots?I’ve said this before, and I’ll say it again:

“If you have a patch management plan without a reboot strategy, you don’t have a patch management plan.”

Three methods:Client-initiatedWSUS-initiatedScript-initiated

Two methodologies:Scheduled reboots vs. rebooting for patch installation

I will argue in favor of scheduled, forced reboots

over mid-day reboots.

Handling RebootsRebootFile = "computers.txt“LogFile = "results.txt"Set fso = CreateObject("Scripting.FileSystemObject")Set f = fso.OpenTextFile(RebootFile, 1, True)Set objTextFile = fso.OpenTextFile(LogFile, 2, True)

On Error resume nextDo While f.AtEndOfLine <> True

strComputer = f.ReadLineSet objWMIService = GetObject("winmgmts:" & _

"{impersonationLevel=impersonate}!\\" & strComputer & "\root\cimv2")If Err.Number <> 0 Then

objTextFile.WriteLine(strComputer & " is not responding.")

Err.ClearElse

Set colOperatingSystems = objWMIService.ExecQuery("Select * from _ Win32_OperatingSystem")

objTextFile.WriteLine(strComputer & " is rebooting.")For Each objOperatingSystem in colOperatingSystemsObjOperatingSystem.Reboot()Next

End IfLoop

Custom Reports

UI supports basic customization (filters)Advanced customization can be built on

WSUS (.Net) APICan use of PowerShell scripts to generate reports

Public read-only SQL viewsCan use SSRS to generate reports (if full SQL)

Samples available from MSDNE.g., compliance against approved updates

Match KBs to MSRCs

Ever wish you had a nice mapping of knowledgebase numbers to MSRC numbers?

“The Q-numbers to the MS-numbers”

This script outputs a .CSV file that provides just that mapping

Add the name of your WSUS server into the top line of the script: strWSUSServer = “<Enter WSUS Server here>"

Match KBs to MSRCsstrWSUSServer = “<Enter WSUS Server here>"

Set fso = CreateObject("Scripting.FileSystemObject")Set objTextFile = fso.OpenTextFile("OUTPUT.csv", 2, True)objTextFile.WriteLine("MS Number,Q Number")

Set conn = CreateObject("ADODB.Connection")Set rs = CreateObject("ADODB.Recordset")dbconn = "Driver={SQL Server};Server=" & strWSUSServer & ";Database=SUSDB"conn.open dbconn

strSQLQuery = "SELECT dbo.tbSecurityBulletinForRevision.SecurityBulletinID, dbo.tbLocalizedProperty.Title FROM dbo.tbLocalizedPropertyForRevision INNER JOIN dbo.tbLocalizedProperty ON dbo.tbLocalizedPropertyForRevision.LocalizedPropertyID = dbo.tbLocalizedProperty.LocalizedPropertyID INNER JOIN dbo.tbSecurityBulletinForRevision ON dbo.tbLocalizedPropertyForRevision.RevisionID = dbo.tbSecurityBulletinForRevision.RevisionID WHERE (dbo.tbLocalizedPropertyForRevision.LanguageID = 1033) ORDER BY dbo.tbSecurityBulletinForRevision.SecurityBulletinID"rs.Open strSQLQuery, conn, 3, 3

While Not rs.EOFobjTextFile.WriteLine(rs.Fields(0).Value & "," &

Replace(rs.Fields(1).Value, ",", ""))rs.MoveNext

Wend

WScript.Echo "Done!"

Agent Control

Use WUA API to control the agentCustom install schedulesUpdating servers in web farmsImplementing “install now” functionality

On-Demand Patching(You Patch Now!)

Ever wish you had a WSUS “big red button”?Such a button might automatically download and install all approved patches and reboot if necessary…

How about this VBScript?Run this script from any server consoleImmediately downloads and installs all approved patches.If a reboot is required, it will then reboot the server.

The WSUS Big Red Button

Set fso = CreateObject("Scripting.FileSystemObject")Set objAutomaticUpdates = CreateObject("Microsoft.Update.AutoUpdate")objAutomaticUpdates.EnableServiceobjAutomaticUpdates.DetectNow

Set objSession = CreateObject("Microsoft.Update.Session")Set objSearcher = objSession.CreateUpdateSearcher()Set objResults = objSearcher.Search("IsInstalled=0 and Type='Software'")Set colUpdates = objResults.UpdatesSet objUpdatesToDownload = CreateObject("Microsoft.Update.UpdateColl")intUpdateCount = 0For i = 0 to colUpdates.Count - 1

intUpdateCount = intUpdateCount + 1Set objUpdate = colUpdates.Item(i)objUpdatesToDownload.Add(objUpdate)

Next

‘<<This is only the first half of the script. Add the code from the next page to ‘create the full script>>

The WSUS Big Red Button

‘<<Add this half to the code on the previous page!>>

If intUpdateCount = 0 ThenWScript.Quit

ElseSet objDownloader = objSession.CreateUpdateDownloader()objDownloader.Updates = objUpdatesToDownloadobjDownloader.Download()

Set objInstaller = objSession.CreateUpdateInstaller()objInstaller.Updates = objUpdatesToDownloadSet installationResult = objInstaller.Install()

Set objSysInfo = CreateObject("Microsoft.Update.SystemInfo")If objSysInfo.RebootRequired Then

Set objWMIService = GetObject("winmgmts:{impersonationLevel=impersonate,(Shutdown)}!\\localhost\root\cimv2")

Set colOperatingSystems = objWMIService.ExecQuery("Select * from Win32_OperatingSystem")

For Each objOperatingSystem in colOperatingSystemsobjOperatingSystem.Reboot()

NextEnd If

End If

Other API Uses

ISVs use APIs for many other features as wellDistribute 3rd party updates (quite complex)Gather software and hardware inventoryDistribute updates to non-Windows devices

Your starting point is http://technet.microsoft. com/en-us/wsus/bb466192.aspx

API SamplesDiagnostic ToolsHeader Files

SummaryWSUS is simple to use, but scales to enterpriseFlexible server deployment options

Single server, scale up, branch office, scale out, disconnected, roaming laptops

Flexible update deployment optionsPeer caching, delta patching, auto approval rules, auto-reapprove revisions

Periodically tune the server (defrag + cleanup)Public API and DB views can be used to extend the base functionality for many advanced scenariosStarting point for all WSUS information

http://www.microsoft.com/updateservices

question & answer

www.microsoft.com/teched

Sessions On-Demand & Community

http://microsoft.com/technet

Resources for IT Professionals

http://microsoft.com/msdn

Resources for Developers

www.microsoft.com/learningMicrosoft Certification and Training Resources

www.microsoft.com/learning

Microsoft Certification & Training Resources

Resources

Windows Server ResourcesMake sure you pick up your copy of Windows Server 2008 R2 RC from the Materials Distribution Counter

Learn More about Windows Server 2008 R2: www.microsoft.com/WindowsServer2008R2

Technical Learning Center (Orange Section): Highlighting Windows Server 2008 and R2 technologies•Over 15 booths and experts from Microsoft and our partners

Complete an evaluation on CommNet and enter to win!

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,

IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.