Post on 22-Dec-2015
transcript
How to Improve Driver Quality With Winqual / WHQL
Scott HerrboldtDirectorWindows Hardware Online Services (WHOS)scotther @ microsoft.comMicrosoft Corporation
Session OutlineSession Outline
Windows Hardware Online Services
State of Driver Quality Today
Participating in the Designed for Windows (DFW) Logo program
Recent improvements
Windows codenamed “Longhorn” Partner Logo Program
Using Windows Error ReportsResponding to customer issues
Updated tools now available
Using Windows UpdateCustomer feedback update
The right driver for the right machine
Planned expansion of Windows Update Content for Longhorn
Windows Hardware Online ServicesWindows Hardware Online Services
The Windows Hardware Online Services (WHOS) team focuses on helping IHV and OEM partners create, support and deliver high quality products running Windows
Key Services offered:Windows Hardware Quality Labs (WHQL)
Winqual.Microsoft.comProduct testing submissions
Windows Error Reports
Driver Distribution Center for Windows Update
Windows Catalog & Marketplace Listings
State of Driver Quality - TrendsState of Driver Quality - Trends
March ‘04 March ‘05
# Category # Category
1 Display 17.9% 1 Display ▬ 17.9%
2 Pool/Memory Corruption 13.6% 2 Pool/Memory Corruption ▲ 15.9%
3 OS Core 13.5% 3 OS Core ▲ 14.2%
4 Networking 12.6% 4 Networking ▼ 10.8%
5 Multimedia 9.51% 5 Multimedia ▼ 7.68%
6 Application Drivers 6.57% 6 Application Drivers ▲ 7.51%
7 Anti-virus 5.54% 7 Hardware Failures ▲ 7.00%
8 CD-Burning 5.39% 8 Anti-virus ▼ 5.42%
9 Hardware Failures 4.99% 9 Storage ▲ 4.94%
10 Storage 4.60% 10 CD-Burning ▼ 4.28%
Customers are Not Getting Needed AssistanceCustomers are Not Getting Needed Assistance
Customer responses exist for only 48% of all crashes
Only 58% of those responses offer the user a solution
20% identify what driver or hardware caused the problem, but no solution is available
We need to work togetherShare symbols with Microsoft so we can better analyze customer crashes
Security & Privacy of symbols covered by legal agreements
Use Winqual to provide assistance to for our mutual customers
Customers Expect Better ReliabilityCustomers Expect Better Reliability
Freq % Cust % V-Sat % D-Sat
0 0% - -
1 33% 70% 3%
2 19% 53% 6%
3 12% 39% 9%
4 8% 40% 10%
5 6% 35% 5%
>5 23% 26% 21%
On average, how many times per month does Windows crash or experience serious bugs?
Driver reliability issues are costing you money
January 2003 survey of 754 homes and 503 small businesses
$0
$5,000
$10,000
$15,000
$20,000
$25,000
$30,000
$35,000
$40,000
$45,000
100 2000 4000 6000 8000
Derived Crash to PSS Cost Ratio (@ $30/call)
Crashes
A large OEM did a study of blue screen related calls
Result: OCA predicted blue screen-related calls on a 6:1 ratio
In a Microsoft survey, crash frequency was found to be highly correlated with decreases in customer satisfaction
Customers Expect Ease of UseCustomers Expect Ease of Use
Expectation
When I report a crash, I will receive help
When I plug in a new device, it works automatically
When a solution exists for a problem I’m having, Windows should automatically update me
Reality
52% of crashes still need solutions
Many product installation experiences require loading software manually
Only 19% of improved drivers that have been submitted to WHQL are available to users through Windows Update
Microsoft Service Investments in Driver QualityMicrosoft Service Investments in Driver Quality
Need
An understanding of what your top customer satisfaction issues are
Mechanisms to validate quality in your products
The ability to reach your customers when an update is needed
Solution
Windows Error Report data analysis for your products
Logo Programs, HCT & WDK, WHQL assistance
Driver Distribution Center for Windows Update
What tools does Microsoft provide to help meet increasing customer expectations?
WHQL Response & Supportability ImprovementsWHQL Response & Supportability Improvements
Dramatic simplification of tech support
Reduced complexity in contacting WHQL support
2-tier support model
Reducing turnaround and improving response quality
Latest HCT releases address most critical errata
Planning regular service pack cycle focused on errata reduction
Top 36 Errata fixed
WHQL Program Improvements & ExpansionWHQL Program Improvements & Expansion
Open for x64 driver businessHCT 12.1.01 released for testing x64 drivers
WHQL is now accepting x64 submissions
x64 submissions free of charge at this time
Improved Universal Test ProcessFor devices where formal logo program does not exist
Test process is changing from full-test to self-test effective May 1, 2005
Submission fees reduced to normal self-test fees
Turnaround time is now 7 vs. 30 days
RequirementsRun & pass the universal section of HCT 12.1.01
Sign the Windows Error Reports Terms of Use agreement
Provide symbols
Introducing the “Premium" Logo Program Introducing the “Premium" Logo Program
A new qualification level for the Longhorn hardware logo program
The “Premium" program will recognize products that deliver the premium Longhorn experience
The “Standard” program equates to today’s DFW program
New level created in response to partner feedback and Microsoft improvement efforts
Need to differentiate products
Relevance of requirements to an evolving industry
Harmonize the different Microsoft logo programs
A Foundation for Future Driver Quality ProgramsA Foundation for Future Driver Quality Programs
Microsoft is producing a set of device class independent tests and tools* to verify driver quality
Drivers that successfully pass these tests will be digitally signed, regardless of logo program participation
““Premium”Premium”
Req
uire
men
ts b
uild
on
each
oth
er
““Standard”Standard”
““Quality”Quality”
Introducing Driver Quality Rating (DQR)Introducing Driver Quality Rating (DQR)
Method of measuring driver quality in the fieldDerived from mathematical probability that a given driver caused a system crash
Data is normalized to treat low and high volume drivers equally
The DQR will be a score from 1 to 9Values 1 to 3 (Red) indicates a high level of problems
Values 4 to 6 (Yellow) indicates a medium level of problems
Values 7 to 9 (Green) indicates a low level of problems
DQR for your products will be available to you on Winqual
Uses of DQRPrioritize driver maintenance work
Help OEMs update preloads for optimal user experience
Gold logo requirement
Clarify Windows Update content policy
Industry DQR Distribution TodayIndustry DQR Distribution Today
Most drivers on today’s scale fall into the green category
91% Green
5% Yellow
5% Red
Providing symbols is critical for an accurate DQR
Providing symbols will likely increase your DQR (higher is better)
Rating Percent of Drivers*
1 2.43%
2 0.75%
3 1.31%
4 1.08%
5 1.83%
6 1.53%
7 5.35%
8 12.19%
9 73.53%
*Most popular drivers from Feb 2005
Using Windows Error ReportsUsing Windows Error Reports
Kernel mode crash data is updated hourly on Winqual
Example of Realized ValueRelease of driver to web increased crashes by 10x in 3 days
In this example it took a number of days to catch problem
This company now uses Winqual to monitor new releases of drivers in real time
Using Windows Error Reports to AssistUsing Windows Error Reports to Assist
Through Winqual you can provide customized responses to customer who are experiencing crashing problems
Using Windows Error Reports to AssistUsing Windows Error Reports to Assist
Microsoft regularly surveys customers and will work with you to provide the best possible customer experience
Announcing Windows Error Reports 4.0Announcing Windows Error Reports 4.0
Improved UI including navigation and shrink & grow sections
New chart modeComparison against industry average
Various trending views
New result set filtering with over 20 new filtering attributes
Now search by bucket, driver, device and PnP ID
New data drill-downs by vendor, driver, and device
Improved bucket details
New CAB details with call stacks, in memory processes and system uptime
Filter CAB files prior to downloading
Available NOW on http://Winqual.Microsoft.com
Windows Update Operational FactsWindows Update Operational Facts
Total Investment $60+ million per year (HW, FTE’s, egress)
$23 million FY’04 egress bandwidth
Data Centers,
Download Servers and Egress
3 Data Centers
436 Servers
2000+ Caching Servers
24 Languages
100Gb/s peak egress capacity
WUv5 Site AvailabilityMicrosoft Domain Availability
99.85% (Keynote report 2004)
Top most available site/service (Keynote) when compared with AOL, Google
Cost Effective Distribution Service
Technological advantages
Economics of scale
Inexpensive in-house hosting
Relationships with CDNs, ISPs
Average Monthly Driver Downloads(Nov04-Mar05)
20,545,964
Customers Requesting Hardware Updates on WUCustomers Requesting Hardware Updates on WU
Consumers rank most important to update:1. Windows
2. Anti-virus
3. Hardware
4. Anti-spam
5. Office
Corporations rank most important to update:1. Outlook/Exchange
2. ISA Server
3. MBSA
4. .NET Framework
5. SQL
Request6/03Rank
12/04Rank
More hardware updates 2 1
Better information 3 2
More update types 1 3
Speed up the process 4 4
Fewer reboots 6 5
Filter updates 5 6
Misc. smaller updates, CD) 7 7
Effectiveness of Windows UpdateEffectiveness of Windows Update
Huge impact on driver quality when we partner for the fix and get the right driver on Windows Update
Audio Driver ExampleFix available for everyone in February 2004
Worked with a key OEM to put fix on WU as a critical update
0%
20%
40%
60%
80%
100%
120%
2003Nov
2003Dec
2004Jan
2004Feb
2004Mar
2004Apr
2004May
2004Jun
2004Jul
2004Aug
2004Sep
2004Oct
2004Nov
2004Dec
2005Jan
All Others OEM with WU Fix
WU Critical Fix released
Drivers on Windows Update TodayDrivers on Windows Update Today
Increased reach with WU v5 release at Windows XP SP2
Hosting 1000’s of driver packages from 100’s of companies
44 active driver packages being tracked for OCA reduction
Drivers can be posted to WU through Driver Distribution Center on Winqual
Critical Driver UpdatesPush critical OCA fixes down to customers
Currently 18 critical update packages posted for drivers
Contact whqlwu @ microsoft.com to participate
Rich Targeting on Windows UpdateRich Targeting on Windows Update
ChallengesOEM/IHV wants custom drivers on generic hardware that supports custom value-add software
Driver is specific to a premium sku of Windows (MCE, Tablet)
Driver should be critical for customers who are experiencing problems, recommended for everyone else
GoalsAllow driver packages to specify granular applicability criteria beyond PnP ID
Provide flexibility in detection mechanism without adding arbitrary code
Make it simple for driver package developers
Rich Targeting Rich Targeting
Technology basisWindows Management Instrumentation (WMI)
A component of Windows that provides management information and control in an enterprise environmentExposes Windows component properties through a consistent object-oriented interface
WMI queries are similar to (SQL) queriesAvailable fields
Win32_ComputerSystem.ManufacturerWin32_ComputerSystem.ModelWin32_BIOS.SMBIOSBIOSVersionWin32_BIOS.Name
Available operators=<> or !=LIKE
Acceptable query valuesVendor provided text strings
Rich Targeting – How It WorksRich Targeting – How It Works
USB\VID_045E&PID_008COEM: Unknown
USB\VID_045E&PID_008COEM: ComputerCo
Standard Retail driver forUSB\VID_045E&PID_008C
Custom OEM driver forUSB\VID_045E&PID_008C
Windows Windows UpdateUpdate
Announcing the Online Provisioning InitiativeAnnouncing the Online Provisioning Initiative
Microsoft plans to expand Windows Update to support all device-related software in Longhorn
What this means:Your device software product can be distributed through Windows Update, not just the device driver portion
Goals:“It Just Works” - deliver an integrated end user experience that makes devices fully functional with Windows automatically
Adjust content policies to balance customer, partner & Microsoft needs
What you need to do:Include your full product software in your WHQL submissions
Testing requirements remain unchanged, focus on the driver
Move to DIFx based setup packages
Planned Improvements to Longhorn Driver SearchPlanned Improvements to Longhorn Driver Search
Windows Update treated as an extension of inbox driver support
Online Provisioning of device related software ensures full functionality after install
If support not found customer can be routed to 3rd party website
Call To ActionCall To Action
Improve Driver Quality using Winqual / WHQLUnderstand what your top driver quality issues are
Provide symbols so we can better analyze your crashes and provide you with an accurate DQR
Distribute fixes to customers via Windows Update
Move forward with your x64 driver plans, demand is growing
Provide feedback on the "Gold" logo requirements
Will be available in the draft 0.6 version of the Windows Logo Program 3.0 requirements document
Include your full product software in your WHQL submissions
Community ResourcesCommunity Resources
Windows Hardware & Driver Central (WHDC)www.microsoft.com/whdc/default.mspx
Technical Communitieswww.microsoft.com/communities/products/default.mspx
Non-Microsoft Community Siteswww.microsoft.com/communities/related/default.mspx
Microsoft Public Newsgroupswww.microsoft.com/communities/newsgroups
Technical Chats and Webcastswww.microsoft.com/communities/chats/default.mspx
www.microsoft.com/webcasts
Microsoft Blogswww.microsoft.com/communities/blogs
Additional ResourcesAdditional Resources
Email: winqual @ microsoft.com
Web Resources:Tools: http://Winqual.Microsoft.com
Information: http://www.Microsoft.com/WHDC/
Related SessionsBusiness Value of the Windows Device Software Life Cycle
The Next Generation Logo Program An Introduction
Windows Driver Quality Signature
The WDK for Engineering Managers and Product Planners: An Introduction
DMI: Driver Install Frameworks Tools and Strategies