Alan PageMicrosoft

Software Testing at Microsoft:

Microsoft ProductsNearly 300 Products Released in last 12 Months

Some Numbers

More than 90,000 Employees World Wide

More than 38,000 Engineers

Nearly 10,000 Testers

Testing at Microsoft

First tester hired in 1979 (high school intern)

First full-time tester hired in 1983

First “wave” of tester hiring in 1985

In the beginning…

“You will design, execute and document tests of application software”

Requirements: background in math, computers, programming

SDETs at Microsoft

Emphasis on Technical SkillsCode to understand and solve problems

SDETs at Microsoft

Equal Career PathsTest / Development / Management

Development

Test

Team Makeup: Feature Team Triad

Program Management

Team Makeup: The Quartet for Software as a Service

Operations

Development

Test

Program Management

VP or General Manager

Product Unit Manager

Dev Test PM

Product Unit Manger

Dev Test PM

Organizational Structures (1)"The PUMdom"

President or Senior Vice President

GM or VP of Dev

Dev Managers Architects

GM or VP of Test

Test Managers

Test Architects

GM or VP of Program Mgmt

PM Managers Etc.

Organizational Structures (2)"The Triad"

Software DevelopmentWorkflow and Test Execution

Abort Checkin

Test CITsDistributed

Submit to Source Depot

Dev or Test?

Dev Test

Test CITsDistributed

No

Yes

Pri0 Bug Exists?

Abort Checkin

PassedFailed Submit

Build product / test*

Dev CITs Distributed

Create Pri0 BugEmail Test Owner

No

Yes

Pri0 Bug Exists?

The Future

TestingChallenges

or – variations on the same old stuff

Testing Challenges

Over-testing, under-testingand

millions and millions of tests

Testing Challenges

Data, data, data

Testing Challenges

ConcurrencyScale

Screens and the Cloud

Testing Challenges

The test / development relationship

andorganizational agility

Testing Challenges

Community and collaboration-

If only we knew what we knew

Productivity GamesRecapturing discretionary

time

http://productivitygames.blogspot.com/http://seriousgames.org/

Ribbon Hero

Localization Testing

• Hard, Large-Scale Problem

• Windows 7 ships in 100 languages

• Thousands of Strings and Screens per Release

link

Amplify Skill with Volume

Individual dialects, nuances, hard to detect with a single vendor – crowd does a better job

Reduce cost with Discovery & Instrumentation

No need to install

Telemetry to direct effort

Results

Significant Quality Improvements for Windows 7Positive Impact on Ship Schedule

Team Morale and Subsidiary Engagement

Total Screens Reviewed: Over 500,000

Total Number of Reviewers: Over 4,500

Screens per Reviewer: Average 119

http://angryweasel.com/bloghttp://twitter.com/alanpage

Photo Credits

Microsoft street sign - Some rights reserved by TechFlash Todd

Microsoft photo - Some rights reserved by Robert Scoble

The Future - Some rights reserved by PVBroadz