The Politics of Performance Engineering

At Mercury World 2004

By Wilson Mar

(wilsonmar.com)

Organizational Challenges

• “We’re not told about projects”

• “We’re left out of discussions”

• “We’re not given enough time to test”

• “We’re not sought out for advice”

• “Our warnings are ignored”

(Notice: These are passive statements)

Resources

• Joel R. DeLuca, Ph.D. http://www.politicalsavvy.com/Political Savvy: Systematic Approaches to Leadership Behind-the-Scenes ISBN: 0966763602x

• Eugene KaluzniackyManaging Psychological Factors in Information Systems Work : An Orientation to Emotional Intelligence (Hershey PA Idea Group Publishing, 2004) 101771 ISBN 1591401984

•

Definitions

• The mission of performance engineering is to identify and measure potential problems!

• We’re paid to be proactive (adjective)– taking the initiative by acting rather than

reacting to events (Encarta)

– taking action by causing change and not only reacting to change when it happens (Freesearch)

9 Ways to Be Proactive

1. Deliver (build trustworthiness)2. Open information flows3. Satisfy Stakeholders - know who wants what4. Follow the money (workflow)5. Speak their language, not your own6. Disclose fully7. Stay within your role8. Walk in their shoes9. Don’t gloat or gossip

1. Deliver

• Performance Test Plan– Application Architecture, Use Case, Data Flows– Application Risks and Test goals – Test scripts & Run parameters– Application & Data Configurations

• Performance Test Results– Response Times and Error Patterns

• Performance Profile Spreadsheet– Predicted Upgrade Points & Timing– Costs of Growth

• Project Status and Completion Projections

1.1 Performance Technical Profile

• The time it takes to start & stop the app.

• Speed timings:– for first use by single user– for subsequent uses and users– Degredation as volume of data increases for

a single user– Degredation as additional users are added

• Longevity:

2. Open Information Flows

• Before – Objectives, Milestones, Timelines

• Starting – Roles, Schedules, Metrics

• During – Progress, Projections, Drafts

• Ending – Sign-offs, Thank You’s

• After – Actual vs. Planned Comparison – Lessons Learned

• Throughout – Lunch, Snacks, Parties, Gifts

Knowledge vs. Wisdom

Satisfying Stakeholders

• Marketing– Ability to handle expected and possible loads.

• Finance– Projected Timing and Costs of Growth.

• Data/Network Operations Center– Expected bandwidth use (by network segment)

• Configuration Management (Archives)– Content of Packages

• Deployment Manager– Installation instructions and cautions

3.b Satisfy Stakeholders

• Developers using the app under test

• Trainers, Documentation Specialists

• Development Manager

• Development Lead

• Developers

• Functional Testers

4. Follow the Money

• Map people & projects– Organize to Org. charts (from Phone Book)– Find them before they hunt you down:

• Who are in the cubicles around you?• Introduce yourself. Mingle at office parties• Offer eats, magazine, website

• Map Technology Connections

• Map your own Use Cases

4.1 People & Projects

Director Manager Dept/Project

Need Status

• Performance engineers often bridge more technologies and organizational units than others.

4.2 Technology Connections

• Have production equivalent

• Have reference services and data available before need. Examples:– User authentication service– Security service to provide SQL connection

strings.– Legal documents in applications

4.3 Performance Use Case

5. Speak Their Language

a. Identify application components and fields using names assigned by developers, not identifiers created by testers.

b. Cross reference:

Test Path Component TestScript

0. Invoke URL Index.asp Invoke

Speak visually - Illustrate

6. Disclose fully

• Disclose full test conditions:“1 out of 1,000 invoke transactions”“encountered an error when”“under a load of 15 simultaneous users”“with 0.5 seconds between transactions”

• Disclose full statistics:“Invoke transactions averaged 20 seconds,with a maximum of 41 and a minimum of 2 seconds.”

7. Stay within your role

• Let executives make judgments:– “It’s not many errors”– “It’s fast enough”– “Users won’t care”– “The change is not worth the risk”

• Is it your place to argue?

• Is it your place to enforce change?

• Who gave you a sheriff’s badge?

Exercise: If you were King

• Write a policy statement delegating authority to the Performance Engineer

• Consider:– Responsibility for reporting what?– Authority to delay what under what

circumstances?– Should be notified of what when by whom?

Mutually Exclusive Roles

An analogy:Are you …

– a family doctor whom a teenager can safely discuss issues she doesn’t feel comfortable discussing with her parents.

OR – a lab technician who takes urine samples for

pre-employment drug testing, with whom one does not usually converse.

The Assumptions Minefield

• Assumptions about roles and responsibilities colors perceptions and communications.

• Inconsistency in assumptions can secretly hinder trust, credibility, communication, cooperation, and effectiveness.

8. Walk In Their Shoes

• Don't ask questions you can answer yourself.

• Focus on solutions rather than problems

• Be skilled (certified) in the tools developers use.

• Read their books. Attend their classes.

• Get VPN access for evenings & weekends

Hidden Agendas

• “My system is perfect”– Response: “We’ll make it even more perfect”– Response: “I’m required to test it”

Methods of Communication

A. Within HTML text, extra blanks, tabs, and carriage return characters among text will have a slight impact on performance.

B. After user logs off, server continues to respond to previous URLs (obtained from browser’s “Back” button).

Offer Possibilities

• Number of file handles

• Time before used handles are released

9. Don’t gloat or gossip

• “We’re in the same boat”• Don’t say “You” or label/nickname people• Instead of asking “why don’t you…

Ask “Is it possible to …• Communicate minor issues quietly (grouped in a

list along with other items, not in a separate/individual email)

• Deliver bad news in person and in private• Don’t tell others about test results• Preview status reports with the team before the

management

Remember: Be Proactive

“You can get everything you want if

you give enough other people

what they want.”– Zig Zigler

“Get close to your friends.

Get even closer to your enemies.”– ?