Date post: | 24-Jun-2015 |
Category: |
Technology |
Upload: | eurostar-software-testing-conference |
View: | 143 times |
Download: | 1 times |
© 2011 CTG, Inc.
Playing around with Risks
Jurgen Cleuren
Nov. 24th 2011
Introduction
• Projects are done in a probabilistic environment
– Incomplete information
– Parameters change over time
– What is true in the beginning of the project, can be false some time
later
• Games in a probabilistic environment
– Incomplete information (cardgames,…)
– Random element (dice, cards,…)
Which games ?
• Poker
• Monopoly
• Backgammon
Succes:
First learn the rules, then play better than everyone else
- Albert Einstein -
• Rules -> requirements
• Every game starts with learning and agreeing on the rules -> every project
starts with defining and agreeing on the requirements
TEXAS HOLD’EM POKER
Rules of Texas Hold’em
• 2 Blind cards
• Betting round
• 3 Open Community cards (flop)
• Betting round
• 1 Open Community card (Turn)
• Betting Round
• 1 Open Community Card (River)
• Betting Round
• Best hand of 5 cards wins
Hand Ranking
• Highest Card
• Pair
• 2 Pair
• 3 of a kind
• Straight (5 consecutive cards)
• Flush (5 cards of the same suit)
• Full House (3 and 2 )
• 4 of kind
• Straight Flush (5 consecutive cards of the same suit)
General test flow vs Poker round
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Bugfixing
• New Software Delivery
• Retesting and regression
• Bugfixing
• Final Software Version
• Regression Test
• Get Blind cards – evaluate risk - Bet
• Flop (3 cards)
• Re-evalute hand and Risk
• Betting round
• Turn (1 card)
• Re-evaluate hand and Risk
• Betting round
• River (1 card)
• Final bets and showdown
Similarities / Differences
3 Recurring phases
1 big phase (Complete Functional testing vs Flop)
2 small phases (retest + regression vs Turn + River)
Determine Risk before 1st test run (Risk Matrix or FTT vs 1st bet)
Adapt Risk Matrix or FTT after each test run
Should the FTT or Risk Matrix be adapted ?
• Every Test run gives new information
• Likelihood of risks change
– Failed test cases get a higher likelihood
– Passed test cases in unchanged code have a lower likelihood
• Closer to deadline => Risks can change
• Reporting is more clear
– Each Risk Matrix/FTT tree is a snapshot of the project at a certain time
Who ?
• Test manager should take the lead
• Preferably Project Board
• Test manager can do it by himself, but the board should at least be aware
that this activity is done
Justification
• A Poker hand needs justification at any point in the game
– Having the best hand in the beginning doesn’t imply that you are going
to win
– You must be prepared to fold when you are not winning anymore
– The money you’ve already bet doesn’t count
• It is in the pot so it is not your money anymore
• Don’t put more money in a losing hand
– No justification anymore: get out of the hand
– Cut your losses
Project justification
• A project needs justification at any point in time
• During testing: justification after each test run
– New information is given
– Risks are updated
• No Justification means project should be closed
• Previous investments do not count
– Don’t put more money in a failing project
• Cut your losses
Test process justification
• Get Blind cards – evaluate risk - Bet
• Flop (3 cards)
• Re-evalute hand and Risk
• Betting round
• Turn (1 card)
• Re-evaluate hand and Risk
• Betting round
• River (1 card)
• Final bets and showdown
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Justification
• Bugfixing
• New Software Delivery
• Retesting and regression
• Justification
• Bugfixing
• Final Software Version
• Regression Test
• Justification
Result Oriented Thinking
• A decision can be right even if the outcome is not favourable
• Focus more on the decision and the ‘why’ instead of the result
– Tester A completes a test set in 4 days by skipping tests
– Tester B completes the same test set in 6 days through thoroughly
testing
– No defects were discovered in production
– Who would you reward the most ?
Thought process
• In poker, to anticipate and understand your opponents actions, you need
to think as your opponent and not as you in his situation
• What motivates you, does not necessarily motivate another person
• Successful people ask better questions
– WHY? is more important than HOW? or WHO?
“Someone who knows HOW will always have a job
Someone who knows WHY will always be his boss”
Luck
“ Luck is when preparation meets opportunity”
- Seneca –
• Be prepared to get lucky
– In poker, sometimes you need to get lucky. When you get lucky, be sure
to take a big pot out of it.
– Sometimes a best case scenario happens, but we need to take
advantage of it.
– Being ahead of planning is more valuable if you can actually do
something with this situation
MONOPOLY
Aspects of the game
• Investing in houses
• The higher the investment, the bigger the payoff
• Some spaces have a higher probability
• Random element: dice
Analogy to Risk Based Testing
• Different probability of landing on spaces = Likelihood
• Higher Payoff = Impact
• What can monopoly teach us ?
– What is more important: Impact or Likelihood ?
Impact and likelihood on the board
Winning Monopoly strategies
• Complete Orange Colour group and invest as soon as possible
– Why Orange ? It has the biggest probability of other players landing on
it
– Be prepared to even trade down to get this colour group
• Complete Red Colour group as a second priority
– Why Red ? It has the second biggest probability of other players
landing on it
What lessons are to be learned
• Likelihood >>>>>> impact
– The weight of Likelihood should be > 50%
– The weight of Impact should be < 50%
BACKGAMMON
Backgammon
Important rules of Backgammon
• Goal: get all your tiles from one end to the other
• Only tiles that are standing alone on a pillar can be captured
• A captured tile has to be brought back in play at the beginning of the
board
• Random element: Dice
Translation to risks
• Your own checkers: Risks
• Opponents checkers: Causes
• Whenever one of your checkers is captured it’s in fact a cause hitting a
risk
• A risk is mitigated when the checker cannot be captured (2 or more on
one pillar)
Translation of risk priority
• Low risk: checker in the first quadrant
• Medium risk: checker in the second or third quadrant
• High risk: checker in the fourth quadrant
What can Backgammon teach us?
• Which risks should be mitigated and which are low priority ?
• There might be a possibility to remove a cause, but in the same way
creating a new risk. Should we do it ?
– Eg: Back-up testmanager vs full time testing
Backgammon Strategies
• Checkers in the 1st quadrant don’t have to be protected. Moving forward
is the better play.
Þ Risks with low priority don’t have to be mitigated. The correcting cost is
usually way less than the mitigation costs
• Checkers in the 4th quadrant need to be protected at all costs.
Þ Risk with high priority need to be mitigated at all costs
• For checkers in the 2nd and 3rd following rule applies: Always take a
chance to capture
Þ If you can remove a cause and therefore create a medium of low risk,
do so
CONCLUSIONS
Conclusions
• What Poker taught us:
– Adapt FTT/Risk tree after each test run
– Priorities of test items can change
– Justification has to be done after each test run
– Don’t focus on results, focus on decisions
– Be prepared to get lucky!
Conclusions
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Bugfixing
• New Software Delivery
• Retesting and regression
• Bugfixing
• Final Software Version
• Regression Test
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Adapt FTT/Risk Matrix
• Justification
• Bugfixing
• New Software Delivery
• Retesting and regression
• Adapt FTT/Risk Matrix
• Justification
• Bugfixing
• Final Software Version
• Regression Test
• Adapt FTT/Risk Matrix
• Justification
Conclusions
• Monopoly
– Likelihood >>>>> Impact
• Backgammon
– Don’t mitigate low priority risk
– Always mitigate high priority risks
– Removing a cause and creating a medium or low risk is the way to play
QUESTIONS AND ANSWERSJurgen Cleuren ([email protected])