Date post: | 14-Apr-2017 |
Category: |
Software |
Upload: | sergio-dutra |
View: | 224 times |
Download: | 1 times |
TESTING AS A RISK CONTROL STRATEGYSergio DutraSoftware Quality Expert
FOUR STAGES OF SOFTWARE TESTING COMPETENCY
STAGE 1: JUST TEST! Take what the creators give you and test it,
using manual or automated tests. Triage issues. Update tests as necessary. Analyse and report on results.
STAGE 2: BUILD A DEEP UNDERSTANDING OF THE PRODUCT, THEN TEST IT. Drive for clear requirements and design, even if
you have to create these yourself! Build models to understand how the system
works. Look beyond problems with the product,
including how it can be misused, or how its dependencies can fail. Create tools to identify common problems, such as
misconfigurations or network issues. These can be used not just for testing but for customers and partners.
STAGE 3: UNDERSTAND HOW THE PRODUCT WILL BE USED, THEN TEST IT. Participate in user research, customer support
and other customer engagement tasks, to understand how the product will be, or is, perceived by customers.
Consider the whole product lifecycle, including acquisition, initial configuration, typical usage, maintenance, upgrade and retirement.
Drive the definition of quality standards, processes, tools and community knowledge to support the product in the ecosystem.
STAGE 4: UNDERSTAND THE BUSINESS, AND USE TESTING TO REDUCE BUSINESS RISK.
Q: WHY DO BUSINESSES INVEST IN TESTING?A: Businesses want to reduce the risk in achieving their goals. Businesses want to avoid risks like: “Deal breakers” for the consumers. Exposure to liability from consumers, regulatory agencies or other parties. Tarnishing of the company’s reputation. Exposure of the business or its assets to threats, such as theft or abuse.
RISK ANALYSIS BASICS
Identify what controls are currently in place and which ones are needed. Design and implement any new controls needed.
Identify the business assets at risk. Assess potential threats to the business and prioritize them based on likelihood and impact. Determine actions to take on each threat. Then:
Accept the risk; Eliminate the threat; Reduce the impact; or Delegate the risk to a third party.
EXAMPLES OF BUSINESS RISKS Widely visible patterns of defects across
products or services provided, or used, by the business.
Unintended exposure of business IP or other sensitive assets (e.g. accounts, passwords, etc.)
Abuse of the product, its customers or partners. Non-compliance with critical standards, or with
legal or geopolitical requirements. Aspects of the ecosystem – including partners
and competitors – that effectively nullify benefits of the product.
EXAMPLE OF ECOSYSTEM THREATS: SSL SSL is a protocol that allows secure communication between a client and a
server. Its ecosystem includes: Cryptographic and public key infrastructure (PKI) components. Entities that issue and manage certificates. Browsers, networks and all other aspects of the network stack.
Some ecosystem issues that need to be considered include: Will fixes in cryptographic dependencies require updates in the protocol? How reliable is certificate issuance and management? What issues could arise,
and how can the protocol itself mitigate these? Are there ways for consumers to improperly configure SSL? Are the users of the browsers and apps made aware – in an effective manner –
that they are communicating securely, and that it’s important to do so in the context of their use?
THOUGHT EXPERIMENT – HIGH LEVEL RISK EVALUATION OF CANDY CRUSH SAGA Key revenue earners for Candy Crush Saga
Ability to charge money for certain game features (e.g. unlock levels) Attention share of users on the game and the ability to keep growing it
Asset Importance
Main Threats
The revenue earning features of the game
Highest 1. Features don’t work and generate no revenue2. Revenue aspect can be bypassed (front or back ends)3. Revenue can be redirected to other parties4. Customers can be charged for features that don’t work
Attention share features of the game
High 1. Game can be altered to introduce unwanted aspects2. Unable to collect usage information from customers 3. Usage information can be redirected to third parties
Intellectual property pertaining to the game
Moderate Third parties can effectively copy the game and attract game users to it
Sensitive operational assets
Highest Sensitive data – such as user identities or credit card data – is stolen, damaging users hence the profitability of the game
Disclaimer:
I have only common knowledge about how Candy Crush Saga works,
and no knowledge of how the company actually evaluates risk or
tests the application.
This exercise is provided merely as an example of how to approach
testing such an app.
ELEMENTS OF A RISK-BASED TESTING STRATEGY Prioritized list of risks to the business, within the
context of the product under test. Areas of focus and how they pertain to each
business risk. Methodology applied to each area of focus.
THOUGH EXPERIMENT – RISK-BASED TEST STRATEGY FOR CANDY CRUSH SAGABusiness risk Areas of focus MethodologiesFeatures don’t work and generate no revenue
Unlock a level; Add time; Add moves; Add lives.
Functional; App sleep/resume cycle; Network reliability.
Revenue earning features can be bypassed
Persistent app state affecting levels, time, moves and lives; Service entry point and protocols; Billing service.
State-based testing; penetration testing; service DoS; false billing data entry; billing service dependencies.
Revenue can be redirected to other parties
Billing service; service protocols.
Secure design evaluation; penetration testing; false billing data entry.