Ibm Smarter Quality Management

8/2/2019 Ibm Smarter Quality Management

1/13


2/13

Smarter Quality Management: The Fast Track to Competitive Advantage

1

PA

GE

Organizations that create and deliver

softwarewhether fortheir own IT

operations, for the packaged applications

market,or as the core of their nal product,

as in the systems spacemust grapple not

only with todays tough economic climate,

but also with increased complexity in their

processes and supply chains. Many factors

serve to complicate software delivery,

but competition lies at the heart of this

complexity.

Here are a few examples. In the products

arena, customers demand more from thesoftware components designed for thelatest

hardware, often with requirements that

change rapidly, even as software projects

are underway. Keeping track of changes

while meeting aggressive (and unaltered!)

deadlines is difcult, ifnot impossible. In

the IT space, more businesses are focused

ontheir operational software for capturing

and providing value to their customers andlines of businesses. E-commerce websites

compete to improve customer relations

and simplify online business; businesses

that create highly optimized supply chains

supporting a fast, efcient ecosystem of

partners quickly rise in the market place.

What does this competitive environment

mean for businesses seeking to deliver

high-quality products and services?

Certainly, effective quality management

creates opportunities to deliver key business

benets, such as improved market share,

higher customer satisfaction, and increased

brand equity. But top quality in the completed

product cannot serve as the single guiding

principle by which products are produced

and delivered. Time to market is also key;

costs and risk factors must also be part ofthe balancing act. Get these things wrong,

and you may face unsustainable costs,

missed windows of opportunity, unhappy

customers, even a massive recall or the

complete failure of a system at a critical

moment. Get these things right, and you

can achieve a positive operational return

on investment from efciencies gained in

development activities.

One of the biggest challenges related to

quality management is how to invest

intelligently to minimize risk, given economic

constraints. However, guring out a) how to

relate quality to business outcome and b)

what constitutes the right level of quality for

individual products is not always clear.

This paper introduces a practical approach

to quality management (QM) that helps

reduce time to market without sacricing

quality in the outcome. The underlying

concepts presented here will be familiar

to software project managers, especially

those with QM experience, but I will explain

some fundamentals as we go along toensure all readers seeking these benets

can understand the essential processes

involved.

The nature ofsoftware

developmentHeres one way to understand the soft in

software: it is relatively easy to change. But for

software designed for the commercial space,

where the competitive pressures described

above govern a software projects success

or failure, the softfeature happens to be

one of its riskiest attributes. Thats because

software projects are seldom designed andmanufactured as in traditional engineering

projectsa bridge, for example. While a

bridge is engineered through traditional

planning and architecture based on the laws


3/13


2

PA

GE

of physics, then produced according to

an organized plan with a division of labor,

software is, at its essence, simply information.

Its development typically resembles a more

creative process than one bound by the

laws of nature. Walker Royce, IBM Software

Group/Rationals chief software economist,

compares software production to movie

production: a collaboration involving a team

of craftsmen and emerging from the naturally

creative process of artistic yet technical

people.

Over the past two decades, this unique

feature of software has been understoodand embraced by iterative development

practitioners, who now tend to develop

software in stages called iterations. Each

iteration delivers a working, functional

version of the software under development,

so it can be reviewed, tested, and

vetted by stakeholders and other teams

seeking adherence to the original project

requirements. This allows project managersto make smaller, incremental course

corrections during the project life cycle

thus ensuring the nal deliverable is close

to expectationsas opposed to having

separate teams work according to a plan,

assemble various components near project

end, and discover major failures due to

integration or deployment complexities.

For testing teams, the iterative development

process integrates quality management

across the states of the project work ow,

as opposed to relegating test activity to

the end of the project. I will describe the

role of iterative development-based quality

management more fully in the next section.

Qualitymanagement in

the softwaredevelopment life

cycle

What is the role of the testing, or QM, team

during the iterative life cycle? What do they

test for, and how do they know what is

supposed to change from one iteration to

the next?

As noted above, traditional software testing

only occurs late in the life cycle, after

multiple coding teams have spent much

time and effort to deliver their components

toward the complete project. Because these

traditionally managed projects proceed

according to strictly described requirement

sets, and various component teams focuson their portions alone, it is up to the

testers to discover the discontinuities and

malfunctions as these components are

assembledthen its the testers who must

deliver the bad news that much rework has

to be done inorder to get the project back

on track.

Iterative software development techniques

improve on that scenario by introducing

test teams to the process much sooner. A

relatively modest, rst iteration may only

address 15 percentof the full set of project

requirements, but as a functional module

of working code, the completed iteration

can be demonstrated, and tested. So any

defects discovered by test teams at this

early stage have a proportionally small

impact on the larger development team, whomake the xes, then proceed to the next

iteration where more of the requirements

can be incorporated into the working version

(iteration) of the software.


4/13


3

PA

GE

The number of iterations required by any

software project depends on many factors, of

course, such as the complexities of the teams

supply chain, the complexity of the software

underdevelopment, the physical locationof team members (are they geographically

distributed, perhaps internationally? or are

they co-located under a single roof?), and

the competitive demands that determine

optimum time to market. During any software

delivery process, When do we release? is

a key question withno simple answer. You

must consider project-specic variables,

such as the cost of delays, the opportunityvalue of early delivery, marketplace quality

expectations, and the costs associated with

defects. Ultimately, the delivery strategy

will be based on the actual or perceived

importance of each variable.

The optimal time

to release

The optimal time to release is when the total

risk exposure isminimal, typically around

the time where the risk associated with

competitive threats starts to outweigh the

risk reduction associated with further quality

improvements, as illustrated in Figure 1.

The best time to release varies widely

based on your software delivery strategy

and your target market. Software delivery

for both IT (internal business systems) and

smart products (products using embeddedsoftware, including system-of-systems

design) is typically dominated by one of two

motivators, depending on the organizations

target market: time to market (schedule

driven), or quality impact (quality driven).

Schedule-driven delivery implies deliver

on time, regardless of other factors and

is often used in industries where Time tomarket is king. Consumer electronics is

one good example, as well as automotive,

segments of the medical industry, and other

markets, where product teams try to gain a

rst mover advantage over their competitors,

(almost) regardless of the risk associated

with inadequate quality. It should also be

noted that schedule-driven delivery is not

limited to the systems space (i.e., the many

embedded software devices industries).

Many IT development teams use schedule-

driven delivery when trying to enhance their

end user experience and increase market

share, taking away from their competitors,

often risking quality in the process.

Figure 2 represents the risks associated

with schedule-driven software delivery. The

green line represents the risks associatedwith the delivery of your product being

reduced over time. The red lines represent

the risk of competitors stealing your market

away increasing over time, as well as risks

associated with opportunity costs.

The intersection is the point in time where

the sum of both lines, i.e. the total risk, is the

lowest. As seen in Figure 2, this point movesto the left as the environment you are in is

more competitive in nature. (Notice that the

intersection point is moving up as well.)

High opportunity cost;

Strong competitionLowestOverallRiskExposure

Manycritical

defects

Low opportunity cost;

Weak competition

Few

minor

defects

Time

RiskExposure

Quality risk ( = Probability of defects x loss due to defects)

Competition risk ( = Probability of competitors x size of loss to competition)

Total risk ( = Sum of all risks)

Figure 1: Minimal risk exposure is when oppor tunity cost and competitivethreats outweigh risk reduction related to quality improvements.


5/13


4

PA

GE

Quality issues are often magnied in a

schedule-driven life cycle, given that

software contractors frequently get paid

on a time and materials basis, regardless

of the quality of software they deliver. In

many cases, you may even end up paying

extra for the delivery team to x their own

defects, so the potential costs of defects

to the end user can add up quickly. And

heres an interesting statistic: According to

the Carnegie Mellon Software Engineering

Institute, Data indicate that 60 - 80 percent

of the cost of software development is in

rework.

Quality-driven deliverycan also be costly

but for different reasons. As shown in Figure3, the more critical any defects might be

regarding quality, the longer it takes to get

to an optimum release point.

The release timing for this approach is

governed by achieving the right quality,

moving the optimal time to release further to

the rightbut how do you dene that? Zero

defects is practically impossible to achieve,given that there is no way to determine how

many defects still exist in a piece of code or

the probability of detecting those defects in

use. A target based ondefects xed might

be more realisticbut its still impossibleto

know the number of remaining defects in the

product.

Risk-driven delivery implies delivering

your software when the risk is minimal.

But in practice, we always need to release

earlyearlier than we can. Which typically

implies increasing the risk, right? At least

this is a commonly held view, but is it always

the case?

Within the risk-driven model, the optimal

release time is when risks are sufciently

reduced (not completely eliminated) and

time to market has not been wasted. Inother words, some time is needed to reduce

the most signicant risks, but the company

cannot afford to address every known

risk because the opportunity to beat the

competition is eeting.

So the question is, how can we get to this

point sooner? How do we compress the

release date from the optimal intersection(shown as a blue circle in Figure 4) to a point

earlier in time?

Time

Risk

Exposure

Time to Market is king!Example: Consumer Electronics

High opportunity cost;Strong competition

Figure 2: The blue dots show possible release points, with points of minimalrisk moving forward as competition intensifies (red lines).

Time

Risk

Exposure

Quality is king!Example: Safety Critical applications

Criticality ofDefects

Figure 3: The more critical the implications of defects are, the more time it

takes to get to the lowest risk point where release is possible.


6/13


5

PA

GE

We cannot simply cut the time requirement,

because as we move left on the green line,

the risk goes higher. But what if we could

compress the curve described by the green

linepush it down, so to speak? Then we

could not only deliver sooner but lower the

overall risk as well. The intersection point will

move down and to the left. This improved

scenario is shown in Figure 4.

Risk-driven delivery offers a practical

improvement over these two extremes (i.e.

schedule driven vs. quality driven) because

it more cost-effectively balances quality

versus time-to-market considerations. A

risk-driven strategy is a renement of a

quality-driven approach that optimizes risk

exposure against development cost and

time.

For the remainder of this discussion, we

will assume that the software delivery life

cycle is based on a risk-driven approach.

We will explore how to bend the green

curve shown in Figure 4 downward and to

the left, for reduced time to market without

compromising the risk prole.

Understandingquality

management: Its

more than simply

testing

If a faster reduction in risk is the goal,

how do you achieve it? The answer is not

through testing, which is focused simply

on discovering defects. In traditional testingpractices, testing is considered a late stage

activity, squeezed between an often-late

development hand off date and an immovable

ship date. Not only does this practice fail to

yield the benets of incremental, iterative

development techniques explained earlier;

it also minimizes, or at best reduces, the

amount of time spent on quality assurance,

and makes xes all the more difcult unlessyoure willing to compromise the release

date.

As noted earlier, iterative development

techniques greatly improve this situation by

having functional units tested incrementally,

in stages, throughout the life cycle, rather

than leaving the testing phase until project

completion.

And quality management takes this

improvement a step further2.. Quality

management, which is the implementation

of best practices to proactively reduce risk

throughout the whole life cycle, is a risk

reduction mechanism in its own right. By

choosing quality management practices

with the potential to deliver a positive ROI

within a relative short amount of time, you

can justify risk reduction measures from not

only a quality stand point but also a nancial

standpoint.

Time

OptimalTime to Release

Time toMarket

Figure 4: To deliver early, at an improved quality, reduce your risk at an earlierpoint in the li fe cycle.


7/13


8/13


7

PA

GE

coverage, one that maps results to testing

activities. The essential principle is this:

prioritize testing according to risk. As

changes occur to working code during the

life cycle, test teams may choose to test

for the most critical requirements rst, and

maybe later test for all critical requirements

remaining (for example, in another iteration).

In other instances, they may choose to run

only the subset of test cases that provide

them with the highest coverage. This

approach is illustrated in Figure 5.

Although you cannot completely remove

risk from a development program, you

can measure it and manage it by taking

the appropriate mitigation actions. This

approach helps ensurethat you are optimally

using your nite testing resources (your

people) to reduce risk as rapidly as possible

in the development cycle. By focusing testing

effort on high-value test cases, from either

the point of view of coverage or contribution

to business value, you essentially prioritize

according to riskand you worry less about

test cases that pertain to noncritical issues.

But does creating the traceability links

require an expensive investment? No, andhere is the basic math. Consider a hypo-thetical medium-size project with some5,000 requirements and 10,000 test cases.Assuming it takes 20 minutes to locate and

link the appropriate test artifacts for each

requirement, it would take approximately10 person-months to create the traceabilitybetween requirements and test cases. You

could potentially reduce this time to oneto two minutes per requirementand atotal of just 10 to 20 person-daysusinga dedicated quality management solutionwith support for capturing traceability linksbetween requirements and test cases. At anominal rate of US$50 per hour, this singlechange corresponds to a potential saving of

around US$75,000. And needless to say, inreal life, the numbers are much bigger.

There are other best practices that contributeto improved quality at a reduced cost. Weare not going to cover them all in this paper,

but here are two for you to consider.

Improved collaboration between theQA team and other stakeholders: From

talking to customers, we learned thaton average, a tester spends only about60 percent of the time performing actual

testing, test planning, or test reporting.The other 40 percent is related toactivities that are collaborative in nature,such as clearing up the requirementswith domain experts or businessanalysts, or exchanging emailsand phonecalls with members of the

development team. This gets worse indistributed organizations. If you couldtrack and manage the collaboration,

it will not only reduce your riskassociated with lack of communicationsand misunderstandings, but also reducethe time for collaborative tasks by 20 -50 percent.

Automated reporting: Creating a report,especially one that goes to high level

management, requires data collectionfrom many sources, sometimes from

teams that are in different time zones,and then formatting this data

appropriately. If you could automate thisactivity, your team will probably useit more often and take the appropriate

Most critical Requirements

All critical Requirements

Low contribution

High Requirements coverageNo. of Requirements

Test Suites

Figure 5:After a change, re-executing all the tests is safe, but expensive andoften unrealistic. By focusing instead on test suites that are the most relevant

to the iteration that is being tested, test teams make more efficient use of their

time and reduce redundancies along with high testing costs.


9/13


8

PA

GE

decisions in real time,thus reducingyour risk. How much would this saveyou?

These are some of the quality management

best practices, each of which contributes torisk reduction and therefore increased qualityand reduced cost. Now, lets consider theoverall impact of quality management on thedefects density and the cost of xing them.

The overall

business impact ofquality management

Quality management best practices centeron quality synchronization points acrossthe whole development life cycle. We havediscussed the benets of traceability torequirements, and we have briey notedcollaboration between stakeholders and the

QA team, as well as automated reporting.Other best practices include: allowing qualityprofessionals to contribute to the team effortfrom the very beginning of a project; theintegration of practitioners doing the testing

as part of quality management; and the useof consolidated quality dashboards.

Together, these quality management bestpractices can benetthe overall business in

measurable ways. Using CMMI

3

as aproxyfor the maturity of the development process,Figure 6 shows that the overall businessimpact of quality management is quitecompelling.

Lets assume an organization at CMMI level 2,with 1000 defectsdetected during functionaltesting. Figure 6 shows that on average,without QM practices, about 30 percent ofthe defectsare being detected in functional

testing (the left, blue bar), and therefore thetotal number of defects is 3300. However, byapplying QM practices, the defect detectionrate increases to 58 percent (the right, greenbar), therefore detecting 1914 (58 percent of

3300), or 914 more defects.

As xing defects during User AcceptanceTesting (UAT) is aboutseven times moreexpensive than during unit/integration test,and assuming a x cost of US$120 perdefect during unit/integration test, xing 914defects in UAT is already increasing the costby over half a million dollars!

And this does not even take into accountthe reduction in the number of defects thatresult from applying QM practices in therst place, which makes the savings evenmore signicant. This also does not takeinto account less tangible savings, suchas increased quality, customer retention,and other implications of quality as adifferentiating asset.

As most software development teams are

around CMMI levels 2 or 3, the benets andthe savings described above apply to most

of the industry. But as development teamsbecome more mature, apply QM practices,and move up to CMMI levels 4 and 5, the focusshifts into less obviousbut for some, evenmore importantbenets, such as reductionin the number of defects that are introducedin the rst place, measured improvementsaround planning and execution of quality

related activities, customer retention, andleveraging quality as a differentiating asset.

Impact of Quality Management on Process Efficiency

15%

30%

60%

75%

85%

32%

58%

76%

85% 87%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5

CMMI Levels W/O QM W QM

20% 40% 40% 40% 10%QM Impact:

Figure 6: Graphing percentage of defects detected (Y axis) against an

organizations software development maturity level (X axis).


10/13


9

PA

GE

Better quality +lower cost

= improved

competitiveness

In this paper, I have described several

improvements to methods used by softwareteams in the design, testing, and deployment

of software for systems or IT. Teams mayuse these quality management methods to

deploy that software more quickly, whilemitigating quality-related risks throughoutthe life cycle. The multidisciplined practiceof quality management is breaking thefunctional and organizational silos thatare so common in todays companies. Itencourages an analytical process thatsclosely integrated with the development life

cycle.

Analyzing the market and best practices

shows that business outcomes can beoptimized, and that smart improvements

within the realm of proven best practices for

requirements management and traceability,collaborative test planning and automatedreportinga combination of disciplines

that denes quality managementcan helpaddress the need for increased innovation

with more competitive products and servicesto your customers.

The IBM Rational organization is readyto demonstrate these techniques to you.With straightforward adjustments to yourinvestments, deployment practices, and

tooling, we can help you realize thesebenets within a time frame that best suitsyour businesss needs.

We look forward to working with you!

For more

information

To learn more about the IBM Rational qualitymanagement offerings, please contact

your IBM marketing representative or IBMBusiness Partner, or visit the followingwebsite:

ibm.com/software/awdtools/rqm/

Additionally, nancing solutions from IBMGlobal Financing can enable effective cashmanagement, protection from technology

obsolescence, improved total cost of

ownership and return on investment. Also,our Global Asset Recovery Services helpaddress environmental concerns with new,

more energy-efcient solutions. For moreinformation on IBM Global Financing, visit:ibm.com/fnancing

About the AuthorMoshe Cohen is the Market Manager for IBMRational quality management offerings. In hiscurrent role, he works closely with customers,including managers and practitioners, todrive IBM Rational quality managementofferings in both the IT and embedded

systems spaces. Prior to this, he was withTelelogic, dening and driving its Model

Driven Testing solutions. He has extensivehands-on experience in the specication,development, and testing of C3I medical and

telecom applications, including technologyadoptions and driving process improvement

programs. He received his EE and M.Sc inmathematics and computer sciences, bothwith honors, from Beer-Sheva University inIsrael.
http://www-01.ibm.com/software/rational/products/rqm/http://www-03.ibm.com/financing/ww/http://www-03.ibm.com/financing/ww/http://www-01.ibm.com/software/rational/products/rqm/


11/13


10

PA

GE

Copyright IBM Corporation 2011 IBM Corporation

Software Group Route 100 Somers, NY 10589 U.S.A.

Produced in the United States of America June 2011

All Rights Reserved

IBM, the IBM logo, Rational and ibm.com are

trademarks or registered trademarks of the

International Business Machines Corporation in

the United States, other countries, or both. If these

and other IBM trademarked terms are marked

on their rst occurrence in this information with atrademark symbol ( or ), these symbols indicate

U.S. registered or common law trademarks owned by

IBM at the time this information was published. Such

trademarks may also be registered or common law

trademarks in other countries. A current list of IBM

trademarks is available on the web at Copyright and

trademark informationatibm.com/legal/copytrade.

shtml

Other company, product, or service names may be

trademarks or servicemarks of others.

The information contained in this document is

provided for informational purposes only and

provided as is without warranty of any kind,

express or implied. In addition, this information is

based on IBMs current product plans and strategy,

which are subject to change by IBM without notice.

Without limiting the foregoing, all statements

regarding IBM future direction or intent are subject to

change or withdrawal without notice and represent

goals and objectives only. Nothing contained in this

documentation is intended to, nor shall have the

effect of, creating any warranties or representations

from IBM (or its suppliers or licensors), or altering

the terms and conditions of the applicable license

agreement governing the use of IBM software.

1See the Carnegie Mellon Software Engineering

Institutes CEO andfounders message at http://

www.sei.cmu.edu/about/message/

2For a more complete discussion of quality

management practices, downloadthe paper Value-

driven quality management for complex systems:

Sixstrategies for reducing cost and risk at http://

www14.software.ibm.com/webapp/iwm/web/

s ignup.do?source=swg-rt l -spsm-wp&S_

PKG=wp_RQM_VALUEDRVN_071510

3Capability Maturity Model Integration. CMMI is a

staged approach to process improvement that denes

incremental levels of maturity in software engineering

organizations. For more information see the Software

Engineering Institute (Carnegie Melon University)

website athttp://www.sei.cmu.edu/cmmi/

Please Recycle
http://www.ibm.com/legal/us/en/copytrade.shtmlhttp://www.ibm.com/legal/us/en/copytrade.shtmlhttp://www.ibm.com/legal/us/en/copytrade.shtmlhttp://www.sei.cmu.edu/about/message/http://www.sei.cmu.edu/about/message/http://www.sei.cmu.edu/about/message/https://www14.software.ibm.com/webapp/iwm/web/signup.do?source=swg-rtl-spsm-wp&S_PKG=wp_RQM_VALUEDRVN_071510https://www14.software.ibm.com/webapp/iwm/web/signup.do?source=swg-rtl-spsm-wp&S_PKG=wp_RQM_VALUEDRVN_071510https://www14.software.ibm.com/webapp/iwm/web/signup.do?source=swg-rtl-spsm-wp&S_PKG=wp_RQM_VALUEDRVN_071510http://www.sei.cmu.edu/cmmi/http://www.sei.cmu.edu/cmmi/http://www.sei.cmu.edu/cmmi/http://www.sei.cmu.edu/cmmi/https://www14.software.ibm.com/webapp/iwm/web/signup.do?source=swg-rtl-spsm-wp&S_PKG=wp_RQM_VALUEDRVN_071510http://www.sei.cmu.edu/about/message/http://www.ibm.com/legal/us/en/copytrade.shtml


12/13

05 08 November 2012 | RAI Amsterdam

TheEuroSTARConerence

attracts testers rom

around theglobewhocome tolearn, network&

discuss all things testing.

Attendees experience

intensive tutorials, track

sessions & inspirational

keynote speakers, & can

visitEuropes largest

Sotware Testing

Exhibition. Return to

theofce inspired,

motivated, andull onew ideas!

www.eurostarconerences.com


13/13

w w w . e u r o s t a r c o n f e r e n c e s . c o m

Join the EuroSTAR CommunityAccess the latest testing news! Our Community strives to provide testprofessionals with resources that prove benecial to their day-to-day roles.These resources include catalogues of free on-demand webinars, ebooks,

videos, presentations from past conferences and much more...

Follow us on Twitter @esconfs

Remember to use our hash tag #esconfs when tweetingabout EuroSTAR 2012!

Become a fan of EuroSTAR on Facebook

Join our LinkedIn Group

Contribute to the EuroSTAR Blog

Check out our free Webinar Archive

Download our latest ebooks

The EuroSTAR BlogWritten by Testers for Testers
http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/

Date post:	06-Apr-2018
Category:	Documents
Upload:	andries-poepjes
View:	228 times
Download:	0 times

Ibm Smarter Quality Management

Documents