Date post: | 24-May-2015 |
Category: |
Technology |
Upload: | konstantin-berger |
View: | 302 times |
Download: | 1 times |
Turning software risk measurement into business value
June 2013
Konstantin Berger – Regional Sales Consultant
Standardized Risk Measurement for IT Executives 101
CAST Confidential
Agenda
1. Who We Are
2. What We Do
3. Why We Do This
CAST Confidential
Who We Are: Driving software measurement in the industry
2
Key Influencers Recognize CAST
250 Global Leaders Rely on CAST
Institutions Engage CAST SIs Resell CAST SIs Use/Resell CAST
• 20+ years in SAM industry
• $100M+ of R&D investment
• Largest Benchmarking DB
CAST Confidential
Who We Are: How Customers Use Us
3
“We measure software quality at a structural level, in addition to the functional level through testing, to make the right tradeoffs
between delivery speed, business risk, and technical debt.”
Thaddeus Arroyo
CIO, AT&T
“Now we can show our business constituents the ROI from quality improvement.”
Peter de Boel
Head of Global Shipping IT, FedEx
“We got feedback from our clients noticing that our quality has gone up.”
Gene Baker
Director of Application Development, Wells Fargo
“CAST makes the relationship with the suppliers very transparent.” Lester Thomas
Head of Architecture, Vodafone
“I have to continue to deliver the same level of functionality and innovation, at the same quality, but with a significantly lower budget. That’s where CAST
helps us.”
Gil Hoffman
CIO, Maritz
“The architectural assessment of design consequences (on software performance, stability, adaptability, maintainability, and security vulnerabilities) is an area in which CAST excels and successfully
differentiates from static analyzers” Melinda Ballou
ALM Research Director, IDC
CAST Confidential
What We Do: Structural Quality vs. Functional Quality
Functional Quality (not us)
That which we can see
10 to 20% of app dev cost
Structural Quality (what we do)
Architecture Analysis
Standardized SW Characteristics
App Reliability
App Performance Efficiency
App Security
App Changeability
App Size (Function Points)
CAST Confidential
What We Do: A natural complement to testing
Design Develop
QA
Deploy Maintain
NEW
Functional Testing
Performance Testing
CAST: Structural
Testing
CAST Confidential
What We Do: Standardized Risk Management for Execs
CAST Quality Score Scale
Description Grade
Low Risk 4
Moderate Risk 3-4
High Risk 2-3
Very High Risk 1-2
Benchmark
Low
Risk
High
Risk
Health Factors Current Grade Last Delivery Since Baseline
Performance 2.38 0.5% 0.7%
Robustness 2.72 0.2% 0.4%
Security 3.14 0.1% 0.2%
Changeability 3.04 0.2% 0.3%
Transferability 2.77 0.2% 0.3%
Consortium for IT
Software Quality
r1 r2 r3 r4 r5 r6
S
R
P
Ongoing Risk Monitoring
CAST Confidential
Why We Do This: Not just quality for quality’s sake
UNIT LEVEL FLAWS
Downtime caused
by system-level flaws!
Of all
code
defects
Of total
repair
effort
92%
8%
52%
48% 90%
10%
SYSTEM LEVEL FLAWS
Software Risk Prevention:
Focus on critical violations
that matter
Focus resources on areas
of highest impact rather
than pursuing hygiene
“Tracking programming
practices at the Unit Level
alone may not translate into the
anticipated business impact,[…]most devastating
defects can only be detected at the System Level.”
- OMG
8 CAST Confidential
Co
nfi
de
nti
al
Why We Do This: Better Communication with Business
Data Storage
Application
Business Questions
•Why does it always take so long to
make small improvements?
•Why does every new release take
weeks to stabilize no matter how much
it was tested? Why can’t it just work?
•Why do we have to scrap timelines to
fix critical defects every time we get
close to project completion?
•Why are we late again?
IT’s Struggle to Answer
Because the environment is so complex, IT
does not have answers that business can
easily understand. This causes frustration
and trust issues.
9 CAST Confidential
Co
nfi
de
nti
al
Why We Do This: Structural risk requires a 3-tier approach A
rch
itec
ture
Co
mp
lia
nce
Intra-technology architecture
Intra-layer dependencies
Module complexity & cohesion
Design & structure
Inter-program invocation
Security Vulnerabilities
Module Level
Integration quality
Architectural compliance
Risk propagation
simulation
Application security
Resiliency checks
Transaction integrity
Function point & EFP
measurement
Effort estimation
Data access control
SDK versioning
Calibration across
technologies
System Level
Data Flow Transaction Risk
Code style & layout
Expression complexity
Code documentation
Class or program design
Basic coding standards
Program Level
Propagation Risk
Java
JSP
EJB
PL/SQL
ASP.NET
Oracle
SQL
Server
DB2
T/SQL
Hibernate
Spring
Struts .NET
C# VB
COBOL
C++
COBOL
Sybase IMS
Messaging
Java Web
Services
APIs
1
2
3
CAST Confidential 10
Why We Do This: Quality impact at a major services brand
Measured impact in a complex enhancement-heavy environment
304 222 196
385 401
231 198 242 279
167 112
258 274
149 140 245
188
61 56 78 97 62 57 81 75 87 40
260
181 167
225 265
220
151
195
232
154
150
295
385
228 163
223
186
148 84
121 136
96 38 11 10 8 11
0
100
200
300
400
500
600
700
R1
R1.1
R1.2
R2
R2.1
R3
R3.1
R4
R5
R6
R7
R7.1
R8
R9
R9.1
R9.2
R10
R10.1
R10.2
R10.3
R11
R11.1
R11.2
R11.3
R12
R13
R14E
Code No RC Non Code Projected Count
Str
uctu
ral
qu
ality
S
yste
m t
est
defe
cts
Trend line
Before CAST implementation
0
500
1000
1500
2000
2500
3000
3500
R1
R1.1
R1.2
R2
R2.1
R3
R3.1
R4
R5
R6
R7
R7.1
R8
R9
R9.1
R9.2
R10
R10.1
R10.2
R10.3
R11
R11.1
R11.2
R11.3
R12
R13
R14E
Order Management Inventory Management Billing Customer Service
New critical violations
CAST Analysis starting point
CLIENT STUDY OVER 24 MONTHS
$2.7 million payback after 12 months