Application Lifecycle Management and process monitoring ... · • CMMI-DEV and ISO certifications,...

http://itconfidence2013.wordpress.com

Application Lifecycle Management and process

monitoring through an integrated and low-cost

solution, mainly based on Open Source Software Products

1°International Conference on IT Data collection, Analysis and Benchmarking

Rio de Janeiro (Brazil) - October 3, 2013

N. Bertazzo, D. Gagliardi, N. Bertazzo, D. Gagliardi, S. Oltolina,S. Oltolina, G. Ruffatti G. Ruffatti

Engineering GroupEngineering GroupInsert here a picture

2IT Confidence 2013 – October 3, 2013 http://itconfidence2013.wordpress.com

ALM and…

G1. Measuring company performances through an effective SPI program G2. Adopting a multi-dimensional performance model deployed mostly with 3D instancesG3. Implementing an integrated and low cost – OSS based – solution for the measurement and governance of software product & process quality

Presentation Goals


Which is users’ & customers’ level of satisfaction?

How productive is myorganization?

Is there REALLY a way to measure performance?

Which is thequality level of my product?

How can I improve the development process?

How can I compare different labs?

Top Top ManagerManager

Is my project on track?

How can I improveperformance?

Project Project ManagerManager

Quality Quality ManagerManager

Which are corporate audits results?

Introduction The problem: Managers’ needs


Engineering At a glance

www.eng.it

http://www.eng.it/

http://www.eng.it/

http://www.eng-it.it/spimq


Technical, Innovation & Research Division Engineering’s Software Labs (ESL)

PRODUCTION

ESL3: Application Management

ESL1-2: Project development

RFPs technical support

MANAGED OPERATIONSInfrastructures & System Services

Architectural design Research&

Development

CompetenceCenters

Resource management

Business Units (BUs) for different market sectors

AccountManagers

Sales Managers

ServiceDesk

Business Analysis

ProjectManagersBusiness

CompetenceCenter

Worldwide Customers

Background Engineering production organization Managers’ needs


• Continuous Quality Improvement in Engineering's projects• Unified Infrastructure supporting quality processes granting flexibility and

adaptability• CMMI-DEV and ISO certifications, as independent method to validate the

compliance of processes and infrastructures with quality standards• Set-up of Engineering Software Labs (ESLs) to enhance and measure

productivity and improve quality practices

Background Compliance of SPI to quality systems

…


Interaction & Information sources (quant. & qual. data)


• Three dimensions of analysis:

1. Economical (EE)2. Social (SS)3. Technical (TT)

QEST model: 3 dimensions

•Performance values for each dimension allow to identify process areas that need improvements


• Spago4Q, the open source platform to measure, analyze and monitor quality of products, processes and services

http://www.spago4q.org

The model• QEST nD model, a conceptual framework for measuring process

performance based on multiple analysis dimensions

http://www.semq.eu/leng/modtechqlm.htm

The tool

Productivity Intelligence

http://www.spago4q.org/

http://www.semq.eu/leng/modtechqlm.htm



The model (1): QEST

Method: Performance is expressed as the combination of the specific ratios selected for each of the 3 dimensions of the quantitative assessment (Productivity - PR) and the perceived product quality level of the qualitative assessment (Quality - Q)

Performance = PR + Q

Model: QEST (Quality factor + Economic, Social & Technical dimensions) is a “structured shell” to be filled according to management objectives in relation to a specific project

Such a model has the ability to handle independent sets of dimensions without predefined ratios and weights - referred to as an open model


The model (2): geometrical indicators

It is possible to measure performance considering at least 3 distinct geometrical concepts:• Distance between the tetrahedron base center of gravity and the center of the plane section along the tetrahedron height – the greater the distance from 0, the higher the performance level;• Area of the sloped plane section – the smaller the area, the higher the performance level;• Volume of the lowest part of the truncated tetrahedron – the greater the volume, the higher the performance level.

• Target: measuring project performance (p) using 3 distinct viewpoints• Input Data: list of weighted ratios for each dimension and quality questionnaires• Output Data: an integrated normalized value of performance


• Integrated quantitative and qualitative evaluation from 3 concurrent organisational viewpoints• A 3D geometrical representation at a single project phase (usually after the project is completed) •Use of de facto and de jure standards (e.g. ISO/IEC 9126 for the Quality Factor now ISO/IEC 25010:2011)• Performance Measurement Model to use for consolidating Balanced Scorecard (BSC) measurement outcomes• Extension of the original 3D model to n possible dimensions-perspectives QEST nD through the simplexsimplex as the mechanism to solve the problem from the 4th dimension on

The model (3): key features


The tool: Spago4Q



• QEST model is fully supported by Spago4Q

• The procedure is coherent with the PMAI (Plan-Measure-Assess-Improve) cycle: PLAN, defining a set of metrics, based on the GQM approach, and

possible dimensions of analysis (perspectives) characterizing the analysis

MEASURE, including the collection of data, and the computation of metric values and global performance value

ASSESS, presenting results through dashboards and reports IMPROVE, analyzing in detail each value that is less than the

expected thresholds, in order to find possible problems or bottlenecks from a process-based viewpoint

The integrated environment: QEST & Spago4Q


The ESL model selected goals for each analysis dimension:

1. Economical (EE)E.G1 Reduce the effort of corrective maintenance (corrective + preventive, ISO/IEC14764:2006)E.G2 Improve ESL resource/assets allocation E.G3 Reduce effort due to hardware system unavailability (‘downtime’)E.G4 Reduce rework (Analysis/Design SLC phases)E.G5 Improve productivity (note: different ‘sizing’ units)

2. Social (S) S.G1 Reduce the number of non-conformity issues (QA inspection)S.G2 Improve artifacts reuse (functional reuse)S.G3 Evaluate training skills for organizational resourcesS.G4 Improve customer satisfaction (e.g. Customers/Prospects, Business Units, Developers) S.G5 Improve knowledge sharing (“social 2.0”, communities)

3.Technical (TT)T.G1 Reduce the resolution time for defects and technical issuesT.G2 Reduce the number of pre-delivery defects (as in ODC analysis) T.G3 Improve delivery time for deliverablesT.G4 Improve code qualityT.G5 Improve the testing process (e.g. coverages, # req’s, # tests, ...)

3D analysis: main goals


Metric ID Metric Desc Formula SourceE.M1.1 Incidence of corrective maintenance effort Corrective Maintenance Effort/Development Effort ALM & prj registry

E.M2.1 Allocation of ESL resources Nr. of Res (hours) allocated on prj/Tot of Res (hours) ALM & Corp. Systems

E.M3.1 Hardware System Availability Percentage System Availability System Monitoring

E.M4.1 Incidence of rework Rework Effort / Development Effort ALM & prj registry

E.M5.1 Development capability FP/Effort ALM & prj registry

S.M1.1 n. Of Non Conformity issue % of NC for project ALM & QA Registry

S.M2.1 Incidence of artifact reuse Nr downloads/tot nr of artifacts stored Component repo

S.M3.1 Skill improvement % new (or modify) skills for resource Skill management tool

S.M4.1 Customer Satisfaction Results of survey Survey tool

S.M5.1 Knowledge sharing improvement % of interaction with collab. tools Collaboration tools

T.M1.1 Incidence of defects % nr. of defects (errors + defects) for project ALM

T.M2.1 Defects Mean Resolution Time Tot. resolution time/Tot. defects ALM

T.M3.1 Incidence of delayed deliverables % nr. delayed deliv. / Tot. deliverables ALM

T.M4.1 Code Quality Results of automatic static test Code analysis tool

T.M5.1 Testing process improvement Test coverage ALM

3D analysis: metrics


ESL Chief Manager

ESL Lab Manager

Level 1

Level 2

Level 3

Project Manager (PM)

3D view: drill-down through the organizational levels

PRJ n

ESL

ESL 1 ESL 2 ESL 3

PRJ 1

PRJ n

PRJ 1PRJ n

PRJ 1

Project developmentProject development Application maintenanceApplication maintenanceProject developmentProject development

Engineering’s Software LabsEngineering’s Software Labs

Top Manager (TM)


3D view: Level 1 - TM dashboard – sample #1

Unified view on Engineering Software Labs

Global performance indicator

Performance comparison (time, labs)


3D view: Level 1 - TM dashboard – sample #2


Metrics

Reqs & Bugs

RisksTasks & Issues

Docs

Detailed view

Tracking and trends

A single dimension view: Level 3 - PM dashboard


A single dimension view: Level 3 - PM ALM tool


A single dimension view: Levels 1/2 - QA audits


Users & customersfeedbacks arenow integrated

with corporate data

Finally I know how productive myorganization is!

Finally we can REALLY measure performance!

I can monitor thequality level of my product

The development processis under control and

I can improve it!Now I can compare Labs performance!

Top Top ManagerManager

I know if my project is on track & I can

identify issues

Conclusions

Productivity Intelligence enables performance

improvement

Project Project ManagerManager

Quality Quality ManagerManager

Through audit dashboards, corporate QA

is under control


• Improve reports and KPIs Introduce/improve reports for new/modified information needs Dynamical update for thresholds

• Integrate tools to collect soft factors measures and indicators from two categories of ESL customers using on-line surveys: “external customer”: feedback on the perceived product quality “internal customer”: feedback on the actual process quality from

project managers of the various Business Units• Build new KPIs on data coming from the Infrastructure

Enhancing Project that collects issues and suggestions by the ALM users e.g. in the BSC ‘Learning & Growth’ perspective

Next steps


25

Buglione L., Misurare il Software. Quantità, qualità, standards e miglioramento di processo nell’Information & Communication Technology, FrancoAngeli, 3/ed, 2008, ISBN 978-88-464-9271-5

Buglione L. & Abran A., QEST nD: n-dimensional extension and generalisation of a Software Performance Measurement Model, International Journal of Advances in Engineering Software, Elsevier Science Publisher, Vol. 33, No. 1, January 2002, pp.1-7

Spago4Q website and resources: www.spago4q.org

Contacts & Info: [email protected]

References


mailto:[email protected]


Q && A

Muito obrigado pela vossa atençaoMuito obrigado pela vossa atençao!!Thanks for your attentionThanks for your attention!!

ALM and...


Our Contact Data

NicolaBertazzo

[email protected]

DanieleGagliardi

[email protected]

GabrieleRuffatti

[email protected]

Sergio Oltolina

[email protected]

ALM and...

http://www.eng.it/

http://www.eng.it/

http://www.eng.it/

http://www.eng.it/

Date post:	19-Mar-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Application Lifecycle Management and process monitoring ... · • CMMI-DEV and ISO certifications,...

Documents