The Metrology Journey towards an 8th Base Quantity for Software: How Far or How Close Are We?

Metrology Journey towards an

8th

Base Quantity for Software:

How Far or How Close

Are We?

Alain Abran École de Technologie Supérieure – Université du Québec (Canada)

[email protected]

1 © Copyrights Abran 2016

Alain Abran

20 years

20 years

Development

Maintenance

Process Improvement

ISO: 19761,

9216, 25000,

15939, 14143,

19759

© Copyrights Abran 2016

+ 35 PhD

Software is different!

• It is an intellectual product:

It cannot be measured like physical

objects!

• Let us revisit this issue….

© Copyrights Abran 2016 3

List of topics

1. Measurement Lessons from the Masters

2. Metrology: Concepts & Terminology issues

3. Software Measurements: Diversity & Immaturity

Key Advances in Functional Size Measurement

4. Software Measurement & Metrology: COSMIC – ISO 19761

5. The future…..?


Measurement

of Time

Evolution of Time Measurement:

Concepts & Tools


Time Museum in Besancon (France)

The public sun dial

Sun shadow

The communal local time


Communal Time : The “horloge”

XVIII Century


Mechanical

Personal Time: Watches

Quartz

Atomic


Measurement

of distance


Geneva Museum of Science &

Technology:

Lack of universally

accepted references

(1802)

Architect Kha Amenhotec II

Dynasty XVIII-XX

(1550-1070 B.C.)

Egyptian Measurement &

Tools


Egyptian Museum – Torino (Italy)

Architect Kha

Measurement

Tools

© Fondazione Museo Antichità Egizie di Tonino – used with permission


Golden étalon

Work étalon

Masters from

the Past

The meter - universal étalon: A product of the French Revolution!

• A consensual definition: A fraction of the Meridian at the Equator

• A practical étalon: it took 7 years to measure the Meridian at the Equator..!

And since then, the physical representation of the meter as an

étalon has changed a few times....


Key Lessons

from

the Masters

1. Evolutionary societal & consensual understanding of measurement concepts – Perfection is not expected first: the search for precision –

when necessary only!

2. Development of Measuring Instruments

3. Establishment of measurement ‘etalons’

4. Specialized measurement training & certification: – Land surveyors

– Accountants,

– Engineers,

– Testing labs,

– etc.


List of topics






5. The future…..?



Counting Measuring Quantitative models

of relationships

Quantifying?


Counting Measuring Quantitative models

of relationships

Quantification

& Metrology

Metrology

Key Metrology Concepts embedded within the International Vocabulary of Metrology – VIM (& GUM)

Widely adopted by Masters in measurements:

International Bureau of Weights & Measures,

International Electrotechnical Commission,

International Federation of Clinical Chemistry & Laboratory Medicine,

International Organization for Standardization (ISO),

International Union of Pure & Applied Chemistry,

International Union of Pure & Applied Physics,

International Organization of Legal Metrology,

International Laboratory Accreditation Cooperation

• + long list of organizations in Sciences & Engineering


Metrology

Metrology components (& articles for the law):

1. Definitions

2. National metrology

3. Traceability & uncertainties

4. Legal units of measurement

5. Transparency of metrological information.

6. Legal metrology

7. Application of the Law

8. Offences

9. Responsibilities & duties

10. Conformity assessment procedures


Criteria for a good design of measurement -

already defined in metrology:

1- components of a measurement system

2- quality criteria



Ref.: Sellami PhD thesis & Abran (2010)

Input quantity in a

measurement model

Output quantity in a

measurement model

Measurement

Function

Measurement Model

Measured

quantity

value

Measurement Procedure

Measurand

OperatorMeasurement

PrincipleMeasurement

MethodInfluence

Quantity

Measurement

Result

Table 3: Description of the measurement elements

Measurement

Measurement

result

Measurement

procedure

Measuring

device

Measurement

conditions

Measurement

error

Measured

quantity value

True quantity

value

Conventional

quantity value

Reference

measurement

procedure

Primary

reference

measurement

procedure

Measuring

system

Repeatability

condition of

measurement

Intermediate precision

condition of

measurement

Reproductibity

condition of

measurement

Systematic

measurement error

Measurement bias

Random

measurement error


Ref.: Sellami PhD thesis & Abran (2010)

Measurement

Precision

Measurement Uncertainty Calibration Metrological traceability

Measurement accuracy

Measurement trueness

Measurement

repeatability

Intermediate

measurement precision

Measurement

Reproducibility

Definitional uncertainty

Type A evaluation of measurement uncertainty

Type B evaluation of measurement uncertainty

Standard measurement uncertainty

Combined standard measurement uncertainty

Relative standard measurement uncertainty

Uncertainty budget

Target measurement uncertainty

Expanded measurement uncertainty

Coverage interval

Coverage probability

Coverage factor

Calibration

hierarchy

Verification

Validation

Correction

Metrological traceability chain

Metrological traceability to a

measurement unit

Metrological comparability of

measurement results

Metrological compatibility of

measurement results


Table 6:

Detailed quality criteria

for a measurement

result


Information Needs

Information Product

Interpretation

Indicator

Derived Measures Derived Measures

Base Measures Base Measures

Measurable

Concept

Entity

Measurement Method

Attribute

Measurement Method

Attribute

(Analysis) Model

Measurement Function

Measurement

Information

Model

in

ISO 15939


Information Needs Information

Product

Interpretati

on

Indicator

Derived

Measures

Derived

Measures

Base

Measures

Base

Measures

Measur

able

Concep

t

Entity

Measurement

Method

Attribute

Measurement

Method

Attribute

(Analysis)

Model

Measurement

Function



VIM

Or

IEEE 24765?


Combination of units:

speed= Miles/hour

VIM

or

IEEE 24765


IEEE 24765


≠

List of topics






5. The future…..?


Measurement

in

Software

Typical new ‘software metrics’: – A modified algorithm

– Additions of conditions to the algorithms

– As many ‘metrics’ as can be extracted automatically from files (codes, models,..)

• Typical empirical analyses: – No hypothesis to be tested!

– Random search: highest correlation with something else!

– Little verification of the relevance to the concept to be measured!


Measurement

& Industry

For industry: measurement has little to do with

maths!

• Measurement as a technology requires

considerable consensual knowledge on:

– the concepts to be measured,

– credible references for measurement

– expected measurement errors

– …….. expected costs


Software

Industry &

Measurement

Out of the +1,000 of ‘software metrics’ proposed in the

literature, which ones have reached some maturity

with respect to industry needs?

– How do you recognize maturity in measurement?


Software

Industry &

Measurement

Out of the +1,000 of ‘software metrics’ proposed in the

literature, which ones have reached some maturity

with respect to industry needs?

– How do you recognize maturity in measurement?

Measurement Maturity = Standardization

• Which of the +1,000 ‘software metrics’ are now

recognized as ISO standards?


Functional Size: an

exception in

software

measurement

Only Functional Size Measurement (FSM) methods have

been adopted as ISO standards!

What has been done differently?


Function Points:

Measurement

Infrastructure

User Groups built the essential metrology support infrastructure:

– Procedural Measurement Manuals

– Central technical authority: • Measurement Practice Committee

– Certification criteria & exams

– Guidelines for specific contexts

– Case studies for reference materials


37

Meta-standard

on Function

Points:

ISO 14143

14143-1

Definition of

concepts

14143-2

Conformity

Evaluation with

14143-1

14143-3 Verification

methods

14143-4

Reference Models

14143-5

Determination of

functional domains

14143-6 Selection

Guidelines

ISO

Function Points

Standards

1. ISO 20926 : IFPUG

2. ISO 24570 : NESMA

3. ISO 20968 : MRKII

4. ISO 29881 : FISMA

5. ISO 19761 : COSMIC

What is common/different in their designs?


1st

Generation

Weaknesses

Weights-like Function Points:

• A structure with weights to integrate multi

variables: An end number with a symbol (FP) but without a well

defined measurement unit!

• A number of structural weaknesses pointed out in

the literature over the past 35 years:

From nominal-ordinal-interval-ratio scale types

Partially discredited in academia for almost 20 years.

Automation in a dead end after +35 years!


List of topics



3. Software Measurements: Myths & Facts

Advances in Functional Size Measurement


5. The future…..?


2nd

Generation

criteria

1- Adoption of consensual concepts from metrology :

measurement method,

measurement procedure,

measurement unit, etc.

many of the ISO definitions explicitly quoted ‘as is’ in COSMIC

glossary.

2- Designed to entirely meet the ISO 14143-1 meta-requirements:

Measurement of FUR

No reference to technical & quality characteristics

No reference to effort

Designed to be application domain independent

3- Open access



What is different in all software?

What is similar in all software?

COSMIC Key

Concepts

4

3

Footer Sample 5/8/2016

Persistent

Storage

Software

being

measured

Boundary

Entries

Exits

Reads Writes

Functional Users • Hardware devices,

• Other software or

• Humans

COSMIC

Generic Model of

Software

Functional

processes

Functional User

Requirements

Functional

Processes

Data

Manipulation

Data

Movement

Sub-processes

Theory:

Functional User

Requirements

Functional

Processes

Data Movements

(account for

associated data

manipulation)

1

1

n

2 - n

In practice:

Triggering

Event

causes a

Boundary that is

moved into

a FP by the

FP’s

Triggering

Entry

Functional

Process

(FP)

Functional

User to

genera

te a

Data

Group

COSMIC

Measurement

Process

Example of

Software Layers

46 5/8/2016

Keyboard VDU

Screen Printer

Hard Disk

Drive

Central

Processor

Keyboard

Driver

Screen

Driver

Print

Driver

Disk

Driver

Operating System Layer

Application Layer App 1 App 2 App ‘n’

Middleware Layer (Utilities, etc.)

Database Management

System Layer DBMS 1 DBMS 2

Software

Layers

2nd

Generation

COSMIC

Correction of all known structural weaknesses:

No weight & no invalid mathematical operations

Adoption of a clear & unique ‘measurement unit’:

– ‘a data movement of a single data group’ to which a size unit of

1 is assigned

• together with the 1CFP as its measurement symbol.

– 1 COSMIC CFP: represents a single measurable concept,

size of a FUR, making it a true ‘base quantity’.


Guideline describing a

range of Approximate

Sizing methods

Guideline on Assuring

the accuracy of

COSMIC measurements

Guideline describing

Measurement Strategy

Patterns – to ensure

comparability

The COSMIC Functional Size Measurement Method

Version 4.0.1

GGuuiiddeelliinnee ffoorr EEaarrllyy oorr RRaappiidd

CCOOSSMMIICC FFuunnccttiioonnaall SSiizzee

MMeeaassuurreemmeenntt bbyy uussiinngg aapppprrooxxiimmaattiioonn aapppprrooaacchheess

July 2015

The COSMIC Functional Size Measurement Method

Version 3.0.1

GGuuiiddeelliinnee ffoorr aassssuurriinngg tthhee aaccccuurraaccyy ooff

mmeeaassuurreemmeennttss

VERSION 0.93

February 2011

COSMIC

• Business applications

• Real-time software

• Data Warehouse software

• SOA software

• Mobile apps (in devt.)

• Agile Developments

www.cosmic-sizing.org

TThhee CCOOSSMMIICC FFuunnccttiioonnaall SSiizzee MMeeaassuurreemmeenntt MMeetthhoodd

VVeerrssiioonn 44..00..11

GGuuiiddeelliinnee ffoorr SSiizziinngg

BBuussiinneessss AApppplliiccaattiioonn SSooffttwwaarree

VERSION 1.3a

Febuary 2016

COSMIC:

Multi-domains


Physical Reality ………& ………… Models

COSMIC

COSMIC in

Industry

COSMIC in the Automotive

embedded software field

By: Sophie Stern

Renault

51

© Copyrights Abran 2016


GOAL/OBJECTIVE DRIVER INDICATOR COMMENTS / EFFECTS FINANCIAL (F)

Asset Management Existing asset utilisation

Total Assets (FSAV) / # employees ($)

FSAV – FSunits Asset Value

PS – Portfolio Size

Revenue & Profitability Revenue Growth

Revenues / FSAV (%)

Revenues from new customers / Total Revenues (%)

New customers acquired using FSM as a contractual condition for measuring the project – Derived (Improve project governance)

Profitability Profits / FSAV (%)

Financial Management Organisational Investments

Investments in IT

Project Investments

PCFS – Project Cost per FSunit

ECFS – Enterprise Cost per FSunit

AMCFS – Application Maintenance Cost per FSunit

Table 1 – FSM-based measures: Financial Perspective

GOAL/OBJECTIVE DRIVER INDICATOR COMMENTS / EFFECTS CUSTOMER (C)

Customer partnership and involvement

Collaboration % projects using integrated teams

SR – Stability Ratio

Customer satisfaction SLA % SLA met if the agreement uses FSM as a basis for the contract

Business Process Support

Innovation usage

% IT solutions supporting process improvement projects

project measurement using FSM

Requirements Management

Requirement Turnover Index [MELI01]

RTI = [(j CRFSj)/ Final FSunits] * 100

CRFS = Change Request Function Size units

Showing the level of turbulence in requisites during the development phase

Problem Management

DR – Defect Ratio

AR – Application Reliability

Business Growth Market Share % Market share increasing % using FSM as an initial contract condition

Table 1 – FSM-based measures : Customer Perspective

GOAL/OBJECTIVE DRIVER INDICATOR COMMENTS / EFFECTS PROCESS (PR)

Application Development & Maintenance

Size FSunit – Functional Size unit,

According to the FSM method used, it can be expressed for instance by:

FP – Function Points

Cfsu - COSMIC functional size units –

PS – Portfolio Size

Effort WE – Work Effort

Productivity PDR – Project Delivery Rate

EP – Enterprise Productivity

Support ASR – Application Support Rate

DDR – Duration Delivery Rate

AMPL – Application Maintenance Load per Person

Defectability & Test

RCR – Repair Cost Ratio

SR – Stability Ratio

DR – Defect Ratio

TPR – Testing Proficiency Ratio

MTTR – Mean Time To Repair ratio

AR – Application Reliability

DER – Defect Detection Ratio

# defects / 100 FSunit according to user acceptance

Reuse FR – Functional Reuse %

TR – Technical Reuse %

Table 1 – FSM-based measures : Process Perspective

COSMIC:

Balanced

Scorecard


Sellami: Verification from

a Metrology Perspective

Khelifi: COSMIC & Design

of Standard Etalons

Almakadmeh: COSMIC

Scaling & Approcimation

Abu Talib – COSMIC

for Early Quality Assessment

Soubra: COSMIC – Automation

& Verification Protocol

Sarayreh: NFR & COSMIC

Measurement

Marin & Trudel: FSM & Defects

Detection

Monsalve: COSMIC & Business

Process Models

COSMIC:

Metrology-related

R&D

Portability in

ISO 9126 &

25000 series


Figure 4 – Quality model for external and internal quality

external and internal quality

functionality reliability

maturityfault tolerancerecoverability

usability

understandabilitylearnabilityoperability

attractiveness

efficiency

time behaviour

resource utilisation

maintainability

analysabilitychangeability

stabilitytestability

portability

adaptabilityinstallabilityco-existencereplaceability

suitability

accuracy

interoperability

security

functionality compliance

reliability compliance

usability compliance

efficiency compliance

maintainability compliance

portability compliance


Figure 1: Mapping system-FR and system-NFR to software-FR

Non Functional

Requirements


European Cooperation for Space Standardization - ECSS

ECSS 40 series:

• Portability

• Reliability

• Performance

• Maintainability

• Security

• Interfaces

• Configuration

• Databases

• …

Non Functional

Requirements

Portability in

ECSS standards


Fig. 3 Software Component Portability functions: system modeling view

Table 8 Function types and functions for portability requirements that may be allocated

to software

Function

Type

Function

Type

Name

Portability Functions

1

Software

Component

Portability

Independence of the Operating System Function (IOSF)

Independence of the Middleware Function (IMF)

Independence of the Programming

Language Virtual Machine Function

(IPLVMF)

Independence of the Browser Function (IBF)

2

Data

Component Portability

Independence of the Database Function (IDF)

Distributed Data Base Management

System Function (DDBMSF)

3

Hardware

Component

Portability

Independence of the Client Function

(ICF)

Independence of the Server Function (ISF)

Independence of the Storage Function (ISTF)

Independence of the Network Function (INF)

4

Isolation of

System

Calls

Portability

Isolation of Software System Calls

Function (ISSCF)


Fig. 4 Software Component Portability functions: COSMIC modeling view

Portability

SOA Example

Fig. 11 COSMIC-SOA reference architectural model of system portability

requirement

1- Software Component Portability 2- Data Component Portability

4- Isolation of System Calls

Portability

3- Hardware Component Portability

Intermediary

Service

Portability Functional Type 1


Portability Functional Type 3 Portability Functional Type 4

Boundary

Functional

Process or

Service

Direct Data Movements

(E) ENTRY (X) EXIT

Indirect Data Movements

(R) READ (W) WRITE

Persistent

Storage

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

Isolation of the Software

System Calls Service (ISSCS)

Persistent

Storage

Independence of the Operating

System Service (IOSS)

Independence of the Middleware

Service (IMS)


Language Virtual Machine Service

(IPLVMS)

Independence of the Browser Service (IBS)

I

N

T

E

R

M

E

D

I

A

R

Y Service

(IS)

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

R

X

W

E

R

X

W

E

R

X

W

E

R

X

W

E

Persistent

Storage

Independence of the Client Service

(ICS)

Independence of the Server Service

(ISS)

Independence of the Storage

Service (ISTS)

Independence of the Network

Service (INS)

I

N

T

E

R

M

E

D

I

A

R

Y Service

(IS)

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

R

X

W

E

R

X

W

E

R

X

W

E

R

X

W

E

E

X

X

E

E

X

X

E

Persistent

Storage

Independence of the Database

Service (IDS)

Distributed Database

Management System Service

(DDBMSS)

R

W

R

W

E

X

X

E

E

X

X

E

Independence of the

Operating System

Function (IOSF)

Independence of

Middleware

Function (IMF)

Independence of the

Programming Language Virtual

Machine Function (IPLVMF)

E

X

X

E

E

X

X

E

E

X

X

E

Independence of the Browser

Function (IBF)

E

X

X

E

Independence of the

Client Function (ICF)

Independence of Server

Function (ISF)


Function (ISTF)


Function (INF)

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E


Function (IDF)

E

X

X

E

E

X

X

E


Management System Function

(DDBMSF)

Isolation of the Software System

Calls Function (ISSCF)

E

X

X

E

IS1IS2

IS2IS3

IS3IS4

IS4IS5

IS5IS6

IS6IS7

IS7IS8

IS8IS9 IS9IS10

IS10IS11


Fig. 7 COSMIC reference framework of system portability requirements that

may be allocated to software 1-Software Component Portability

2- Data Component Portability

3- Hardware Component Portability

4- Isolation of System Calls Portability

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

Isolation of Software System Calls

Function

(ISSCF)

Persistent

Storage

Independence of the Operating

System Function (IOSF)

Independence of the Middleware

Function (IMF)


Language Virtual Machine Function

(IPLVMF)

Independence of the Browser Function

(IBF)

I

N

T

E

R

M

E

D

I

A

R

Y Service

(IS)

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

R

X

W

E

R

X

W

E

R

X

W

E

R

X

W

E

Persistent

Storage

Independence of the Client Function

(ICF)

Independence of the Server Function

(ISF)


Function (ISTF)


Function (INF)

I

N

T

E

R

M

E

D

I

A

R

Y Service

(IS)

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

E

X

X

E

R

X

W

E

R

X

W

E

R

X

W

E

R

X

W

E

E

X

X

E

E

X

X

E

Persistent

Storage


Function (IDF)


Management System

Function (DDBMSF)

R

W

R

W

E

X

X

E

E

X

X

E

Intermediary

Service





Boundary

Functional

Process

Direct Data Movements

(E) ENTRY (X) EXIT

Indirect Data Movements

(R) READ (W) WRITE

Persistent

Storage

E

X

X

E

E

X

X

E

Table 9

COSMIC-SOA measurement example for the interactions between a functional

process and its own service process

COSMIC-SOA Types

Data Movement

Description

Data

Movement

Type Functional

Process

Service

Process

Independence of the

Operating

System Function

(IOSF)

Independenc

e of the

Operating

System Service

(IOSS)

IOSF sends a data group to

IOSS X

IOSS receives a data group

from IOSF E

IOSS sends a data group to

IOSF X

IOSF receives a data group

from IOSS E

Functional Size (subtotal): 4 CFP

Table 10

COSMIC-SOA measurements of the interactions between the 11 functional

processes and their service processes

Fun

ID

COSMIC-SOA Types of exchange services for

System Portability NO. Data Movements Functional Process Service Process

1 Independence of the

Operating System

Function (IOSF)

Independence of the

Operating System

Service (IOSS)

4


Middleware Function

(IMF)

Independence of the

Middleware Service

(IMS)

4


Programming Language Virtual Machine

Function (IPLVMF)

Independence of the

Programming Language Virtual Machine Service

(IPLVMS)

4


Browser Function (IBF)

Independence of the

Browser Service (IBS)

4


Database Function (IDF)

Independence of the

Database Service (IDS)

4

6 Distributed Database Management System

Function (DDBMSF)

Distributed Data Base Management System

Service (DDBMSS)

4


Client Function (ICF)

Independence of the

Client Service (ICS)

4


Server Function (ISF)

Independence of the

Server Service (ISS)

4

9 Independence of the Storage Function (ISTF)

Independence of the

Storage Service (ISTS)

4


Network Function (INF)

Independence of the

Network Service (INS)

4

11 Isolation of the Software

System Calls Function (ISSCF)

Isolation of the Software

System Calls Service (ISSCS)

4

Functional Size: 44 CFP

Portability

SOA Example

List of topics






5. The future…..?


An analogy...

2000 sf 4000 sf

Software

Functionality

Software

Functionality

120 CFP 240 CFP

CFP = COSMIC Function Point


- In the physical software itself

- In description of software functional requirements

- In business process modeling

- In engineering applications

- In information processing

- In information theory……………?

Universality

of

COSMIC

Concepts?


How do we measure in the physical world?

with concepts & 7 base units

+ traceability to SI measurement ‘etalons’

What can we measure & how do we measure in the

software world?

Are we getting close to an 8th base unit ?

+ traceability to an SI measurement ‘etalon’?

Measurement

as a

Technology

What does it take for software measurement to be adopted as a new technology across an industry?

– Software measurement must already have been proven

to work well in a large variety of contexts: • i.e. it must be mature as a technology, or maturing rapidly.

– Software measurement must become integrated into the technological environment of the software industry.

– It must become integrated into the business context (which includes its legal & regulatory aspects).


Conclusion


When will we get:

1. An International Base Unit

for software?

2. Our golden measurement

étalons for software design

& control?

Questions?


[email protected]

Date post:	13-Apr-2017
Category:	Software
Upload:	cosmic-common-software-measurement-international-consortium
View:	827 times
Download:	1 times