TOS-GCOPS-GD

Three Years of ECSS Software Standards

An Appraisal and Outlook

OPS-G Forum20th January 2006

D. Ponz & M. Spada

20th Jan 2006

2OPS-GD

Outline Why Software Engineering

The Standardisation Process

ECSS Software Standards applicable to ESA Software

Key issues in application of the ECSS Standards

Tailoring of ECSS Standards for Ground Segments First step: SEMG Second step: SETG

Next Steps

Conclusions

20th Jan 2006

3OPS-GD

What is Software Engineering?

Software engineering is about systematic development, evaluation and maintenance of software

Surveys have shown that the average corporation pays 3 to 7% of its annual revenues for hardware and software for corporate data processing and another 2 to 5% for maintenance: that’s 5-10% of the economy in the developed world

Another measure: software project failures cost the US economy about 60 B$ annually – double this for the whole world*

Information technology is mission critical for many businesses - rely on the efficiency and integrity of their IT systems for their business needs

Means that software engineering is vital  * Source 2002 study by America's National Institute of Standards (NIST),

quoted in Economist of Nov 25 2004

20th Jan 2006

4OPS-GD

Software Engineering Standards

Many people have to develop or maintain software or manage the related processes It follows that software engineering standards are used

by many people The standards must therefore be readily

understandable to a wide community

This is not the case for many other engineering standards: e.g. we all use TCP/IP many times on a daily basis,

most of us without any special knowledge of TCP/IP, but implementing TCP/IP packages and tuning TCP/IP communications systems is very specialised activity

20th Jan 2006

5OPS-GD

The Standardisation Process

Standardisation organisations like the European Cooperation for Space Standardisation (ECSS) and the Consultative Committee for Space Data Systems (CCSDS) exist to produce and maintain standards

They have established structures/procedures for reviewing and approving standards WGs, Independent peer review, committee hierarchy

etc.

In addition, some technical standards can be verified by prototyping, bread-boarding or an initial implementation before release of the standard E.g. CCSDS SLE standards

But not possible for a software engineering standard

20th Jan 2006

6OPS-GD

Standardisation Process - Software Engineering Standards

Not practical to “test “a software engineering standard before release because

Would involve a significant implementation project – would take too long, would not easily find volunteer projects

A software engineering standard reflects best practices as understood by the writers of the standards

Incorporates the authors’ development cultures

Users of the standard are in effect verifying it!

Users’ development cultures may differ from those of authors It can result in interpretation problems

Tailoring complicates it further: Users may exercise the processes in ways which were not thought

about by the authors or only subset of the processes

20th Jan 2006

7OPS-GD

Standards Applicable to ESA Software

ECSS-E-40B Space Engineering – Software

E-40 Part1B: Principles and Requirements (Nov. 2003) Covers all aspects of space software engineering

Complete life cycle Is defined in terms of

Engineering processes,which may be broken down into activities and tasks

Process interfaces with management and product assurance Inputs and outputs to processes

E-40 Part2B: Document Requirements Definitions (DRDs) (Mar. 2005)

Defines the contents of documents in line with expected outputs of processes/actrivities defined in E-40 Part1

20th Jan 2006

8OPS-GD

Standards Applicable to ESA Software (Cont.)

ECSS-E-40B is or will be complemented by the following level 3 (technical domain specific) standards:

Approved for start in 2007Software development for reuseECSS-E-40-06

Approved for start in 2006Simulator development process and interface

ECSS-E-40-07

In drafting (started summer 2005)

Software Verification, Validation and testing

ECSS-E-40-05

CompleteSoftware Life-CycleECSS-E-40-04

CompleteGround Segment SoftwareECSS-E-40-03

CompleteSpace Segment SoftwareECSS-E-40-01

StatusTitleIdentifier

Approved for start in 2007Software development for reuseECSS-E-40-06

Approved for start in 2006Simulator development process and interface

ECSS-E-40-07

In drafting (started summer 2005)

Software Verification, Validation and testing

ECSS-E-40-05

CompleteSoftware Life-CycleECSS-E-40-04

CompleteGround Segment SoftwareECSS-E-40-03

CompleteSpace Segment SoftwareECSS-E-40-01

StatusTitleIdentifier

20th Jan 2006

9OPS-GD

Standards Applicable to ESA Software (Cont.)

ECSS-Q80-B: Space Product Assurance – Software (Oct. 2003) Covers

SW product assurance programme implementation: organisation, responsibilities, etc. for SW PA

SW process assurance: lifecycle and requirements applicable to different phases

SW product quality assurance: metrification, product quality requirements, etc.

Management Standards – not software specificM-00B Policy and Principles (August 2003)M-10B Project Breakdown Structures (June 2003)M-20B Project Organisation (June 2003)M-30A Project Phasing and Planning (April 1996)M-40A Configuration Management (April 1996)M-50A Information/Document Management (April 1996)M-60A Cost and Schedule Management (April 1996)M-00-03A Risk Management (April 2000)

20th Jan 2006

10OPS-GD

Application of Standards

Solution: Use prescriptive standards but allow tailoring

ECSS Approach: Tailoring of the standards to Programme/Project needs

Tailoring influencing factors Programme/Project characteristics e.g.

Programmatic and financial constraints Technical parameters Scope and type of procurement

Inhomogeneous implementationsLeaves details to usersFree

Too heavy to applyDefines all details & covers all possible cases

Prescriptive

CommentCharacteristicsType of standard

Inhomogeneous implementationsLeaves details to usersFree

Too heavy to applyDefines all details & covers all possible cases

Prescriptive

CommentCharacteristicsType of standard

20th Jan 2006

11OPS-GD

The Challenge of Tailoring aSoftware Engineering Standard

Requires detailed understanding of the whole standard, Average software developer or project manager does

not have this

Standard is open to interpretation - ambiguous

Almost no guiding literature available – also for ISO 12207, the “parent” standard of E-40

Limited advice in ECSS literature, e.g. a short annex to ECSS-E-40B

20th Jan 2006

12OPS-GD

Tailoring Options

1. Use a tailoring tool Questionnaire based The answers are used to define a table with

Retained requirements Output documents required

TEC-SWE have developed a useful tailoring tool for E-40

2. Organizational tailoring Suited for an organisation with similar projects The same tailoring could be used for all

3. No tailoring at all! Let the contractor do it In conflict with ESA Policy (IPOL(2004)6) – customer should do it Only works where there is no competition – otherwise “playing field not

level”

20th Jan 2006

13OPS-GD

Particular problems of tailoring E-40

E-40 is defined in terms of Processes Inputs and outputs to processes Processes may be broken down into activities with their own

inputs and outputs

Problem: Have to map outputs into a set of supplier deliverable

documents Outputs must be aggregated into manageable set of

deliverables If not done can get large number of deliverables

Not practicable, very costly, difficult configuration management

Problem for both customer and supplier The E-40 DRDs were not available for a long time

20th Jan 2006

14OPS-GD

The “ Many Standards” Issue for Software

Problem: ECSS-E-40 addresses the technical processes needed to define,

design, build and maintain software Producing and maintaining software also involves the disciplines of

management and product assurance In the ECSS scheme need to use standards from the

PA branch (ECSS-Q series) Management branch (the ECSS-M series)

Solutions possible: Use E-40 only or E-40 and Q-80

E-40 and Q-80 contain basic management elements Combine all relevant ECSS standards into a single volume

(ESOC and Galileo approach)

20th Jan 2006

15OPS-GD

Stated with BSSC ESA Ground Segment Software Engineering and Management Guide – SEMG, BSSC(2002)4 adopted by ESOC in 2002 became applicable to new projects, e.g. GOCE and Herschel/Planck,

TMTCS Backend System

Structured in three Parts: Part A: Software Engineering

Software lifecycle processes in accordance with the E-40 Standard Part B: Software PA & Project Management

Software project management, configuration and documentation management and software product assurance in accordance with ECSS Q-80 and M-series Standards

Part C: Document Templates

First ECSS Software Engineering Standards Tailoring: the SEMG

20th Jan 2006

16OPS-GD

Experience with the SEMG

ECSS-E-40 and ECSS-Q-80 are consistent E.g. Tailoring of Q-80 flows naturally from that of E-40

ECSS-M standards are Focused more on overall space project management Not adapted for software engineering Stylistically inconsistent with E-40 and Q-80

Following a period of usage of the SEMG, the following needs arose

To align the SEMG with the latest versions of the ECSS Standards To provide traceability to the ECSS applicable standards To retrofit the users experience with the usage of SEMG To address some specific problems with the first tailoring e.g.: far too

many (over 50) deliverables Due to lack of clarity on mapping of deliverables to process outputs, especially in

ECSS-E-40 Tailoring not critical enough (did not go far enough)

20th Jan 2006

17OPS-GD

Tailoring of ECSS Software Standards for Ground Segments in ESA – SETG, BSSC 2005(1)

BSSC established a support contract with Fraunhofer Institute for Experimental Software Engineering (IESE) in July 2004 IESE has experience in the software standards domain IESE is an experimental institute which brings together a solid

theoretical background with a down-to-earth mindset

Main process steps Interview of main stakeholders: feedback on SEMG, proposals for

improvement Review of latest ECSS standard versions against SEMG Review by BSSC and interviewees New tailoring started in June 2004 SETG published in June 2005

A step forward – The SETG

20th Jan 2006

18OPS-GD

The process descriptions must be streamlined and made more structured, with clear identification of the activities to be performed

Explicit identification of process outputs is required

The review/update cycle for outputs/documents should be made more explicit

Explicit identification of reviews inputs and outputs should be provided

The processes outputs should be mapped to documents as far as possible

The number of documents to be produced (about 50 in SEMG) was excessive to be reduced as far as possible by elimination of redundancies and

aggregation of outputs

No traceability to ECSS requirements. Tailoring is not transparent

Feedback on SEMG

20th Jan 2006

19OPS-GD

Traceability to ECSS requirements has been provided

The processes and the terminology have been aligned with the latest relevant ECSS standards version

The processes have been consolidated in line with ESA Ground Segment practice

The processes outputs have been detailed and associated with documents and reviews as far as necessary

Reviews inputs and outputs have been detailed

Traceability of outputs vs. documents has been provided

The number of mandated documents has been considerably reduced (from 50 to 16!) and the document life-cycles have been made explicit

Main changes in the SETG

20th Jan 2006

20OPS-GD

The SETG has four parts:

Part A: Software Engineering Software lifecycle processes in accordance with the E-40 Standard

Part B: Product Assurance and Management Software project management, configuration and document

management and software product assurance in accordance with ECSS Q-80 and M-series Standards

Part C: Document TemplatesPackaging of processes outputs

Part D: Traceability Against ECSS StandardsTraceability Matrix with respect to ECSS Standards RequirementsOutputs vs. Documents

The SETG

20th Jan 2006

21OPS-GD

The SETG Structure

Development Maintenance

Verification

Validation

Project management

Configuration management

Document management

Product assurance

Process assurance

Primary life cycle processes Support life cycle processes

Part B

Part A

20th Jan 2006

22OPS-GD

ECSS processes and the SETG

Maintenance

Delivery and acceptance

Design and Implementation

Architecture and interface design

Delivery and acceptance

Design and implementation

engineering

Requirements and architecture engineering

Validation

Verification

Maintenance

Operation

Management

PDR CDR QR AR

PDR CDR AR1 AR2SWRR

Requirements engineering

Management

Validation

Verification

E-40This volume

System engineering processes related to software

SETG

20th Jan 2006

23OPS-GD

ECSS processes and the SETG (Cont.)

ESTEC

Ground Segment Manager/ Final User

ESOC engineering

OPS

Industry

Customer that performs the

System requirement engineering

Customer that performs the

System requirement engineering Customer that

makes only System/Software

requirement engineering

Supplier that does only

development

Supplier that does only

development

software spec

-CDR

system software

spec – AR1

system software

spec – AR2

Customer that delegates the

system requirement

engineering to its Supplier

Supplier that does

everything

Supplier that does

everything

Supplier that does the rest

Supplier that does the rest

E40 applicable

SETG as a valid

tailoring of E40

SETG as OPS/

industry sw std

20th Jan 2006

24OPS-GD

Unchanged 529Tailored 20Tailored out 78

Tailoring Statistics

85%

3%

12%

unchanged

Tailored

Tailored out

20th Jan 2006

25OPS-GD

Table indicates when documents are produced/updated/reviewed

Documents Life-cycle

20th Jan 2006

26OPS-GD

The SETG is a tailoring of ECSS Standards for ESA Ground Segment Software

It is a more user-friendly version of the BSSC SEMG, aligned with the latest versions of the ECSS Standards

The traceability to the ECSS Standards is fully documented [SETG-Part-D]

The SETG is intended to be used as the applicable tailoring for ESOC software development and maintenance contracts

ESRIN is considering adopting the SETG for their Ground Segment Software

The SETG is available On the BSSC Web Sitehttp://www.estec.esa.int/wmwww/EME/Bssc/stds.htm On the BSSC LN Database

Summary

20th Jan 2006

27OPS-GD

Review OPS-QMS procedures to align them with SETG WG established in Q4 2005 WG reviewed 39 procedures DCRs raised on 19 documents Work has been completed

Tailoring required for small projects and studies Can be based on tailoring of ECSS-E-40 only Approach will be to keep this tailoring consistent with the

SETG Will be a BSSC activity for 2006

Next Steps

20th Jan 2006

28OPS-GD

Conclusions

Improvement of ECSS software standards needed These standards are not easy to use

Not “readily understandable” and unambiguous, inconsistent in some cases

Tailoring of ECSS software engineering standards is a significant challenge

E-40 and Q-80 give all options, but not what actually required in given project

Solutions have been identified, but there is still some way to go Proliferation of tailorings must be avoided

Define families of software applications Have tailoring done by experts Promising work done by

TEC-SWE: Tailoring toolhttp://www.estec.esa.nl/wmwww/EME/index.htm BSSC : Tailoring of ECSS Software Engineering Standards for

Ground Segments

20th Jan 2006

29OPS-GD

Conclusions (Cont.)

Software engineering combines the E, Q and M disciplines. The ECSS standards separation into E, Q and M documents is not helpful for software

Promising work done by BSSC Integrated tailoring of software engineering related

ECSS standards

Reviews of standards should not be for “elite” People with experience in the field should be included as far

as possible Reviews should be means to expose the standards to the

people who will use them Feedback Awareness Commitment