european space agency / agence spatiale européenne Issue 2.1 (25 Nov 02) Implementing ECSS Software...

Issue 2.1 (25 Nov 02)

european space agency / agence spatiale européenne

Implementing ECSS Software Engineering Standards at ESOC

A course on using the Ground Segment Software Engineering and Management Guide (GS SEMG)

John Barcroft

2

Course Contents


PART 1 OVERVIEW – PSS-05, ECSS and GS SEMG

PART 2

PART 3

PART 4

SOFTWARE ENGINEERING

MANAGEMENT and PRODUCT ASSURANCE

CONTRACTS and TAILORING

3

Overview Session – Contents

Introductions

Introduction to ECSS

Overview of ECSS-E-40 process-based standards

customer/supplier relationships

lifecycles

documentation

ECSS-E-40 and PSS-05-0

Applying ECSS software standards to ground segment software – GS SEMG

4

ESA Standards Policy

• In June 1994, ESA (Council resolution ESA/C/CXIII/Res 1)– adopted the new standards being prepared by the European

Cooperation for Space Standardisation (ECSS)

– decided to phase out the ESA PSS standards

5

ECSS Architecture Levels

• Level 0 (ECSS-P-00 Standardization Policy)– policy and objectives of the ECSS system and its architecture

– principles for document creation, validation and maintenance

• Level 1 documents– policy and principles in the specific domain

– global view of the requirements

– outline of interfaces between the elements (and the documents) at Level 2

• Level 2 documents – requirements and functions (“what to do” and expected output) for all aspects

in the individual domain

• Level 3 documents – methods, procedures and tools to achieve the requirements of Level 2

documents (“how to do”)

6

ECSS Standards Architecture

E-40 Software

E-30 Mechanicalengineering

Q80 Software productassurance

M-40 Configurationmanagement

M-20 Project organization

M-10 Project breakdownstructures

MANAGEMENT

M-00 Policy and principles

P-00 Standardization policy

Q-00 Policy and principles E-00 Policy and principles

P-01 Glossary of terms

M-30 Project phasing andplanning

M-50 Info./documentationmanagement

M-60 Cost and schedulemanagement

Q70 Material, mechanicalparts and processes

Q-60 EEE components

Q-40 Safety

Q-30 Dependability

Q-20 Quality assurance

E-60 Control systems

E-50 Communications

E-20 Electrical andelectronics

E-10 System engineering

PRODUCT ASSURANCE ENGINEERING

M-70 Integrated logisticsupport

E-70 Ground systems andoperations

Level 3standards

Level 3standards

Level 3standards

Level 0

Level 1

Level 2

7

Software Product Assurance: ECSS-Q-80

Software Project Management: Policy and Principles: ECSS-M-00Project Breakdown Structures: ECSS-M-10Project Organisation: ECSS-M-20Phasing and Planning:ECSS-M-30Configuration Management: ECSS-M-40Information/Documentation Management: ECSS-M-50

Engineering Standards: Systems Engineering: ECSS-E-10 Ground Segment Engineering: ECSS-E-70 Software Engineering: ECSS-E-40

ECSS Standards relevant to Ground Segment Software

8

ECSS Level-3 Software standards

• ECSS-E-40 is a “level 2” standard

• In preparation: level 3 standards:– ECSS-E-40-1 Space Segment Software

– ECSS-E-40-3 Ground Segment Software

– ECSS-E-40-4 Software Lifecycles

• Level 3 standards contain no new requirements – provide guidance (not “normative”)

9

ECSS-E-40 (SW Engineering) and ECSS-Q-80 (SW PA)

• Q. Who wrote them?

• A. ECSS Working Groups with participation of ESA, CNES and Industry

• A single WG has been responsible for both documents

• N.B. they are produced by ECSS and are not ESA or BSSC documents!

• Q. Where do I find ECSS documents?

• A. At http://www.ecss.nl

10

Basis of ECSS-E-40

• ECSS-E-40 is derived from ISO/IEC 12207

• ISO/IEC 12207 is the leading international software engineering standard– issued in 1995

– Information Technology – Software Life Cycle Processes

– intended for use in contractual situations

• ECSS-E-40 is a tailoring of ISO 12207 for space projects

11

Process-based Standards

• ISO 12207 and ECSS-E-40 are based on – processes

– requirements on those processes (component activities and their expected outputs)

• They are in effect “standards for making standards”– meta-standards, templates for standards

• Process approach comes from vogue for business process re-engineering of the 1990’s

• Process model can always be projected into a set of phased activities

AB

C

PrimaryProcesses

S1

S1 is a supporting process for a repetitive activity like review, which can be“called” by a primary process.

12

Tailoring

Tailoring is a process by which individual requirements or specifications, Standards and related documents are evaluated and made applicable to a specific project, by selection and in some exceptional cases, modification of existing or addition of new requirements

ECSS-E-40B

13

ECSS-E-40 and ISO 12207 Processes

ECSS-E-40 and ECSS-Q-80 are based on ISO 12207

5. PRIMARYLIFE CYCLE PROCESSES

5.1 Acquisition

5.3 Development

5.4 Operation

5.2 Supply

5.5 Maintenance

6. SUPPORTINGLIFE CYCLE PROCESSES

6.1 Documentation

6.2 Configuration Management

6.8 Problem Resolution

6.3 Quality Assurance

6.4 Verification

6.5 Validation

6.6 Joint Review

6.7 Audit

7. ORGANIZATIONAL LIFE CYCLE PROCESSES

7.1 Management

7.2 Improvement

7.3 Infrastructure

7.4 Training

ISO 12207

Processes:

Q-80

E-40

E-40/Q-80, M-

M- series

14

ECSS-E-40/Q-80 Scope

Concerns the “Product software”, i.e. software that is part of a space system product tree and developed as part of a space project.

Applicable to all the elements of a space system: the space segment, the launch service segment and the ground segment.

N.B. By comparison, PSS-05 applied to “all deliverable software implemented for ESA in house or by industry”

15

Customer-Supplier Relationships

Reviews are the main

interaction points

between the

customer and

supplier

16

E-40 Processes and Activities

Maintenanceprocess

Softwareverification

andvalidation

(supporting)process

Software management process

Software operationsengineering process

Software requirementsengineering process

Software design engineeringprocess

Software validation andacceptance process

System engineering processesrelated to SW

Development processes

17

ECSS-E-40 Overview

Legend:

SRR System Requirements ReviewSWRR Software Requirements Review

(GS SEMG option) PDR Preliminary Design ReviewCDR Critical Design ReviewQR Qualification ReviewAR Acceptance ReviewORR Operational Readiness

Review

Software Maintenance 5.8

Software Operations Engineering 5.7

ORRPDRSWRR

RequirementsEngineering 5.4

specified

DesignEngineering 5.5

CDR

defined

QR AR

Validation and Acceptance 5.6

qualified accepted

SRR

System Engineering 5.2

functionalState:

18

ECSS-E-40 Overview - Standard Customer-Supplier Relationship

Customer Supplier Customer

N.B. Analogue of FFP PSS-05 Development based on URD

Legend:



ReviewDesignEngineering

PDRSRR CDR QR AR

System Engineering

Validation and Acceptance

SWRR

RequirementsEngineering

Software Maintenance

Software Operations Engineering

ORR

19

ECSS-E-40 Overview - Alternative Customer-Supplier Relationship (GS SEMG)

N.B. Analogue of FFP PSS-05 Development based on SRD

Legend:



ReviewDesignEngineering

PDRSRR CDR QR AR

System Engineering


SWRR




ORR

Customer CustomerSupplier

20

Software Lifecycles

• The lifecycle defines the sequencing and dependency of the processes

• As with PSS-05, choice of the lifecycle is a management activity– the supplier must document the choice in the Software Development

Plan

• E-40 does not impose any lifecycle - it only requires that one is defined

21

Waterfall Model

Systemengineering

Requirementsengineering

Designengineering

Validation andacceptance

SRR SWRR PDR CDR ARQR

22

Evolutionary Model

Systemengineering


Designengineering



Systemengineering


Designengineering



23

ECSS-E-40 Overview: Document Outputs

Requirements Baseline (RB)Interface Requirements Document (IRD)Technical Specification (TS)Interface Control Document (ICD)Design Definition File (DDF)Design Justification File (DJF)Maintenance File (MF)Operations Plan (OP)

RB(IRD)

Customer DJF

TS(ICD)DDFDJF

DDFDJF

DDFDJFMF

DDFDJF

DJF OP

DesignEngineering

PDRSRR CDR QR AR

System Engineering


SWRR




ORR

24

Software documentation

N.B. Expected contents at each review specified in Annex A of ECSS-E-40

RBRequirements

Baseline

RBRequirements

Baseline

TSTechnical

Specification

TSTechnical

Specification

DDFDesign

Definition File

DDFDesign

Definition File

DJFDesign

Justification File

DJFDesign

Justification File

Customer’s requirements

Interface Requirements

Supplier Specification

Interface Control Document

Release information

Design of all components Justification of design trade-offs

Verification and Validation Plans

... ...

Software code

...

... ...

Milestone reports, test results

...

25

Special requirements

• Man-machine interfaces

• Software reuse

26

ECSS-E-40 to PSS-05 Relationship – 1 of 2

• Detailed analysis: “PSS-05-0 and ECSS-E-40 Comparison”, BSSC(98)2

• Main conclusion :

– PSS-05 mandatory practices cover many (~70%) of the ECSS-E-40 requirements,

– identification of ECSS-E-40 requirements not covered by PSS-05-0 practices

27

ECSS-E-40 to PSS-05 Relationship – 2 of 2

PA FASR/R

URAD

DDTR

OM

UR/R AD/R DD/R

SR

DesignEngineering

PDRSRR CDR QR AR

System Engineering


SWRR




28

Q-80 organisation

Software life-cycle

Requirements applicable to allsoftware development processes

Requirements applicable toindividual software developmentprocesses

6 Software Process Assurance

Product quality objectives andmetrication

Product quality requirements

Supporting documentation

Standard hardware for operationalsystem

Firmware

7 Software Product Quality Assurance

Organization and responsibility

Contractual aspects

Software product programme management

Subcontractor control

Risk assessment

Purchasing

Tools and supporting environment

Improvement process

5 Software Product Assurance Process Implementation

29

ECSS-E-40/PSS-05 Features Comparison

Processes versus practices

E-40 is to be used with other M and Q standardscf. PSS-05 is a “one-stop” shop

Process outputs “files” versus documents with strict tables of

contents

Integration within the system life-cycle (reviews)

versus software only projects

Customer/supplier relationship

Tailoring versus mandatory and recommended practices

E-40 is based on ISO 12207cf. PSS-05 is based on IEEE SW standards

E-40 addresses space system product tree: PSS-05 is a general

software standard

30

ECSS-E-40 and PSS-05

• Could you use PSS-05 as a response to ECSS-E-40?

• In principle yes

• But not recommended because :

– different terminology for reviews, outputs, etc

– PSS-05 does not completely cover ECSS-E-40

31

Transition from PSS-05 to ECSS SW Standards

• Difficulties:

– transition from a practice-based standard to a process-based standard

– many standards (E40, Q-80, ECSS-M-) instead of one (PSS-05).

32

ESA Ground Segment Software Engineering and Management Guide (ESA GS SEMG)

• Solves problems by

– providing ECSS compliance in the form of a set of practices

– covering all relevant ECSS standards in a single document

33

ESA Ground Segment Software Engineering and Management Guide(ESA GS SEMG)

• SEMG is based on “B” versions of E-40 and Q-80 – improved versions undergoing public review

• SEMG has three volumes– Part A:

Software Engineering covering ECSS-E-40

– Part B: Management covering ECSS-Q-80 and ECSS-M- series

– Part C: Document Templates

34

SEMG Part A: Software Engineering

• Drafted by an ESOC WG:– Members: Y Doat, G di Girolamo, G Gienger, A Head, M Jones, E

Perdrix and S Valera

• Technical Author from Anite (Richard Jack, then John Brinkworth and John Barcroft)

• Is in line with ECSS-E-40-3 “Space Engineering - Ground Segment Software”– ECSS guide to tailoring E-40 for ground segments

• Introduces another review (between SRR and PDR) to line up with PSS-05 practice.

35

ECSS-E-40 and ECSS-E-70 Relationship?


DesignEngineering


System Engineering

SWRR

SW Requirements Analysis

Architectural Design

PDRSRR CDR QR AR

E-40 Processes

E-40 Reviews

ECSS-E-70ACTIVITIES

Identify Characteristics,Constraints, Concepts. Assess feasibility.(G/S)perspective

GSRR GSPDRGSTVVRR

GSCDR

Define requirementson G/S& G/SBaseline

Complete G/S design to element level & start implementation

Procure G/S facilities & elements

Integrate, Verify & Validate G/S systems (includes preliminary validation of mission data

Train people& Validate full G/S (i.e. includes peopleand mission data

ECSS-E-70REVIEWS

GSTVVR

OVRR

A B C D

ORR

36

SEMG Part B : Management

Problems of using the ECSS-M standards

• requirements drawn from many ECSS source documents– inconsistent terminology

• requirements must be filtered out that do not apply to software and/or ground segment (spacecraft prime management aspects, spacecraft model philosophy…)

• not complete for software

37

SEMG Part B : Management

SEMG solves the problems by

• using consistent terminology and style in line with ECSS-E-40/Q-80

• removing inapplicable material

• where needed, providing missing software-specific material (mainly from PSS-05)

38

Status of GS SEMG

• Final revision by BSSC completed in March

• Officially published and now applicable

• Companion tailoring template available in draft

• Applicability: primarily intended for use in ground segment software development in D/TOS at ESOC

39

Afterthought: PSS-05 and the GS SEMG

• PSS-05-0 Issue 2 appeared in 1991

• The revision cycle for such standards is ca. 5-6 years

• PSS-05-0 is overdue for revision!

• Revision ruled out by the change in ESA policy (no new PSS)

• If it had been revised, it would probably have been brought in line with ISO12207

• The result may well have had similarities to the GS SEMG!

40

Transition to ECSS Software Standards at ESOC

• Approach described in BSSC(2000)2, July 2000 “Transition to ECSS Software Standards”– ECSS-E-40 and ECSS-Q-80 are applicable for the software activities

of ESA projects

– Not required to apply ECSS-E-40 retroactively to projects already using ESA PSS-05-0

• SEMG is implementation of ECSS software standards

• ESOC QMS changes identified (minimal), mainly– references to PSS-05

– use of ECSS terminology

41

Tailoring Template (draft)

• Lists inputs and outputs by process

• Gives

– tailoring condition (i.e. the circumstances in which the tailoring can be done)

– tailoring possibility (i.e. the details of the tailoring actions that can be applied)

• Can be used as a form to record a specific set of tailoring actions and the reasons for them

42

Course Contents



PART 2

PART 3

PART 4




43

Software Engineering Session – Contents

Recap

Overview of E-40 / SEMG software engineering processes

44

Recap

• SEMG– Part A:

Software Engineering covering ECSS-E-40

– Part B: Management covering ECSS-Q-80 and ECSS-M- series

– Part C: Document Templates

• Tailoring Template (draft)– Inputs and outputs for each process

– Tailoring condition

– Tailoring possibility

45

Scope

Software product Equipment

Software component

Software unit

Space system

Ground segmentLaunch service segment Space segment

Ground segment subsystem

ground operations organisation +

group of subsystems (facility) = ground segment entity

46

Critical software6.6

Mapping of SEMG Part A to E-40

System engineering for software3

Software management4

Software requirements engineering5

Software design engineering6

Software validation and acceptance7

Software operations engineering8

Software maintenance9Software verification and validation(supporting) processes

5.9

Software reuse10

Man-machine interfaces11

Space segment software6.2

Ground segment software6.3

5.2

5.3

5.4

5.5

5.6

5.7

5.8

6.4

6.5

Not applicable

Handled in-line in SEMG A

Refers reader to Q-80

E-40SEMG A

KEY

General requirements = 5.n

Special requirements = 6.n

Refers reader to E-40-3

47

Processes, activities and tasks

Activity

Task

‘shall’ mandatory

‘should’ recommended

‘may’ optional

Process OutputsInputs

Note:

ECSS-E-40 uses only ‘shall’

SEMG uses PSS-05 approach

48

Software Engineering Processes and Activities

handled in-line inprimary processes

Software requirements analysis

Software architectural design

Coding and testing

Design of software items Integration

Validation against TS

SW V&V

Validatin testingagainst RB subset-2

Validation testingagainst RB subset-1

Delivery andinstallation

Validation testingagainst RB

Operational planning

Operational testing

Systemoperation

Usersupport

Problem and modificationanalysis

Modification implementation

Maintenance review/acceptance

Software migration

Software retirement

Planning Selection of the software life cycle model Technical budget and margin management

SYSTEM ENGINEERING FOR SOFTWARE

SOFTWARE REQUIREMENTS ENGINEERING PROCESS

SOFTWARE DESIGN ENGINEERING PROCESS

SOFTWARE VALIDATION AND ACCEPTANCE PROCESS

SOFTWARE MANAGEMENT PROCESS

SOFTWARE OPERATIONSENGINEERING PROCESS

MAINTENANCE PROCESS

SOFTWAREVERIFICATION ANDVALIDATION(SUPPORTING)PROCESSES

DEVELOPMENT PROCESSES

49

System engineering for software

Done by Customer

• System requirements analysis

• System partitioning

• System-level requirements for software verification and validation

• System-level requirements for software integration

milestone is SRR

Note: ECSS-E-40 sees each software element in relation to the system e.g. to the next higher level,not just a “stand-alone” user requirements statement

RB DJF

PDRSWRR


DesignEngineering

CDR QR AR


SRR

System Engineering

IRDIRD

50

Software management

• Planning

• Selection of the software life cycle model

• Technical budget and margin management

SDP

(Software Development Plan)

51

Life cycle models

Systemengineering


Designengineering



Waterfall

Systemengineering


Designengineering



Systemengineering


Designengineering



Evolutionary

52

Group Activity

SCENARIO

• Incremental delivery model

• 3 phases with increasing functionality

TASKS

• Map the processes and reviews onto the development life cycle

(consider which processes and reviews to iterate)

53

Software requirements engineering process

• Software requirements analysis

– roughly equivalent to PSS-05 SR phase

– milestone is SWRR (GS SEMG option)

• Software architectural design

– roughly equivalent to PSS-05 AD phase

• Software verification and validation [planning]

milestone is PDR

DDF DJFTS

PDRSWRR


DesignEngineering

CDR QR AR


SRR

System Engineering

ICDICD

54

Software design engineering process

• Design of software items

• Coding and testing

• Integration

• Validation testing against the technical specification (equivalent of PSS-05 System Testing ST)

milestone is CDR

cf. PSS-05 DD Phase

DDF DJF

+ updates to ICD

PDRSWRR


DesignEngineering

CDR QR AR


SRR

System Engineering

55

Software validation and acceptance process• Validation testing against RB subset-1

– milestone is QR

– carried out in supplier’s environment

(no exact PSS-05 equivalent)

• Delivery and installation

• Validation testing against RB subset-2 (or TS)

– carried out at customer’s site on development platform

• Validation testing against RB

– milestone is AR

– carried out in operational environment

(like PSS-05 AT)

N.B Middle two activities are like PSS-05 TR Phase

DDF DJF

PDRSWRR


DesignEngineering

CDR QR AR


SRR

System Engineering

56

Validation testing – summary of the four stages

Validation against Name Site / platform Rev. Also known as

TS Validation Testing against TS

Supplier Premises

CDR System Tests (can also be carried out as FAT: Factory Acceptance Tests)

RB subset Validation Testing against RB Subset – 1

Supplier Premises

QR Preliminary Acceptance Tests. Factory Acceptance Tests (FAT)

RB larger subset and optionally a subset of the TS (2)

Validation Testing against RB Subset –2

Customer Premises on Development Environment

AR Preliminary Site Acceptance Tests PSAT (1)

RB (ideally all requirements) and optionally a subset of the TS

Validation Testing against RB

Customer Premises on Operational Environment

AR (Final) Site Acceptance Tests SAT (1)

(1) PSAT and SAT are known together as Operational Acceptance Tests

(2) PSAT can also be carried out against the TS

57

Software operations engineering process

• Operational planning (i.e. plans, procedures, standards for the following …)

• Operational testing

• System operation

• User support and anomaly handling

– (cf. PSS-05 first-line maintenance)

OP

58

Software maintenance process

• Problem and modification analysis

• Modification implementation

• Maintenance review/acceptance

• Software migration (cf. PSS-05 “adaptive” maintenance)

• Software retirement

Like the PSS-05 OM phase, but E-40 • places more emphasis on migration and retirement • separates first-line maintenance into software operations

MF

59

Software Re-use

• Developing software for intended re-use

• Customer requirements

• Supplier requirements

• Re-using software from other projects

• Use of third party COTS

RB TS SDP DJF

60

Man-Machine Interfaces

• Determine prototyping requirements

• Determine MMI standards

• Supplier consideration of MMI aspects

RB TS DJF

61

Course Contents



PART 2

PART 3

PART 4




62

Management and Product Assurance Session – Contents

Overview of Part B Project Management

Configuration Management

Product Assurance

Software Reuse and Purchasing of COTS Software

Product and Process Metrics

Process Improvement

Feedback on SEMG and tailoring template

63

Mapping of SEMG Part B to Q-80

Software Project Management2

Configuration Management3

Software Product Assurance4

Product Assurance Management4.2

Product Assurance4.3

Process Assurance4.4

Q-80

5

6

7

Software Product Assurance Process Implementation

M- seriesSEMG B

KEY

Q-80

Software Process Assurance

Software Product Quality Assurance

Software Product Assurance Plan4.5

Product Assurance and the Software Lifecycle

4.6

64

Software Project Management

• The Software Development Plan

• Project Management Tasks

• Project Management and the Software Lifecycle

65

The Software Development Plan

• Customer specifies project management requirements

• Supplier responds with Software Development Plan, including

– project organisation

– project phasing and planning

– project work breakdown structures

– cost and schedule management.

66

Software Project Management

• Organising the Project

• Tailoring

• Management Interface

• Roles, Responsibilities and Authority

• Risk Management

• Technical Management

• Cost Management

• Schedule Management

• Reporting Project Progress

business agreement

by customer in RFP, response by supplier

customer access for audit, inspection

customer and supplier project managers, organisation

risk management policy and risk register; product assurance involvement

methods and tools and enforcing use; organising assurance, CM, V&V

costs by WBS and category, payment milestones, CCNs

timings, resource allocation, ‘exception’ reporting

regular reports and progress meetings (per requirements or contract)

67

PDRSWRR


DesignEngineering

CDR QR AR


SRR

System Engineering

Project Management and the Software Lifecycle

• ‘Each planning document exists in outline from the start of the project. It is updated prior to the start of each phase to provide increasing levels of detail, so that each phase is fully planned prior to its commencement’

• SEMG B spells out what this means for each life cycle process

68

Configuration Management

• Configuration Management Processes

• The Configuration Management Plan

• Configuration Management and the Life Cycle

69

Configuration Management Processes

• Configuration Identification

• Choosing Configuration Items

• Configuration Item Content

• Configuration Item Storage and Distribution

• Baselines

• Forming and Maintaining Baselines

• Media

• Configuration Change Control

70

Configuration Management Plan

• Defines organisation, methods, means and procedures to manage the configuration

• Exists for each phase, but appropriately scoped

• Defines

– document approval/agreement procedures

– related document management activities

– process for receipt, assessment, authorisation and distribution of documents.

71

Configuration Management and the Life Cycle

Document Review at Which Document is Brought under Configuration Control

Requirements Baseline Documents

SRR

Technical Specification Documents

PDR

Design Definition File Documents

CDR

Design Justification File Documents

Review for which they provide evidence

Product Assurance File Documents

Review for which they provide evidence

Operations/Maintenance File Documents

QR

72

Product Assurance (1 of 2)

• This completes Part A in relation to quality, specifying when product assurance is performed (and often repeating A)

• Purpose of software PA: provide adequate assurance that the software product and processes conform to their specified requirements and adhere to the established plans, by measuring and controlling quality of

– product, indirectly, using metrics

– process, directly, by ensuring that

• documented and verified processes are used

• a system of process improvement is employed

• Direct control of product quality is software validation and acceptance (Part A)

73

Product Assurance (2 of 2)

• Product Assurance Manager (engineer also mentioned)

• Product Assurance Plan, maintained throughout project

• Larger, more risky projects need a product assurance function

74

Overview of ContentsPROCESS ASSURANCE

• Process Improvement

• Sub-Supplier Control

• Purchasing COTS Software

• Re-use of Software

• Tools, Techniques and Methods and Supporting Environment

• Hardware Environment for Operational System

• Process Metrics

• Verification and Validation

• Software Problem Reporting

• Non-Conformance Reporting

• Assurance of the Configuration Management Process

SOFTWARE PRODUCT ASSURANCE PLAN

PRODUCT ASSURANCE AND THE SOFTWARE LIFE CYCLE

PRODUCT ASSURANCE

• Product Quality Objectives and Metrics

• Procedures and Standards

• Software Dependability and Safety

75

Re-use process (from Part A)

• Re-use of existing software (also COTS, public domain, open source) should give

– cost and time savings

– improved quality (already tested)

• Software developed for re-use

– may involve additional costs

• Topics addressed:

– Developing software for intended re-use

– Customer requirements

– Supplier requirements

– Re-using software from other projects

– Use of third party COTS

76

Purchasing COTS software

• Customer identifies acquisition process for COTS/OTS/MOTS

• Selection criteria (ref. to Part A)– financial stability

– supplier track record with customer

– support and maintenance

– access to documentation and source code (esp. for critical software)

– licensing and IPR

– suitability for intended use

• Component list in DJF for customer approval– description, ordering criteria, inspection criteria, maintenance and upgrade

arrangements, back-up if unavailable, licensing

• COTS product subject to configuration control

• Receiving inspection by supplier: report detailing any problems

77

Re-use (Part B) – 1 of 2

• Supplier finalises investigations on re-use of existing software– check validity of re-using tools and methods

– identify necessary adaptation

– check level of validation

– check documentation status

– check quality status (NCRs, waivers)

– supplier certification re all relevant tests on relevant platform

• Document in software re-use records (DJF)

• Same standards for re-used as for bespoke (‘purchased non-COTS’)

78

Re-use (Part B) – 2 of 2

• If levels of product assurance fall short, supplier may

– analyse software life cycle data

– generate documentation by reverse engineering

– use product service history• configuration management and change control information

• effectiveness of problem reporting records

• stability and maturity (from change control records)

• relevance of product service history to new environment (e.g. problems confined to areas not used)

• error rates and maintenance records

• impact of modifications

• If remaining doubts on fitness for purpose, additional verification may be agreed

• Customer approves re-use based on software re-use records (DJF)

79

Metrics

• General

– quality model (e.g property, metric, evaluation method)

– collect, store, analyse, report, act

• Product metrics

– size (design, code)

– complexity (design, code)

– …

• Process metrics

– actual duration of phases/tasks compared to plan

– effort used in phases/tasks compared to the plan

– number of SPRs generated during verification (i.e. review or audit)

– number of SPRs generated during integration and validation testing and use.

80

Group Activity

SCENARIO

• The list of possible product metrics is to be extended

TASKS

• What properties might you want to evaluate?

• What metrics would you collect for them?

• How would you derive them?

81

Process Improvement (1 of 2)

• ‘Collect, store, analyse, report, act’ on process and product metrics

• Identify weaknesses by analysing metrics collected

• Identify improvements to address the weaknesses

• Implement those improvements

Process improvementusing metrics

COLLECT

STORE

ANALYSEREPORT

ACT

82

Process Improvement (2 of 2)

Management responsibility

Resource management

Measurement, analysis and improvement

Product realization

Customers (and other interested

parties

Customers (and other interested

Parties)

Requirements

Satisfaction

Product

Continual improvement of the quality management system

ISO9001:2000

CMM SPICE for SPACE (S4S)

83

Course Contents



PART 2

PART 3

PART 4




84

Contracts and Tailoring Session – Contents

Software procurement and the QMS

Statements of Work

Software Metrics (revisited)

Tailoring

Recap

Feedback on SEMG and tailoring template

85

What the ESOC QMS says

86

PR-7110

• … specifies how the existing ECSS software engineering standards will be applied within ESOC.

• … addresses in particular […]:

– responsibilities associated with software procurement and development

– activities associated with the software lifecycle

– tailoring criteria to be applied as function of software project criticality, size and lifetime

87

PR-7110

• … is applicable to – all software developed for ESOC and incorporated in or used

by ESOC products and services

• … is not applicable to– software developed to support feasibility studies and

calculations, administrative tools

– software available as commercial of the shelf (COTS).

• Any deviations from this and associated procedures, or any additional requirements, to be specified in SOW and reflected in SDP

88

PR-7110 – applicable documents

Software Engineering and Management Guide

Part B Management


Part C Document Templates

The Tailoring Template

QMS PROCEDURES AND WORK INSTRUCTIONS

ON CONTRACTS AND PROCUREMENT

PR-6200-TOS Anomaly and Problem Identification,Reporting and Resolution

WI-2102-TOS Software Tools and Methods in Software Validation

PR-7410-TOS Project Control and Management

PR-5100-TOS Configuration Management

PR-7300-TOS Critical Supplier Evaluation and Control

PR-7100-TOS Control of Procurement via Contracts

PR-0400-TOS Risk Management

PR-1500-TOS Management of Operations Training Activities

PR-7200-TOS Procurement Using Purchase Orders

WI-7500-TOS Service Level Agreements, Format and Management

PR-5300-TOS Document Production and Change


Part A Software Engineering

89

PR-7110 – roles and responsibilities (1 of 3)

• Quality Manager– designates (with GS Manager) the PA Representative

– reviews procurement contract

• Technical Officer– writes statement of work (SOW) and contract technical specification (CTS)

– monitors contract at technical level

– approves software development, configuration, validation and verification plans

– reviews software product assurance plan

– interfaces with initiator(s), end-users, ESA-internal support units, and Supplier Software Project Manager

– monitors SW Quality Control activities and Problem Trend Analyses

90


• Product Assurance Representative– reviews tailoring as specified in SOW and reflected in the Software

Project Management Plan for compliance with this procedure

– approves the Software Product Assurance Plan

– authorises formal release of software for operational [validation and] use following AR.

91


• Software Review Board (SRB)– control authority for approving changes from QR onwards (QR, AR, problems

reports and change requests during operation and maintenance)

– comprises • Technical Officer

• Software PA Rep.

• counterparts on supplier’s side

• user representatives.

• Configuration Control Board (CCB)– control authority for authorising changes [proposed/]approved by the SRB

during hand-over to the end user and during execution of operation and maintenance processes (role and responsibility covered by Configuration Management procedure).

92

PR7110 - Procurement

ContractStatement of Work

(SOW)

ContractualTechnical Specification

(CTS)

Service Level Agreement

(SLA)

Requirements Liste.g. SSD from RB

has as appendices

plus or

which contains or refers to

If SOW addresses a subset of the lifecycle, CTS refers to outputs of prior

process

SoftwareMaintenance

SoftwareDevelopment

93

PR7110 – SoW Table of Contents

• Introduction including purpose and scope of work, related documents

• Work to be done

• Schedule, Milestones and Deliveries

• Constraints (platforms, environment, work location (on-site/off-site ESOC), SW re-use, etc..)

• Customer furnished items

• Risk Management

• Reporting

• Acceptance testing

• Software Engineering Aspects including tailoring of standards

• Quality and Safety Assurance Requirements

There is also work going on in the QMS Improvement Group on a generic SoW.

94

Statements of Work

• ‘Classic’ approach: SWRR as start of development– FUP to do requirements analysis under responsibility of TO

• alternatively, TO may write the requirements

– FFP to do rest of development

– FUP to support operations validation (including SVTs, simulations programme) and LEOP

– At ESOC 1st and 2nd line maintenance are typically combined in one contract

• covers software operations engineering and software maintenance

95

PR7110 – Life cycle and requirements

• Life cycle model specified in SOW and confirmed in supplier’s SDP

• Always starts with system requirements, defined in SSD and IRD if applicable

• If part of larger system, software project management agree with overall project management (typically the GSM)

– responsibilities for system requirements definition

– incorporation of requirements into higher-level document (e.g. Flight Dynamics Requirement Compilation)

• Agreement documented in Mission Implementation Plan (MIP)

96

PR7110 – Development

• Procedure outlines the processes– Software requirements analysis resulting in the software requirements

specification and ending with the Software Requirements Review (SWRR)

– Top level architectural design documented in the Design Definition File (DDF) and the Design Justification File (DJF) and reviewed at the Preliminary Design Review (PDR)

– Software design engineering, also documented in the Design Definition File (DDF) and the Design Justification File (DJF), which includes the detailed design and production of code, reviewed at the Critical Design Review (CDR)

– Software validation and acceptance: this includes validation testing against a subset of the SSD at the suppliers premises (Factory Acceptance Testing – FAT), qualification review (QR), delivery, installation and validation against the SSD in the operation environment. It concludes with the Acceptance Review (AR)

• Typical ESOC approach is preliminary Site Acceptance Test (similar to PSS-05 Provisional Acceptance ), followed by check out period (typically 2 months, specified in SoW) followed by (final) Site Acceptance Test

• … and invokes GS SEMG Parts 1, 2 and 3

97

PR7110 – Operations and maintenance

• Procedure outlines the processes– Software Operations Engineering normally starts after the

acceptance review and ends with the software retirement

– Software Maintenance normally begins after the acceptance review

• … and invokes GS SEMG Parts 1, 2 and 3

98

PR7110 – Tailoring criteria (in this order)

• Functional criticality– Level 1 = Safety critical: Risk to human life and Equipment or loss of mission

– Level 2 = Mission critical: Risk of degraded mission

– Level 3 = Performance critical: Performance related parameters such as time of response, numerical accuracy, etc.

– Non critical = none of the above

• Project size– Very small (one independent program, one main function, < 6 man-months or

team <= 2 for <= 1 year or <= 2000 lines of source excluding comments)

– Small (< 2 man-years or team <= 5 people for <= 1 year or <= 10,000 lines of source code excluding comments)

– Normal (not belonging to above categories)

• Operational lifetime– Short (< 1 year and no re-use)

– Long (> 1 year or re-use)

99

PR7110 – Tailoring of V&V activities

Apply only size and lifetime criteriaNon-critical

As SEMGLevel 3 –

Performance critical

As SEMG, but independent team producing independent verification and validation plans and reports

Level 2 –

Mission critical

As SEMG, but independent team producing independent verification and validation plans and reports, plus specific Q80 dependability and safety requirements if relevant

Level 1 –

Safety critical

Verification and validation activitiesFunctional criticality

Safety critical – risk to human life Mission Critical – risk of degraded mission

100

PR7110 – Tailoring: size and operational lifetime

as specified in Tailoring Template: the system requirements process and the software engineering process may be combined into one process.

Very small with short lifetime

As defined in Tailoring Template for such a projectVery small with long lifetime

As defined in Tailoring Template for a small projectSmall

As SEMGNormal

Documentation (size affects management plans, lifetime affects life cycle documentation)

Size / operational lifetime

101

PR7110 – Tailoring: documentation

• The estimated criticality level, size and lifetime of each software sub-system

• The resulting tailoring measures

• The list of documents derived from tailoring

are recorded in the SOW and reflected in the SDP

102

PR7110 – Also covers

• Further ‘Development’ topics– Management and Management Plans

– Reviews

– Acceptance and Release

• Software Verification and Validation

• Problem Reporting

• Configuration Management and Control

• Tools and Methodologies

• Software Metrics

• Supplier Audits

… and these sections refer to SEMG and other procedures and

standards

103

PR7110 – Software Metrics

• Metrics to be defined in SOW and reflected in Software Product Assurance Plan (SPAP)

• Software Project Manager or maintenance leader must collect and periodically analyse, for each process:

– Number of statements of source code, broken down to sub-system level, distinguishing between executable and comment statements

– For object-oriented systems, number of classes and optionally other statistics (e.g. number of methods)

– SPR statistics, including SPR history (trend).• SPR statistics are typically reported monthly in the monthly progress

report (also with graph)

• Additional metrics may be requested (and, again, defined in SOW and reflected in SPAP)

104

Tailoring in SEMG

• E-40 says: select and exceptionally modify or add requirements

• SEMG elaborates: The tailoring process is the deletion of non-applicable processes, activities and tasks. The addition of unique or special processes, activities or tasks is permitted, as specified in the contract. Tailoring may also involve the deletion of outputs or the limitation of applicability to certain parts of the system. The existing requirements may be refined or specified.

• Some criteria to consider:

– overall space project risk

– project/equipment/product characteristics – criticality, longevity, size, operational or non-operational status, real-time constraints, level of definition of the requirements

– cost.

105

Tailoring (ex Part B) – typology (1 of 2)

Category Definition

1 Loss of mission would be unacceptable. The allocated budgets and development schedules shall be sufficient to obviate major technical deadlocks. This category normally does not apply to ground segment software for unmanned missions.

2 A project aimed at achieving overall control of project risks. Project risk/total cost compromises that minimise risk are sought.

3 A project aimed at achieving overall containment of cost. Project risk/total cost compromises that minimise cost are sought. The level of accepted risk is higher.

4 A minimum cost project. The mission is only worthwhile if its cost is kept down.

Project category definitions

106

Tailoring (ex Part B) – typology (2 of 2)

Classification of ground segment components

Type Typical Elements

On-line Operational Mission Control System

Ground Communication Subnet

Ground Station System

Off-line Operational Flight Dynamics

Mission Planning

Spacecraft Performance

Payload Processing

Prototype Simulators

Study Software

107

Group Activity

SCENARIO

• You are in charge of a small, non-critical, project for software with a short operational lifetime and need to decide what documentation is required

TASKS

• Review the tailoring conditions critically and comment on them

• Decide what tailoring you would apply to the project in question. (Fill in the template with the appropriate conditions and actions)

108

Related guideline …

109

Recap

SEMG Tailoring Template

Inputs and outputs for each process

Tailoring condition

Tailoring possibility

Contractual (PR7110)

Statement of Work

Contract Technical Specification

Requirements List

SEMG

Part A: Software Engineering covering ECSS-E-40

Part B: Management covering ECSS-Q-80 and ECSS-M- series

Part C: Document Templates

110

Feedback

• Suggestions for improvement to SEMG and Tailoring Template to BSSC: Mike Jones and/or Uffe Mortensen

• AR end-of-project assessment of lessons learned: make contractual and feed into improvement of SEMG?

Please don’t forget: Course evaluation form

THANK YOU

Date post:	11-Jan-2016
Category:	Documents
Upload:	arline-eaton
View:	214 times
Download:	1 times