Competency Guideline – Systems Engineering€¦ · Competency Guideline – Systems Engineering...

Competency Guideline – Systems Engineering

T MU CY 05000 GU

Guide

Version 1.0

Issue date: 21 September 2018

© State of NSW through Transport for NSW 2018

T MU CY 05000 GU Competency Guideline – Systems Engineering

Version 1.0 Issue date: 21 September 2018

Important message This document is one of a set of standards developed solely and specifically for use on

Transport Assets (as defined in the Asset Standards Authority Charter). It is not suitable for any

other purpose.

The copyright and any other intellectual property in this document will at all times remain the

property of the State of New South Wales (Transport for NSW).

You must not use or adapt this document or rely upon it in any way unless you are providing

products or services to a NSW Government agency and that agency has expressly authorised

you in writing to do so. If this document forms part of a contract with, or is a condition of

approval by a NSW Government agency, use of the document is subject to the terms of the

contract or approval. To be clear, the content of this document is not licensed under any

Creative Commons Licence.

This document may contain third party material. The inclusion of third party material is for

illustrative purposes only and does not represent an endorsement by NSW Government of any

third party product or service.

If you use this document or rely upon it without authorisation under these terms, the State of

New South Wales (including Transport for NSW) and its personnel does not accept any liability

to you or any other person for any loss, damage, costs and expenses that you or anyone else

may suffer or incur from your use and reliance on the content contained in this document. Users

should exercise their own skill and care in the use of the document.

This document may not be current and is uncontrolled when printed or downloaded. Standards

may be accessed from the Transport for NSW website at www.transport.nsw.gov.au

For queries regarding this document, please email the ASA at [email protected] or visit www.transport.nsw.gov.au © State of NSW through Transport for NSW 2018



Standard governance

Owner: Manager Competency Systems, Asset Standards Authority

Authoriser: Director Industry and Technical Development, Asset Standards Authority

Approver: Executive Director, Asset Standards Authority on behalf of the ASA Configuration Control Board

Document history

Version Summary of changes

1.0 First issue

© State of NSW through Transport for NSW 2018 Page 3 of 77



Preface

The Asset Standards Authority (ASA) is a key strategic branch of Transport for NSW (TfNSW).

As the network design and standards authority for NSW Transport Assets, as specified in the

ASA Charter, the ASA identifies, selects, develops, publishes, maintains and controls a suite of

requirements documents on behalf of TfNSW, the asset owner.

The ASA deploys TfNSW requirements for asset and safety assurance by creating and

managing TfNSW's governance models, documents and processes. To achieve this, the ASA

focuses on four primary tasks:

• publishing and managing TfNSW's process and requirements documents including TfNSW

plans, standards, manuals and guides

• deploying TfNSW's Authorised Engineering Organisation (AEO) framework

• continuously improving TfNSW’s Asset Management Framework

• collaborating with the Transport cluster and industry through open engagement

The AEO framework authorises engineering organisations to supply and provide asset related

products and services to TfNSW. It works to assure the safety, quality and fitness for purpose of

those products and services over the asset's whole-of-life. AEOs are expected to demonstrate

how they have applied the requirements of ASA documents, including TfNSW plans, standards

and guides, when delivering assets and related services for TfNSW.

Compliance with ASA requirements by itself is not sufficient to ensure satisfactory outcomes for

NSW Transport Assets. The ASA expects that professional judgement be used by competent

personnel when using ASA requirements to produce those outcomes.

About this document

This guideline specifies the minimum generic competency criteria and evidence criteria for

various systems engineering functions. This guideline has been developed in response to an

identified industry need for assistance in articulating and assessing competency of certain

systems engineering functions undertaken on the TfNSW Transport Network.

The competencies specified in this guideline (specifically, those contained in Section 6) have

been adopted from the International Council on Systems Engineering (INCOSE) Systems

Engineering Competencies Framework. Users of this guideline are made aware that the

copyright for such content is retained by INCOSE UK.

This document is a first issue.




Table of contents 1. Introduction .............................................................................................................................................. 6

2. Purpose .................................................................................................................................................... 6 2.1. Scope ..................................................................................................................................................... 6 2.2. Application ............................................................................................................................................. 7

3. Reference documents ............................................................................................................................. 7

4. Terms and definitions ............................................................................................................................. 7

5. TfNSW competence and compliance..................................................................................................... 8 5.1. Role functions ........................................................................................................................................ 9 5.2. Systems engineering asset life cycle stages ......................................................................................... 9 5.3. Proficiency levels ................................................................................................................................. 11 5.4. Applicability and scalability .................................................................................................................. 12 5.5. Evidence to support competence ........................................................................................................ 12

6. TfNSW systems engineering competencies ....................................................................................... 13 6.1. Holistic life cycle view competencies ................................................................................................... 17 6.2. Systems engineering management competencies .............................................................................. 55 6.3. Systems thinking competencies .......................................................................................................... 70




1. Introduction Authorised Engineering Organisations (AEOs) have a responsibility to ensure that engineering

activities undertaken on behalf of Transport for NSW (TfNSW) are performed by suitably trained

and experienced personnel; that is, personnel designated as being 'competent'. This is required

to meet the legislative obligations and TfNSW requirements.

This guideline has been developed to set generic systems engineering competency criteria for

TfNSW. Refer to T MU CY 01000 GU TfNSW Competency Standards Guidelines and Glossary

for key competency definitions and further guidance.

2. Purpose This guideline establishes the competency pathways and generic competency criteria of

personnel to be recognised as competent to perform systems engineering functions for TfNSW.

This guideline is intended to provide systems engineering practitioners with information on

function-specific minimum generic competency criteria so that practitioners can be assessed

and their competence determined in a consistent manner.

The competency functions and associated evidence criteria presented within Section 6 of this

guideline encompass competency criteria of functions within a role. AEOs should determine the

applicability of each competency function based on the scope of work being carried out and

tailor the criteria accordingly. Refer to T MU AM 06006 ST Systems Engineering standard and

T MU AM 06006 GU Systems Engineering guide for details on scaling and tailoring systems

engineering effort. AEOs may choose to specify more than is documented within this guideline.

The pathways and associated competency standards are provided for those functions where

limited information is available to enable AEOs to develop reasonable assessment criteria. They

are not intended to cover the exhaustive list of functions that exist over the full asset life cycle.

2.1. Scope This guideline is limited to generic competency criteria only and does not cover specific domain

or product knowledge which may be required by the AEO for network specific licensing and

accreditation.

This guideline provides information to an AEO for the development of its auditable competence

management system (CMS). It also provides guidance to enable assessment of personnel who

are required to demonstrate competency.

This guideline is limited to systems engineering and does not cover other areas such as safety

assurance.




2.2. Application This guideline applies to AEOs developing or updating their competence management system.

This guideline may also be used as a reference by any other organisation (including

government agencies or departments, and non-AEO contractors) in the assessment of their

own systems engineering competency.

3. Reference documents The following documents are cited in the text. For dated references, only the cited edition

applies. For undated references, the latest edition of the referenced document applies.

International standards

ISO/IEC/IEEE 15289 System and software engineering – Content of life-cycle information items

(documentation)

Australian standards

AS/NZS ISO/IEC/IEEE 15288 Systems and software engineering – System life cycle processes

Transport for NSW standards

T MU AM 06006 ST Systems Engineering

T MU AM 06006 GU Systems Engineering

T MU CY 01000 GU TfNSW Competency Standards Guidelines and Glossary

T MU CY 10503 GU AEO Guide to Engineering Competence Management

T MU MD 00009 ST AEO Authorisation Requirements

Other reference documents

INCOSE Systems Engineering Competencies Framework

INCOSE Annex A – Guide to Competency Evaluation

4. Terms and definitions The following terms and definitions apply in this document:

ASA Asset Standards Authority

AEO Authorised Engineering Organisation

CMS competence management system

non-functional requirements statements expressing the levels of safety, security, reliability,

and so on that are necessary

RAMS reliability, availability, maintainability and safety




SEMP systems engineering management plan

SFAIRP so far as is reasonably practicable

SRS systems requirements specification; a description of what the system should do, in terms

of the system’s functions, interactions and interfaces with its operational environment. It

communicates the stakeholder requirements to the technical community who will specify and

build the system; alternatively it can be referred to as a system requirements document

subsystem the level below the system of interest

super-system the level above the system of interest

TfNSW Transport for New South Wales

TfNSW Transport Network the transport system (transport services and transport

infrastructure) owned and operated by TfNSW, its operating agencies or private entities upon

which TfNSW has power to exercise its functions as conferred by the Transport Administration

Act or any other Act

5. TfNSW competence and compliance Competency requirements for TfNSW are generic and are considered minimum requirements to

work on the TfNSW Transport Network.

As a part of the AEO accreditation process, an AEO is required to have its own competency

assessment process as described in T MU MD 00009 ST AEO Authorisation Requirements and

T MU CY 10503 GU AEO Guide to Engineering Competence Management. The AEO

competency assessment process should incorporate the competence and compliance factors

as described in the T MU CY 01000 GU.

The selection of systems engineering functions, if any, and the level for each function that is

applicable is at the discretion of an AEO. One or more functions are then selected to form a

role.

When achieved, competencies are intended to be ‘portable’ allowing the individual holding the

competencies to transfer competence evidence between AEO employers and transfer

competence evidence between the projects for the AEO employer.

An AEO may tailor the competency criteria to meet project or industry needs in consultation with

relevant stakeholders. The level and depth of the criteria can depend on industry requirements

or those required by standards.

Refer to T MU CY 01000 GU for more information on competence, compliance and licensing.




5.1. Role functions This guideline specifies a range of generic functions; that is, the functions that a jobholder might

be expected to perform. The function is a speciality area of a role comprised of a collection of

tasks. Refer to T MU AM 06006 ST and T MU AM 06006 GU for details on defining system

engineering roles.

The AEO should select the functions required to perform the role and the minimum competency

levels for each function.

For example, the role of a systems engineer might involve a range of different functions such as

determine and manage stakeholder requirements, functional analysis, architectural design,

validation, transition to operation, planning, monitoring and controlling and system concepts.

See Figure 1.

Figure 1 - Functions within a role

5.2. Systems engineering asset life cycle stages The functions reside within the whole of the TfNSW asset life cycle model. The functions may

also reside within the ‘operate / maintain’ and 'dispose' stages depending on whether the asset

requires a mid-life functional or performance upgrade that differs from the original system

specification. Figure 2 shows the alignment of the systems engineering functions within the

TfNSW asset life cycle model. Refer to Section 6 for details of each of these systems

engineering functions.




Figure 2 - Systems engineering system life cycle model

AcceptNeed Concept Specify Procure Design Build Integrate Operate and Maintain Dispose

Concept Development Production Utilisation and Support Retirement

Plan Acquire Operate/Maintain Dispose

Exploratory

Evolve

Demand/Need

Determining and Managing Stakeholder Requirements

Systems Integration & Verification

Interface Management

Maintain Design Integrity

Architectural Design

Functional Analysis

Concept Generation

Design for ...

System modelling and Simulation

Validation

Select Preferred Solution

System Integrity

Transition

Concurrent Engineering

Enterprise IntegrationIntegration of Specialisms

Lifecycle Process DefinitionPlanning, Monitoring and Controlling

System Concepts

Super System Capability IssuesEnterprise and Technology Environment

Holistic life cycle

view

Systems engineering

management

System thinking




5.3. Proficiency levels T MU CY 10503 GU specifies that an AEO’s CMS defines the proficiency levels relevant and

current for any particular function. Whilst the AEO is responsible for setting a proficiency level

framework, the proficiency levels shown in Table 1 are suggested as the most appropriate to be

used with this guideline. Each of these proficiency levels also satisfies the lower level

proficiencies.

Note: It is quite acceptable and normal for a systems engineering practitioner to have

proficiency of Level 0 – Awareness or Level 1 – Supervised practitioner for those

functions that they do not practice. For example, a systems integration engineer may

have Level 0 - Awareness of modelling and simulation.

Table 1 - Proficiency levels

Proficiency Description

Level 0 – Awareness The awareness level is relevant for roles requiring a basic understanding within the organisation or within the relevant industry of the tasks associated with the overall systems engineering function. The person would not work on the tasks associated with the overall function.

Level 1 – Supervised practitioner

A supervised practitioner has sufficient knowledge and basic understanding of best practice, within the organisation or within the relevant industry, to be able to work on the tasks associated with the overall function without placing an excessive burden on the practitioner or lead practitioner that may compromise safety. A practitioner or lead practitioner checks a supervised practitioner's work. Supervised practitioners may not have previous experience of working on a complex project. Their competencies may have been developed through targeted training and work on non-related projects. An assessor may need to extrapolate from evidence of technical skills derived from another project environment to determine competence and the level of supervision.

Level 2 – Practitioner A practitioner has sufficient knowledge and detailed understanding of best practice, and sufficient demonstrated experience to be able to work on the tasks associated with the overall function with supervision by a peer or lead practitioner less than 10% of the practitioner time. A practitioner will maintain their knowledge and be aware of all current developments in their field of work. The practitioner may be required to perform detailed checks on the work carried out by a supervised practitioner.




Proficiency Description

Level 3 – Lead practitioner A lead practitioner will have an authoritative understanding of why things are done in certain ways, and sufficient demonstrated managerial skills, to be able to undertake overall responsibility for the performance of a function. A lead practitioner will be familiar with the ways in which systems, and previous systems or projects, have failed in the past. A lead practitioner will keep abreast of technologies, architectures, application solutions, standards, and regulatory requirements, particularly in evolving fields. A lead practitioner will have sufficient breadth of experience, knowledge and deep understanding to be able to work in novel complex situations. A lead practitioner will be able to deal with multiple problems under pressure without compromising safety.

5.4. Applicability and scalability Section 6 includes the competency criteria for the function groups. They are presented in a

tabular form showing functions, proficiency level, evidence criteria and evidence. The order in

which the functions and competency items are presented is not intended to imply relative

importance to each other.

Assessors should determine the applicable competency functions based on the scope of work

being carried.

Assessors should determine the applicable proficiency levels for each competency function

based on the systems engineering role and tailor the criteria accordingly.

To meet the requirements of the applicable competency pathway, candidates should meet all

the applicable functions and provide acceptable evidence criteria for each item within an

individual pathway.

Assessors should seek scalable evidence that reflects the range and complexity of the work

being undertaken to ensure the candidate has the required breadth of knowledge and the

experience and expertise to enable them to conduct a function in a competent manner.

5.5. Evidence to support competence When assessing the competency of a candidate, it is important that the candidate presents

specific objective verifiable evidence to support their claims of competency. For example,

documents should have been reviewed or approved to be acceptable. The assessor has the

flexibility to interpret the presented evidence against the required evidence. Typically, such

evidence is shown in the practitioner level examples provided in Table 2.




Table 2 - Objective evidence of competency

Functions Evidence criteria Generic evidences

Examples of evidence criteria

Determining and managing stakeholder requirements

Elicits and validates stakeholder requirements

E01, E02, E06, E20, E21

Stakeholder map Independently assessed requirements specification Requirements validation analysis

System concepts Identifies and manages complexity with appropriate techniques in order to reduce risk

E01, E02, E06, E17, E19, E20, E21, E22, E23, E34

System studies tackling the issues of complexity and recommending suitable approaches

6. TfNSW systems engineering competencies The competency criteria provided in this section are separated into the following key systems

engineering function groups:

• group 1, holistic life cycle view (refer to Section 6.1)

• group 2, systems engineering management (refer to Section 6.2)

• group 3, systems thinking (refer to Section 6.3)

The requirements for these functions have been adopted from INCOSE Systems Engineering

Competencies Framework. Refer to the INCOSE Systems Engineering Competencies

Framework for a detailed explanation of different function groups.

An assessment of competency requires evidence demonstrating that a candidate has the

breadth of knowledge, experience and expertise to enable them to carry out functions in a

competent manner.

The contents in this section may also be used as a guide for evidence gathering by prospective

AEOs undertaking the AEO authorisation process.

The evidence criteria for the proficiency level for a function also include the evidence criteria for

lower proficiency levels for the same function. For example, to reach proficiency level

practitioner for a function all the evidence criteria for the proficiency level awareness and

proficiency level supervised practitioner also apply for the function.

Table 3 describes the typical evidence that is applicable for each type of evidence. These types

of evidence and the corresponding typical evidence are based on ISO/IEC/IEEE 15289

Systems and software engineering – content of life-cycle information items (documentation).




Table 3 - Typical evidence

Type of evidence reference Type of evidence Typical evidence

E01 Understanding Training Assessments Workshops records Minutes of meetings Team structures Log book Certificates Qualifications Recommendations Correspondences Curriculum vitae (CV)

E02 Understanding Interview results Consultation with references

E03 Acquisition Request for information Request for proposal Supplier selection reports Supplier assessment reports Change request Acceptance report

E04 Supply Proposal Contract Change request

E05 Life cycle model management Life cycle policy and procedures Improvement plan Audit plan Process assessment procedure Process improvement report

E06 Infrastructure management System requirement specification System element description Change request

E07 Portfolio management Project management plan Progress initiation report Progress report Project closure report Evaluation report

E08 Human resource planning Training plan Training documentation Evaluation report Competency framework





E09 Quality management Quality management plan Quality management policy and procedures Monitor and control report Audit reports

E10 Knowledge management Knowledge management plan Training documentation

E11 Project planning SEMP Acceptance plan Acquisition plan Resource planning

E12 Project assessment and control

Project management plan Progress report Monitor and control report Change requests Reviews

E13 Decision management Problem reports Reports Change management plan

E14 Risk management Risk management plan Risk action request Risk register Monitoring and control report

E15 Configuration management Configuration management plan Configuration management procedure Change request Configuration status report Configuration evaluation report

E16 Information management Information management plan Configuration status report

E17 Measurement Monitoring and control report Measurement procedures Evaluation report Reviews Report

E18 Quality assurance Quality management procedures Evaluation report

E19 Business or mission analysis Concept of operations





E20 Stakeholder needs and requirement definitions

Concept of operations Business requirement specifications (BRS) System requirement specifications (SRS) Product need assessment

E21 System requirement definitions

SRS System architecture description

E22 Architecture definition System architecture description Interface description Evaluation report

E23 Design definition Interface description System element description Evaluation report

E24 Implementation Implementation procedures User documentation Integration and test Report

E25 Integration User documentation Integration and test report Problem report

E26 Verification Verification procedure Integration and test report Problem report

E27 Transition Release plan Installation report Problem report

E28 Validation Validation procedure Validation report Problem report

E29 Operation User documentation Information security procedure Monitoring and control report Customer satisfaction survey Problem report Operations concept

E30 Maintenance Maintenance plan Maintenance procedure Change request Service report Problem report Maintenance concept





E31 Disposal Configuration status report

E32 Tailoring Life cycle procedure

E33 Enterprise definitions Enterprise modelling Enterprise architectures Enterprise frameworks Supporting evidences, such as emails, minutes

E34 Safety Process hazard analysis (PHA) Safety requirement specifications Safety assurance statement or argument

E35 Thought leadership Journal articles Peer reviewing position papers Development of novel concepts Validation and verification of concepts Presentations to industry

6.1. Holistic life cycle view competencies The minimum competence requirements and minimum evidence requirements for holistic life

cycle view competencies are explained in Section 6.1.1 through to Section 6.1.13.

6.1.1. Determining and managing stakeholder requirements The evidence required for the function of determining and managing stakeholder requirements

is explained in Table 4 through to Table 7. The purpose of this function is to analyse

stakeholder needs and expectations to establish and manage the requirements for a system.

Table 4 - Evidence criteria for determining and managing stakeholder requirements – Proficiency level: Awareness

Evidence criteria Generic evidences


Awareness of who are stakeholders

E01, E02 Aware of who are applicable and authorised stakeholders and their respective roles: • business • customers • operators • maintainers • community (commercial, industrial,

local government, residential) • suppliers (including utilities services)




Evidence criteria Generic

evidences Examples of evidence criteria

Awareness of stakeholder needs E01, E02 Aware of what stakeholder needs are versus business requirements

Awareness of determining stakeholder requirements

E01, E02 Aware of process of determining business requirements that satisfy stakeholder needs

Awareness of different types of stakeholder documents

E01, E02 Aware of the purpose of the stakeholder documents: • business case • operations concept definition (OCD) • maintenance concept definition (MCD) • BRS • SRS

Table 5 - Evidence criteria for determining and managing stakeholder requirements – Proficiency level - Supervised practitioner



Identifies all the relevant and authorised stakeholders

E01, E02, E20

Can develop stakeholder map Can develop stakeholder matrix

Supports the elicitation of requirements from stakeholders

E01, E02, E20, E21

Support of meetings, use cases, scenarios, development of questionnaires Identification of gaps, questions, traceability, coverage, constraints

Understands the characteristics of good quality stakeholder requirements

E01, E02, E20, E21

Understands good requirements are verifiable, unambiguous, complete, concise and consistent

Understands methods used in stakeholder requirements gathering

E01, E02, E20, E21

Understands elicitation methods, interviews, workshops, brainstorm, seminar, prototyping, demonstrations, standards Understands bias and sampling Understands to check for completeness and follow up on an incomplete set of requirements

Understands the need for traceability in the requirements definition and allocation process

E01, E02, E20, E21

Can develop requirements analysis, allocation and traceability matrix (RATM) Can develop requirements management tool

Understands the relationship between requirements and acceptance against those requirements

E01, E02, E20, E21, E26, E28

Can develop requirements verification and traceability matrix (RVTM) Can develop requirements management tool

Able to establish acceptance criteria for simple requirements

E01, E02, E20, E21, E26, E28

Can develop RVTM Can develop requirements management tool






Understands the relationship between design and requirements

E01, E02, E06, E20, E21, E22, E23

Understands that requirements specify what is required Understands that design defines how the set of requirements may be implemented

Table 6 - Evidence criteria for determining and managing stakeholder requirements function – Proficiency level: Practitioner



Elicits and validates stakeholder requirements

E01, E02, E06, E20, E21

Can develop stakeholder map Can develop independently assessed requirements specification Can perform requirements validation analysis

Writes good quality, consistent requirements

E06, E20, E21

Can develop independently assessed requirements specification

Derives requirements from analysis of the super-system design

E06, E20, E21, E22, E23

Understands transition from user requirements to system requirements Can develop architectural models

Establishes acceptance criteria for requirements for the system of interest

E20, E21, E26, E28

Can develop applicable and feasible acceptance criteria

Resolves and negotiates requirement conflicts in order to establish a complete and consistent requirement set for the system of interest

E01, E20, E21, E22, E23

Can develop requirements trade study Can develop minutes of meetings, for example minutes from a design review meeting

Identifies areas of uncertainty and risk when determining requirements

E06, E14, E20, E21, E22, E23, E34

Can develop risk register Can develop assumption analysis Can develop dependencies Can develop constraints

Challenges appropriateness of requirements in a rational way

E01, E06, E20, E21

Minutes of stakeholder engagement or requirements definition meeting

Defines and documents an approach for requirements elicitation and management

E11, E18 Can develop requirements management plan

Assesses the impact of changes to requirements on the solution and program

E06, E11, E13, E15, E18, E20, E21

Impact and traceability analysis, including judgement of significance analysis Can develop change requests






Guides supervised practitioner E08, E10 Examples of on the job training objectives, guidance, and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in determining and managing stakeholder requirements Evidence of assignment as a mentor

Table 7 - Evidence criteria for determining and managing stakeholder requirements – Proficiency level: Lead practitioner



Acknowledged as an authority in the elicitation and management of requirements

E01, E35 Facilitation or establishment of guidelines for facilitation of requirements elicitation workshops

Reviews and judges the suitability of the approach to requirements elicitation and management

E11, E20, E21

Can develop approved requirements management plan Can develop requirements review comments

Reviews and judges the suitability and completeness of the requirements set

E01, E20, E21

Can develop requirements review comments Can develop requirements analysis

Advises on the sensitive requirements negotiations on major program

E01, E20, E21

Minutes of stakeholder requirements elicitation or scoping meetings Establish and lead or participate in requirements communities of interest Stakeholder approval of requirements

Coaches other requirements practitioners in this field

E08, E10 Examples of coaching activities and the outcome of the process Conducted formal training courses and authored training material supported by successful post-training evaluation data Listed as an approved trainer in the organisation






Champions the introduction of novel techniques and ideas in this field which produced measurable improvements

E35 Documented examples of the introduction of novel techniques of determining and managing stakeholder requirements and can provide evidence of the improvement made Published papers in refereed journals or company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books Authored details of improvements to process and appraisal against a recognised process improvement model

6.1.2. System design - system architectural design

The evidence required for the function of systems design - architectural design is explained in

Table 8 through to Table 11. The purpose of this function is to define the system architecture

and its derived requirements to produce a solution that can be implemented to enable a

balanced and optimum result that considers all stakeholder requirements.

Table 8 - Evidence criteria for systems design - architectural design function – Proficiency level: Awareness



Aware of the principles of system architectural design and its role within the life cycle

E01, E02 Describes the boundary of a system, identifies major interfaces to the system and allows functional analysis Can associate the role of architectural design within the overall system life cycle Can describe the importance of architectural design (for example, common vehicle for communication between stakeholders, allows quality attributes such as performance to be modelled, and so on) and understands the criteria for good design Can describe architecture in terms of a decomposition of a system into its components, their interrelationships and the constraints that apply

Aware of the different types of system architecture

E01, E02 Recognises that there is not a single approach that fits in all instances of system architectural design Can abstract a system into a structured representation, such as functional or physical breakdown Types of system architecture may include physical, logical, operational and so on






Aware that system architectural decisions can constrain and limit future system use and evolution

E01, E02 Aware of system limitations and constraints

Table 9 - Evidence criteria for systems design - architectural design function – Proficiency level: Supervised practitioner



Uses techniques to support system architectural design process

E01, E02, E11, E22, E23

Has contributed to developing system architectures as part of the concept phase of the system engineering life cycle Has experience of using a set of architectural design principles

Supports the system architectural design trade-offs

E01, E02, E11, E22, E23

Has participated in a system architecture design review that has considered system design trade-offs

Contributes to alternative system architectural designs that are traceable to the requirements

E01, E02, E11, E22, E23

Can provide examples of a system architectural design (conceptual, functional, logical and physical) to which they have contributed and can discuss the merits of the system design chosen

Interprets a system architectural design

E01, E02, E11, E22, E23

Has contributed to review of a system architectural design

Understands that the system architectural design best ensures safety so far as is reasonably practicable (SFAIRP)

E01, E02, E22, E23, E14, E34

Has contributed to system architectural designs that best ensures safety SFAIRP

Table 10 - Evidence criteria for systems design - architectural design function – Proficiency level: Practitioner



Generates alternative system architectural designs that are traceable to the business and system requirements

E01, E11, E20, E21, E22, E23

System architectural design (conceptual, functional, logical and physical) which they have produced and can discuss the merits of the design chosen

Assesses a range of system architectural designs options and justify the selection of the optimum solution

E19, E21, E22

Trade study showing alternatives and the solution selected system architectural design

Defines a process and appropriate tools and techniques for system architectural design

E11 Authored system architectural process definition and tool selection in a document such as systems engineering management plan (SEMP), other project or program plan or organisational process






Chooses appropriate system analysis and selection techniques

E01, E11 Documented examples of using system analysis and selection techniques such as: • cost-benefit analysis • user panels • multi-criteria decision analysis • convergence criteria • minutes of meetings, reports, design

documents

Partitions between discipline technologies and derives discipline-specific system or subsystem requirements

E06, E20, E21, E22, E23

Documented examples of partitioning

Specifies system architectural designs that best ensure safety SFAIRP

E14, E19, E21, E22, E34

Documented examples of system architectural designs that best ensures safety SFAIRP

Guides supervised practitioner E08, E10 Examples of on the job training objectives and guidance. Organisational breakdown structure for system development of project or program showing responsibility for managing those involved in architectural design Evidence of assignment as a mentor

Table 11 - Evidence criteria for systems design - architectural design function – Proficiency level: Lead practitioner



Can demonstrate a full understanding of system architectural design techniques and their appropriateness, given the levels of complexity of the system of interest

E11, E20, E21, E22, E23

Documented use of system architectural design techniques such as: • solution abstraction • clustering • interface minimisation • layering

Reviews and judges the suitability of system architecture designs

E19, E21, E22, E23

Can provide records of a review process in which they have been involved Can provide evidence of a system architectural design on which they have provided advice, can summarise the advice given and the resulting changes made Can provide evidence of system architectural design approvals in which they have been involved






Has coached new practitioners in this field

E08, E10 Can provide examples of the coaching activities and the outcome of the process Conducted formal training courses and authored training material supported by successful post-training evaluation data Listed as an approved trainer in the organisation

Has championed the introduction of novel techniques and ideas in this field which produced measurable improvements

E35 Documented examples of the introduction of novel architectural design techniques and tools and can provide evidence of the improvement made Published papers in refereed journals or company literature Evidence of development or introduction of novel facility supporting systems engineering technique (for example, simulated environment, concurrent design facility) Published articles or books, and so on Authored details of improvements to process and appraisal against a recognised process improvement model

Has contributed to and leads best practice in this field

E35 Published papers in refereed journals or company literature Published articles or books, and so on Ideas assimilated into international standards

6.1.3. Systems design – concept generation The evidence required for the function of systems design – concept generation function is

explained in Table 12 through to Table 15. This function involves the generation of potential

system concepts that meets a set of needs and demonstrates that one or more credible,

feasible options exist.




Table 12 - Evidence criteria for system design – concept generation function –

Proficiency level: Awareness



Aware of the need to explore alternative and innovative ways of satisfying the need

E01, E02 Aware that: • first idea is not always the best

alternatives for different needs • an 80% solution might be sufficient if

the extra 20% costs the majority of the customer budget

• not to rely on adaptation of existing solutions

• cognitive bias or decision traps need to be avoided

• use of creative thinking techniques or formal design methodologies that aid in exploring solution space

Awareness that alternative discipline technologies can be used to satisfy the same requirement

E01, E02 Aware that different technologies might do the same thing but in a different way Aware of use of different technologies as an example, such as software vs. hardware, radio-based train control systems vs. fixed colour-light signalling

Table 13 - Evidence criteria for system design – concept generation function – Proficiency level: Supervised practitioner



Can contribute candidate concepts

E01, E02, E11, E22, E23

Evidence of using creativity techniques to generate concepts

Can support assessment of the feasibility of alternative concepts

E01, E02, E11, E22, E23

Participated in feasibility studies, trade studies

Understands that concept generation best ensures safety SFAIRP

E01, E02, E11, E14, E22, E23, E34

Has contributed to concept generation that best ensures safety SFAIRP

Table 14 - Evidence criteria for system design – concept generation function – Proficiency level: Practitioner



Understands the strengths and weaknesses of relevant technologies in the context of satisfying the requirement

E01, E11, E22, E23

Written reports or papers drawing conclusions of trade studies, feasibility analysis






Creates and is open to a range of alternative and innovative interdisciplinary and technology concepts

E01, E11, E22, E23

Reports or minutes of brainstorming sessions Identified new technologies

Converges to a number of possible alternative options and demonstrates that credible, feasible options exist

E01, E11, E22, E23

Trade study reports or conclusions of cost-benefit or effectiveness analysis

Creates concepts that best ensure safety SFAIRP

E14, E22, E23, E34

Documented examples of concept generation that best ensures safety SFAIRP

Guides supervised practitioner E08, E10 Examples of on the job training objectives, guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in concept generation Evidence of assignment as a mentor

Table 15 - Evidence criteria for system design – concept generation function – Proficiency level: Lead practitioner



Guides and advises practitioners in techniques for concept generation

E22, E23 Developed system concept document

Reviews down selected concepts for credibility, feasibility and so on

E01, E22, E23

Developed system concept review comments

Coaches other practitioners in this field

E08, E10 Provides examples of the coaching activities and the outcome of the process Conducted formal training courses and authored training material supported by successful post-training evaluation data Listed as an approved trainer in the organisation







E35 Documented examples of the introduction of novel concept generation techniques and can provide evidence of the improvement made Published papers in refereed journals or company literature Evidence of development or introduction of novel facility supporting systems engineering technique (for example, simulated environment, concurrent design facility) Published articles or books, and so on Authored details of improvements to process and appraisal against a recognised process improvement model

Contributes to and leads best practice in this field

E35 Published papers in refereed journals or company literature Published articles or books and so on Ideas assimilated into international standards

6.1.4. Designed for all requirements The evidence required for the function of 'design for …' is explained in Table 16 through to

Table 19. This function involves ensuring the requirements of all stages of life cycle are

addressed at the correct point in the system design. During the design process, consideration

should be given to the design attributes such as manufacturability, testability, reliability,

maintainability, safety, security, flexibility, interoperability, capability growth, disposal, cost and

natural variations.

Table 16 - Evidence criteria for the design for function – Proficiency level: Awareness



Aware of the need to design for the requirements of all life cycle stages

E01, E02 Identifies ‘design for…’ attributes of a system within their domain Identifies from later parts of the life cycle those activities for which ‘design for…’ expertise would be beneficial during the design phase Can talk about the advantages of the left-shifted approach of considering such design attributes early on to mitigate against increased costs further downstream to account for the requirements associated with these attributes Aware of the importance of the whole-of-life cycle cost Aware of the need for design trade-offs




Table 17 - Evidence criteria for the design for function – Proficiency level: Supervised practitioner



Can describe the 'design for…' attributes and how they influence the design

E01, E02, E06, E20, E21, E22, E23

Participated in workshops for developing ‘design for…’ design attributes within a system development

Supports the identification and balancing of these 'design for…' attributes throughout the design process

E01, E02, E06, E20, E21, E22, E23

Been involved in a design when these 'design for…' attributes have been taken into account involvement in peer review of designs

Understands the design attribute of safe SFAIRP

E01, E02, E11, E14, E20, E23, E25, E34

Has contributed to a design that is safe SFAIRP

Table 18 - Evidence criteria for the design for function – Proficiency level: Practitioner



Identifies and balances the 'design for…' attributes throughout the design process

E11, E20, E21, E22, E23

Developed relevant section of systems engineering management plan or systems requirement document Developed relevant section of SEMP or other project or program plans System design notes and reports System design decision logs

Works with appropriate 'design for…' specialists to ensure that the design effectively addresses the attributes at the correct time

E06, E20, E21, E22, E23

Document or model showing abstraction of system in terms needed by 'design for…' specialists Requirements document showing appropriate translation of specialists requirements into system requirements

Works to ensure that the design is safe SFAIRP

E11, E14, E20, E21, E22, E23, E34

Documented examples of design that is safe SFAIRP

Guides supervised practitioner E08, E10 Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in 'design for....' Evidence of assignment as a mentor




Table 19 - Evidence criteria for the design for function – Proficiency level: Lead

practitioner



Reviews and judges the suitability of plans for the incorporation of all life cycle design attributes at the correct point within the design process

E06, E11, E21, E22, E23

Terms of reference for, and evidence of membership of, an oversight committee

Advises on complex issues and resolve conflicting design requirements

E35 Authored 'design for…' report (or equivalent) of such a formal study


E08, E10 Can provide examples of coaching activities and the outcome of the process Conducted formal training courses and authored training material supported by successful post-training evaluation data Listed as an approved trainer in the organisation

Champions the introduction of novel techniques, tools and ideas in this field which produced measurable improvements

E35 Documented examples of the introduction of novel 'design for...' techniques and can provide evidence of the improvement made Published papers in refereed journals or company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model


E35 Published papers in refereed journals or company literature Published articles or books Ideas assimilated into international standards

6.1.5. Functional analysis The evidence required for the function of functional analysis is described in Table 20 through to

Table 23. Functional analysis is used to determine the functions that are required by the system

to meet the requirements. It consists of the decomposition of higher-level functions to lower-

levels and the traceable allocation of requirements to those functions.




Table 20 - Evidence criteria for the function analysis function– Proficiency level:

Awareness



Aware of what functional analysis is

E01, E02 Aware of: • 'what' the system has to do, not 'how' it

does • functional vs. non-functional

requirements

Aware of the need for functional models

E01, E02 Aware of the need to develop functional architecture Aware of the need to establish the system boundary Aware that functional models take many forms such as behaviour diagrams, context diagrams, control flow diagrams, data flow diagrams, data dictionaries

Aware of the relevance of the outputs from functional analysis and how these relate to the overall system design

E01, E02 Aware of functional analysis outputs; context diagrams, detailed specifications, functional hierarchy diagram, functional matrix (N2 diagram), functional flow block diagram and so on An awareness that functional analysis identifies missing functional requirements and develops derived requirements Realises that functional analysis helps identify poorly written or unrealistic requirements

Table 21 - Evidence criteria for the function analysis function – Proficiency level: Supervised practitioner



Uses appropriate tools and techniques to conduct functional analysis

E01, E02, E19, E20, E21, E22, E23

Using appropriate tools and techniques such as timeline analysis, N2 and so on

Contributes to functional analysis activities

E01, E02, E19, E20, E21, E22

Examples of functional analysis models and diagrams

Table 22 - Evidence criteria for the function analysis function – Proficiency level: Practitioner



Defines the strategy and approach to be adopted for the functional analysis of the system

E11, E19, E20, E21, E22, E23

Authored project or program-level system functional analysis plan






Performs functional analysis E22, E23 Functional model elements, relationships and flows

Defines a process and selects appropriate tools and techniques for functional analysis

E05, E11 List of approved tools Authored documents defining process

Guides supervised practitioner E08, E10 Examples of on the job training objectives and guidance Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in functional analysis Evidence of assignment as a mentor

Table 23 - Evidence criteria for the function analysis function – Proficiency level: Lead practitioner



Demonstrates a full understanding of the techniques and their appropriateness, given the levels of complexity of the system of interest

E05, E11 Authored project or program plan or document

Reviews and judges the suitability of functional analyses

E01, E11, E19, E20, E21, E22, E23

Minutes of reviews Policy documents developed




E35 Documented examples of the introduction of novel functional analysis techniques and can provide evidence of the improvement made Published papers in refereed journals or company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books Authored details of improvements to process and appraisal against a recognised process improvement model







E35 Published papers in refereed journals company literature Published articles or books Ideas assimilated into international standards

6.1.6. Technical interface management The evidence required for the function of technical interface management is explained in

Table 24 to Table 27. Interfaces occur where system elements interact; for example, human,

mechanical, electrical, thermal, data, and so on. Technical Interface management comprises

the identification, definition and control of interactions across system or system element

boundaries.

Table 24 - Evidence criteria for the technical interface management function – Proficiency level: Awareness



Aware of the need for technical interface management and its impact on the integrity of the system solution

E01, E02 Can describe what a technical or system interface is Can describe interface stakeholders Can describe the reason why management of technical interfaces is necessary Can describe the importance of technical interface ownership Can describe the potential impact on the system of failure to manage interfaces Can describe the importance of configuration management when managing technical interfaces

Aware of the possible sources of complexity in technical interface management; for example, multinational programs, multiple suppliers, different domains, novel technology and so on

E01, E02 Can describe different types of technical interfaces across different domains (messages, electrical connections, mechanical, environmental and so on) Can describe possible sources of system and technical interface complexity

Table 25 - Evidence criteria for the technical interface management function – Proficiency level: Supervised practitioner



Follows technical interface management procedures

E01, E02, E06, E11, E22, E23, E24, E25

Experience of using and following technical interface management procedures






Identifies and defines simple technical interfaces

E01, E02, E06, E22, E23, E24, E25

Participated in the identification and definition of simple technical interfaces

Table 26 - Evidence criteria for the technical interface management function – Proficiency level: Practitioner



Defines a process and appropriate techniques to be adopted for the technical interface management of system elements

E06, E11, E22, E23, E25

Examples of process and appropriate techniques adopted for the technical interface management of system elements

Identifies, defines and controls system element technical interfaces

E06, E20, E21 E22 E23, E24, E25

Examples of identification, definition and control of system element technical interfaces

Describes the sources of complexity for the technical interface management of the system; for example, multinational programs, multiple suppliers, different domains, novel technology and so on

E01, E02, E03, E04, E11, E13, E20, E21, E22, E23, E24, E25

Examples of identification of the sources of complexity for the technical interface management of the system, for example, multinational programs, multiple suppliers, different domains, novel technology, and so on

Liaises and arbitrates where there are conflicts in the definition of technical interfaces

E06, E19, E20, E21, E22, E23

Evidence of liaison and arbitration where there have been conflicts in the definition of technical interfaces

Identifies consequences of changes to technical interfaces on the system elements, system, system of systems, or both, such as a change to a mechanical interface may impact thermal performance

E06, E09, E11, E13, E15, E16, E19, E20, E21, E22, E23, E25

Change notes to technical interface descriptions

Guides supervised practitioner in this field

E08, E10 Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in interface management Evidence of assignment as a mentor




Table 27 - Evidence criteria for the technical interface management function – Proficiency

level: Lead practitioner



Demonstrates expertise in technical interface management

E11, E19, E20, E21, E22, E23, E25, E26

Documented use of technical interface management techniques such as: • service level agreements • interface control documents (ICD)

system level/configuration item level • interface development plans • information repositories • interface control drawings/models • interface emulators • approval, revision, archiving • ICD plan

Reviews and judges the suitability of technical interface management strategies

E06, E11, E19, E20, E21, E22, E23, E25

Provides records of a technical interface review process in which they have been involved Provides evidence of a technical interface management strategy on which they have provided advice, can summarise the advice given and the resulting changes made

Negotiates on the issues of technical interface complexity

E06, E20, E21, E22, E23, E24, E25

Provides records of a technical interface negotiation process in which they have been involved




E35 Documented examples of the introduction of novel interface management techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model






Contributed to and leads best practice in this field

E35 Published papers in refereed journals/company literature Published articles or books and so on Ideas assimilated into international standards

6.1.7. Maintaining design integrity The evidence required for the function of maintaining design integrity is explained in Table 28

through to Table 31. This function ensures that the overall coherence and cohesion of the

'evolving' design of a system is maintained in a verifiable manner, throughout the life cycle, and

retains the original intent.

Table 28 - Evidence criteria for maintaining design integrity function – Proficiency level: Awareness



Aware of the need to maintain the integrity of the system design

E01, E02 Aware that design integrity: • assists robustness of the design • reduces risk or uncertainty of the

design at acceptance • reduces ambiguity in the design • can reduce unexpected system

behaviours • can reduce unexpected system design

features • can give early indication of future

system development problems • can identify system design variance or

inconsistency early • can describe system design margins

Table 29 - Evidence criteria for maintaining design integrity function – Proficiency level: Supervised practitioner



Tracks specific aspects of the system design to the original intent

E01, E02, E06, E19, E20, E21, E22, E23, E24, E25, E26, E27, E28

Can develop RATM Can develop RVTM






Supports remedial actions and change control

E01, E02, E11, E12, E13, E15, E18, E19, E20, E23, E24, E25, E26, E27, E28

Provides examples of changes and remedial actions on a recent project or program or activity

Understands the process of change control and configuration management


Provides examples on a recent project or program or activity

Table 30 - Evidence criteria for maintaining design integrity function – Proficiency level: Practitioner



Identifies parameters to track critical aspects of the design

E09, E11, E13, E15, E18, E19, E20, E23, E24, E25, E26, E27, E28

Periodic project or program reviews with parameter tracking SEMP outlining metrics to be tracked

Relates the current design to the original intent throughout the supply chain

E01, E03, E04, E11, E12, E13, E18, E19, E20, E21, E22, E23, E24, E25, E26, E27, E28

Design review minutes Parameter budget allocation tables with margin

Takes remedial actions in the presence of inconsistencies in design

E13, E19, E21, E22, E23

Updated plans, parameter budgets

Establishes a system which allows the tracking of specific aspects of the design

E09, E11, E13, E15, E16, E18, E19, E20, E23, E24, E25, E26, E27, E28

Information management process, specifically for system design






Manages and trades technical margins both horizontally across levels and vertically through levels of the system hierarchy


Can develop collaborative agreements Parameter budget allocation tables with margin


E08, E10 Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in maintaining design Integrity Evidence of assignment as a mentor

Table 31 - Evidence criteria for the maintaining design integrity function – Proficiency level: Lead practitioner



Reviews and judges the suitability of the complete set of critical design parameters that allows the tracking of the system design

E09, E11, E13, E15, E16, E18, E19, E20, E23, E24, E25, E26, E27, E28

Policies for tracking of the system design Set of measures for tracking of the system design

Influences system design trade-offs


System trade studies Minutes of design trade-off meetings Can develop design reviews Can develop design documentation

Advises on the allocation of technical margins to designs

E09, E11, E13, E15, E16, E18, E19, E20, E23, E25, E26, E27, E28

Can develop SEMPs Minutes of design review meetings Can develop design technical reports Can develop technical budget history

Coaches practitioners in this field E08, E10 Can provide examples of the coaching activities and the outcome of the process Conducted formal training courses and authored training material supported by successful post-training evaluation data Listed as an approved trainer in the organisation







E35 Documented examples of the introduction of novel techniques, maintaining design integrity techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model



6.1.8. System modelling and simulation The evidence required for the function of system modelling and simulation is explained in

Table 32 through to Table 35. System modelling is a physical, mathematical, or logical

representation of a system entity, phenomenon, or process. Simulation is the implementation of

a model over time. A simulation brings a model to life and shows how a particular object or

phenomenon will affect the system function and performance.

Table 32 - Evidence criteria for modelling and simulation function – Proficiency level: Awareness



Aware of the need for models as system representations

E01, E02 Aware that system modelling and simulation: • allows early understanding of the

system function and performance • facilitates understanding of system

complexity and cost of implementation • the need to perform trials and 'what ifs' • virtual systems and demonstrators • interactions, interfaces, boundaries and

flow diagrams






Aware of the scope and limitations of system models and simulations, including definition, implementation and analysis

E01, E02 Aware that: • there are different types of system

models • they are abstractions of the real-world

solution • models and simulations contain

assumptions and approximations (garbage in, garbage out)

• real-time and iterative simulations • models can be hierarchical • models and simulations need to be

validated to an appropriate level • all models are wrong to varying

degrees, some models are useful

Aware of the different types of system modelling and simulation

E01, E02 Can name different types of modelling and simulation, for example, live, virtual, constructive

Table 33 - Evidence criteria for modelling and simulation function – Proficiency level: Supervised practitioner



Uses system modelling and simulation tools and techniques to represent a system or system element

E01, E02, E11

Operating a model, a simulation or both

Understands the risks of using system models and simulations which are outside the validated limits

E01, E02, E11, E14, E34

Has identified the system risks associated with the validity of the modelling results Presented the modelling and simulation results in the context of the system

Table 34 - Evidence criteria for modelling and simulation function – Proficiency level: Practitioner



Defines an appropriate representation of a system or system element

E11, E12 Can develop rationale for model selection

Uses appropriate representations of a system or system element in order to derive knowledge about the real system

E11, E12, E28

Validation of system model results against actual system performance Use of appropriate validated system model

Implements the strategy and approach to be adopted for the modelling and simulation of a system or system element

E11, E12, E28

Project or program documentation, system modelling reports, reviews







E08, E10 Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in modelling and simulation Evidence of assignment as a mentor

Table 35 - Evidence criteria for modelling and simulation function – Proficiency level: Lead practitioner



Demonstrates a full understanding of complex simulations for a system or system element

E11, E12, E28

Technical document detailing consistent, validated system performance

Advises on the suitability and limitations of system models and simulations

E11, E12, E28

Documented advice of adoption or rejection of system models and simulations

Defines the strategy and approach to be adopted for the modelling and simulation of a system or system element

E11, E12, E28

SEMP or modelling plans published work




E35 Documented examples of the introduction of novel modelling and simulation techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model






6.1.9. Select preferred solution

The evidence required for the function of select preferred solution is explained in Table 36

through to Table 39. A preferred solution will exist at every level within the system and is

selected by a formal decision making process.

Table 36 - Evidence criteria for select preferred solution function – Proficiency level: Awareness



Aware of the need to select a preferred solution

E01, E02 Aware of baseline solution needs to be selected and communicated to the development team to allow continuation to the next systems engineering process

Aware of the relevance of comparative techniques such as trade studies, make or buy, and so on to assist decision processes

E01, E02 Formal processes used to enable the decision making process and aid in arriving at a preferred solution Should talk about trade studies, make or buy, cost-benefit analysis, quality function deployment (QFD) or other formal decision making processes Aware of the difference between ‘musts’ and ‘wants’

Table 37 - Evidence criteria for select preferred solution function –Proficiency level: Supervised practitioner



Participates in the selection of preferred solutions

E01, E02, E12, E13, E14, E22, E23

Contributes to a formal decision making process where a preferred solution was selected Contributes to the definition of selection criteria as part of a decision making process Carries out cost analysis as part of a decision making process Carries out risk analysis as part of a decision making process

Understands the need to select solutions that are safe SFAIRP

E01, E02, E11, E13, E14, E22, E23, E34

Has contributed to selection of solutions that are safe SFAIRP




Table 38 - Evidence criteria for select preferred solution function – Proficiency level:

Practitioner



Defines selection criteria, weightings of the criteria and assess potential solutions against selection criteria

E11, E13, E14, E22, E23

Authored output from the decision making process

Chooses the appropriate tools and techniques for selecting the preferred solution; for example, trade analysis, make or buy analysis

E11, E13 Can develop SEMP

Performs trade analysis and justifies the result chosen in terms that can be quantified and qualified

E11, E13 Authored output from trade analysis

Negotiates solution trade-offs E01, E11, E13

Minutes of meeting describing solution trade-off decision made

Selects preferred solutions that are safe SFAIRP

E11, E13, E14, E22, E23, E34

Selected solutions that are safe SFAIRP


E08, E10 Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in select preferred solution Evidence of assignment as a mentor

Table 39 - Evidence criteria for select preferred solution function – Proficiency level: Lead practitioner



Guides and advises practitioners in techniques for selection of preferred solutions

E08, E10 Documented evidence in guiding and advising practitioners in techniques for selection of preferred solutions

Reviews selected solutions and the criteria for selecting the solution

E11, E13, E22, E23

Authored report outlining such a solution selection review

Acts as an arbitrator in marginal cases for trade analyses

E11, E13, E22, E23

Meeting minutes or report outlining role as arbitrator for trade analyses

Carries out sensitivity analysis on selection criteria

E11, E13 Authored report of sensitivity analysis

Negotiates complex trades E11, E13, E22, E23

Authored report of complex trade analysis









E35 Documented examples of the introduction of novel select preferred solution techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique, for example, simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model



6.1.10. System design – System integrity The evidence required for the function of system design - system integrity is explained in

Table 40 through to Table 43. A system with integrity is tolerant of misuse, out of specification

scenarios, component failure, environmental stress and evolving needs.




Table 40 - Evidence criteria for system design – system integrity function – Proficiency

level: Awareness



Aware of how the design, throughout the life cycle, affects the integrity of the system solution

E01, E02 Aware of: • relationship between system design

and overall system life cycle • that integrity has to be 'designed in' • that integrity affects system reliability

and therefore availability • that human factors are likely to play a

part in the ultimate integrity of a system (both explicitly in a system containing humans and through human involvement in the systems engineering process)

Awareness of analytical techniques and the importance of design integrity, legislation, whole-of-life costs and customer satisfaction

E01, E02 Appreciates that there are many drivers in determining the necessary level of integrity for a system Aware of a number of techniques for analysing system integrity such as reliability block diagrams, fault trees, reliability models, failure mode, effects and criticality analysis (FMECA), failure mode and effects analysis (FMEA), problem or failure reports and so on

Table 41 - Evidence criteria for system design – system integrity function – Proficiency level: Supervised practitioner



Uses tools and techniques to ensure delivery of robust designs

E01, E02, E17, E18

Use of techniques for analysing system integrity

Supports integrity trade-offs E01, E02, E20, E21, E22, E23, E23

Supports trade-off activities that affect integrity Documents agreed trade-offs in project or program or program documentation

Understands the relationship between reliability, availability, maintainability and safety

E01, E02, E14, E17, E18, E29, E34

Availability, reliability, maintainability and safety calculations

Table 42 - Evidence criteria for system design – system integrity function – Proficiency level: Practitioner



Defines the strategy and approach to be adopted for ensuring system integrity

E09, E11, E13, E18

Independently assessed documentation defining the strategy and approach to be adopted for ensuring system integrity






Selects the appropriate techniques for ensuring system integrity

E09, E11, E13, E18

Independently assessed documentation selecting the appropriate techniques for ensuring system integrity

Understands the operational environment and underlying domain specific issues related to integrity

E06, E20, E21, E22, E23

Independently assessed documentation describing the operational environment and underlying domain specific issues related to integrity

Performs integrity trade-offs E11, E13, E18

Independently assessed integrity trade-off report

Use case scenarios to determine integrity

E19, E20, E21, E22

Independently assessed scenarios for determination of integrity

Specifies procurement of system elements in terms of reliability, availability, maintainability and safety (RAMS)

E03, E04, E06, E20, E21, E22, E23

Independently assessed procurement specifications of system elements in terms of RAMS RAMS system analysis reports


E08, E10 Examples of on the job training objectives, guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in system integrity Evidence of assignment as a mentor

Table 43 - Evidence criteria for system design – system integrity function – Proficiency level: Lead practitioner



Predicts evolving needs and their impact on the system

E17, E20, E21, E22, E23

Documentation of complete prediction of evolving needs and their impact on the system Review comments Documented advice

Reviews and advises on trade-offs between non-functional requirements, cost and schedule

E06, E11, E13, E19, E20, E21, E22, E23

Acted as an internal or external consultant in the relevant areas

Defines scenarios to determine integrity

E11, E19, E20, E21, E22, E23

Acted as an internal or external consultant in relevant areas

Coaches practitioners in this field E08, E10 Can provide examples of coaching activities and the outcome of the process Conducted formal training courses and authored training material supported by successful post-training evaluation data Listed as an approved trainer in the organisation







E35 Documented examples of the introduction of novel system integrity techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model



6.1.11. Systems integration and verification The evidence required for the function of systems integration and verification is explained in

Table 44 through to Table 47. Systems integration is a logical process for assembling the

system. Systems verification is the checking of a system against its design. Systems integration

and verification includes testing of all interfaces, data flows, control mechanisms, performance

and behaviour of the system against the system requirements, and qualification against the

super-system environment. For example, electromagnetic compatibility, thermal, vibration,

humidity and fungus growth environments.

Table 44 - Evidence criteria for systems integration and verification function – Proficiency level: Awareness



Aware of the importance of verification against the system requirements

E01, E02 The system should be verified against the requirements (system requirements) in order to ensure that the specified design requirements are fulfilled by the system






Aware of the need to integrate the system in a logical sequence

E01, E02 Integration is conducted using a progressive, logical process of assembling system elements, evaluating them and then assembling the next level (system build) Alternative system integration sequences may be assessed in order to define the most appropriate sequence in terms of overall cost and risk (this means that the integration sequence should not necessarily be based on a 'success assumed' process) If system integration is performed in the wrong sequence, rework and extra cost may be incurred (dependency on suppliers, development, new technology, obsolescence and so on)

Aware of the need to plan for systems integration and verification

E01, E02 Planning for systems integration and verification should occur as early as possible in the beginning of the project or program Failure to plan could result in a delay to system integration and verification; procedures may not be written; the sequences may not have been defined and the environment may not be available The systems integration sequence should be recorded To identify the resources, equipment and develop test requirements (influence the design)

Aware of the relationship between system verification and system acceptance

E01, E02 A system may be verified against the system requirements but may not be accepted by the customer as fit for purpose (that is, not meeting the business requirements) System verification evidence may support system acceptance May have built the system right but it may not be the right system

Table 45 - Evidence criteria for systems integration and verification function – Proficiency level: Supervised practitioner



Conducts system integration and test according to the plan

E01, E02, E11, E12, E23, E24, E25, E26, E27

Runs system integration and verification tests






Writes a system integration and verification plan for a small non-complex system

E01, E02, E11, E12, E23, E24, E25, E26, E27

Has written or contributed to a system integration, verification plan or both

Diagnoses simple system faults, document, communicate and follow up corrective actions

E01, E02, E17

Has diagnosed simple system faults Has used the appropriate process, tool to record system faults

Table 46 - Evidence criteria for systems integration and verification function – Proficiency level: Practitioner



Traces verification requirements back to system requirements and vice versa

E06, E17, E20, E21, E22, E23, E24, E25, E26, E27

Can develop RATM Can develop RVTM

Writes a systems integration and verification plan for a complex system, including identification of method and timing for each activity

E11, E24, E25, E26, E27

Can develop RVTM Can develop integration plan or schedule Can develop verification plan or schedule

Demonstrates effective management of systems integration and verification activities

E11, E17, E24, E25, E26, E27

Systems integration and verification measures showing actual performance against plan Minutes of system test readiness review including relevant action log

Writes detailed system integration and verification procedures

E11, E25, E26, E27

Approved system integration procedures Approved system verification procedures verification matrix

Diagnoses complex system faults, record, communicate and follow up corrective actions

E13, E17 System fault and defect logs Corrective actions logs Minutes of system fault analysis meetings

Plans and prepare evidence for customer acceptance and certification

E11, E24, E25, E26, E27, E28

Can develop RATM Can develop RVTM System and subsystem test reports System certification data package System acceptance data package

Identifies the system integration and verification environment

E11, E24, E25, E26, E27

System integration and verification plans Procurement of equipment







E08, E10 Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in integration and verification Evidence of assignment as a mentor

Table 47 - Evidence criteria for systems integration and verification function – Proficiency level: Lead practitioner



Acts as an authority in the development of systems integration and verification strategies

E11, E24, E25, E26, E27, E35

System integration and verification strategies that proved successful

Reviews and judges the suitability of systems integration and verification plans

E11, E24, E25, E26, E27

Review comments for system integration and verification plans

Leads complex systems integration and verification activities

E11, E12, E24, E25, E26, E27

System integration and verification measures showing actual performance against plan




E35 Documented examples of the introduction of novel integration and verification techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model






6.1.12. Validation

The evidence required for the function of validation is explained in Table 48 through to Table 51.

Validation checks whether the operational capability of the system meets the needs of the

customer and end user, that is, to answer the question – 'Did we build the right system?'

Table 48 - Evidence criteria for validation function – Proficiency level: Awareness



Aware of the purpose of validation E01, E02 Aware: • that validation comprises ‘product’

validation, that is, the product satisfies user needs in operation and ‘requirements’ validation, that is set of system requirements meets the user needs

• that the role of validation is to reduce the risk of system failure to an acceptable level

• that validation activities should be undertaken by someone different from the people who designed and built the system

• distinction between verification activities, which address whether a system has been built correctly in accordance with the system requirements, and validation, which addresses whether the correct system has been built against the user needs

Aware of the need for early planning for validation

E01, E02 Can explain the need for early planning Can describe the system engineering activities associated with validation in relation to a chosen life cycle model Can describe the reasons why every user requirement should have an associated validation activity Aware of the need to plan for the validation of the system in the correct operational environment wherever practicable (or through simulated environments where that is impracticable)




Table 49 - Evidence criteria for validation function – Proficiency level: Supervised

practitioner



Conducts system validation activities according to the plans

E01, E02, E20, E28

Has experience of undertaking testing, analysis, inspection, demonstration, stimulation and simulation activities, such as operational, user trials and testing Has experience of using design documentation, prototypes, final products and systems documentation for validation activities

Collates validation results E01, E02, E11, E20, E28

Has experience of collating and presenting validation data Describes methods for handling exceptional and unexpected data

Table 50 - Evidence criteria for validation function – Proficiency level: Practitioner



Focuses on customer needs and able to communicate in the terminology of the customer or user

E11, E20, E28

Can present validation plans and discuss how they were carried out Can develop validation requirements Can develop test scripts Can develop validation test reports Can develop RVTM

Traces validation requirements back to user needs and vice versa

E20, E28 Can develop RVTM

Writes validation plans for a complex system, including identification of method and timing for each activity

E11, E28 Can develop validation plan

Writes detailed validation procedures

E11, E28 Can develop validation scenarios Can develop validation procedures Can develop validation test documentation

Demonstrates effective management of system validation activities

E11, E28 Organisational structures management documentation, such as metrics

Assesses validation results E11, E17, E28

Validation records

Plans and prepares evidence for customer acceptance

E11, E28 Minutes of customer acceptance reviews






Guides supervised practitioner E08, E10 Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in validation Evidence of assignment as a mentor

Table 51 - Evidence criteria for validation function – Proficiency level: Lead practitioner



Acts as an authority in the development of validation strategies

E11, E28, E35

Documented advice on validation strategies to others that has been implemented

Writes validation plans for a highly complex system

E11, E28, E35

Authored validation plans and have been successfully implemented

Reviews and judges the suitability of validation plans

E11, E28 Evidence of review and approval of validation plans

Leads the validation activity E11, E28 Evidence of leading validation activities; for example, job specifications, minutes of meetings and so on

Advises the customer on validation issues

E11, E28 Evidence of advice to customers on validation issues, for example, letters, emails and so on

Conducts the sensitive negotiations in the terminology of the customer and end user

E11, E28 Evidence of sensitive negotiations, taking into account customer’s background and knowledge, for example, in minutes of meetings, position papers, emails and so on




E35 Documented examples of the introduction of novel validation techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model






Contributes to best practice E35 Published papers in refereed journals/company literature Published articles or books and so on Ideas assimilated into international standards

6.1.13. Transition to operation The evidence required for the function of transition to operation is explained in Table 52 through

to Table 55. Transition to operation is the integration of the system into its super-system. This

includes provision of support activities such as site preparation, training and logistics.

Table 52 - Evidence criteria for transition to operation function – Proficiency level: Awareness



Aware of the need to carry out ‘transition to operation’

E01, E02 Aware of: • achieving user satisfaction in operation • sustained use of the system • the transition phase between

completion of development, production and readiness for use

• transition into service

Aware of the type of activities required for transition to operation

E01, E02 Aware: • that the system is ready for installation,

delivery and use • that the system has to be supported in

operation • that the people are trained • of the provision of guides, manuals,

demonstrations, instructions and so on • of the consideration for packaging,

storage, export controls and so on

Table 53 - Evidence criteria for transition to operation function – Proficiency level: Supervised practitioner



Plans simple transition to operation activities

E01, E02, E11, E27, E28

Has experience of transition planning

Conducts ‘transition to operation’ activities according to a plan

E01, E02, E11, E27, E28

Has participated in the transition to operation of a system






Aware of the system’s contribution to the super-system

E01, E02, E11, E27, E28

Has participated in transitioning a system into operation within a super-system

Table 54 - Evidence criteria for transition to operation function – Proficiency level: Practitioner



Communicates in the terminology of the user

E29 Manuals and guides written in the vocabulary of the user

Understands the system’s contribution to the super-system

E11, E27, E28, E29

Can develop transition plan Can develop operations plan

Plans and oversees a transition to operation activity

E11, E27, E28, E29

Project or program transition to operation authored plans and reviews

Guides supervised practitioner E08, E10 Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in transition to operation Evidence of assignment as a mentor

Table 55 - Evidence criteria for transition to operation function – Proficiency level: Lead practitioner



Plans and oversees highly complex transition to operation activities

E11, E27, E28

Transition plan or other project or program engineering plans, transition completion reports

Successfully transitioned a system to operation

E11, E27, E28, E29

System being used successfully for the required period of time









E35 Documented examples of the introduction of novel transition to operation techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model


6.2. Systems engineering management competencies The minimum competence requirements and minimum evidence requirements for systems

engineering management competencies are explained in Section 6.2.1 through to Section 6.2.5.

6.2.1. Concurrent engineering The evidence required for the function of concurrent engineering is explained in Table 56

through to Table 59. This function involves managing concurrent life cycle activities and the

parallel development of system elements.

Table 56 - Evidence criteria for concurrent engineering function – Proficiency level: Awareness



Aware that life cycle activities and the development of systems elements can occur concurrently

E01, E02 Aware that: • different aspects are developed

concurrently • multidisciplinary team exists • development moves forward over

diverse range of disciplines and teams • for concurrent design the focus is on the

design part of the life cycle • SE processes can overlap • practical implementation of the concept

of left shift with regard to resources






Aware of the advantages and disadvantages of concurrency

E01, E02 Aware of: • reduced development time in an attempt

to reduce cost • reduced development time and hence

‘time to market’ • optimised solution through increased

communication • compromise design through lack of in

depth analysis time • increased risk • need for increased management

vigilance • need for efficient information control

infrastructure

Table 57 - Evidence criteria for concurrent engineering function – Proficiency level: Supervised practitioner



Describes the systems engineering life cycle processes that are in place on their program

E01, E02, E05, E07, E11, E12

Identifies concurrent engineering tasks Identifies base-lining milestones and describing the significance to associated tasks Identifies task interdependencies

Supports coordination of concurrent engineering activities

E01, E02, E05, E07, E11, E12

Schedules multiple concurrent tasks

Table 58 - Evidence criteria for concurrent engineering function – Proficiency level: Practitioner



Identifies which system elements can be developed concurrently

E05, E07, E11, E12

Project or program schedules showing concurrent engineering tasks

Manages the interactions within a systems engineering life cycle

E05, E07, E11, E12, E15, E20, E21, E22, E23, E24, E25

Can develop configuration control plan Can develop ICD Schedule for base-lining or interface definition milestones in a project or program schedule Evidence of identifying task performance or schedule variance and appropriate intervention

Coordinates concurrent activities and deals with emerging issues

E05, E07, E11, E12

Resource budget and evidence of the management of this Performance budget and evidence of the management of this






Contributes to the SEMP E11, E12 Example of authored SEMP with identification of section relating to dealing with concurrent engineering

Advises on concurrency issues and risks

E05, E07, E11, E12, E14, E34

Technical notes, reports, email highlighting individual issues or the generic issues and risks Periodic project or program reports showing awareness of and highlighting these issues and risks

Guides supervised practitioner E08, E10 Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in concurrent engineering Evidence of assignment as a mentor

Table 59 - Evidence criteria for concurrent engineering function – Proficiency level: Lead practitioner



Known as an authority in systems engineering management

E35 Published papers in refereed journals New facility supporting concurrent engineering

Develops new strategies for concurrent engineering

E35 Published papers in refereed journals New facility supporting concurrent engineering

Advises customers and senior program managers on concurrency issues and risks

E05, E09, E14, E18, E34, E35

Meeting minutes or report showing authored advice

Reviews and judges the suitability of SEMPs

E05, E11, E12

Sign-off on multiple SEMPs

Influences the implementation of concurrent engineering within the enterprise

E35 Meeting minutes or report showing authored recommendation


E08, E10 Provides examples of coaching activities and the outcome of the process Conducted formal training courses and authored training material supported by successful post-training evaluation data Listed as an approved trainer in the organisation







E35 Documented examples of the introduction of novel concurrent engineering techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model


6.2.2. Enterprise integration The evidence required for the function of enterprise integration is explained in Table 60 through

to Table 63. Systems engineering management involves the support of other functions such as

quality assurance, marketing, sales, and configuration management. This includes the

management of the interfaces between the different functions.

Table 60 - Evidence criteria for enterprise integration function – Proficiency level: Awareness



Aware that an enterprise is a system in its own right

E01, E02 Aware of: • analogies between systems and the

business infrastructure • importance and relevance of interfaces,

processes and methodologies governing operations

• influences and interactions • organisational culture






Aware that other functions of the enterprise have inputs to and outputs from the systems engineering process

E01, E02 Aware of: • relationships and interfaces • outputs and dependencies from systems

engineering process to other functions • inputs and dependencies to systems

engineering process from other functions • models and frameworks • allocation of functions and

responsibilities • gate review processes and peer

assessment

Table 61 - Evidence criteria for enterprise integration function – Proficiency level: Supervised practitioner



Understands other functions such as quality assurance, marketing, sales, strategic management, configuration management, research, human resources and relationships that make up an enterprise

E01, E02, E07, E08, E09, E11, E12, E15, E16, E18, E33

Engagement and team building Planning and task allocation

Manages the creation of systems engineering products required by other functions

E01, E02, E11, E12, E13, E15, E16, E18, E33

Has had responsibility for managing the creation of systems engineering products required by other functions

Table 62 - Evidence criteria for enterprise integration function – Proficiency level: Practitioner



Manages the relationship between the systems engineering function and other elements of the enterprise

E05, E07, E11, E12

Resolution of conflict Allocation of tasks Information on time and to the right place

Identifies systems engineering products required by other functions and vice versa

E05, E07, E11, E12

Can develop tasks and resource maps Plan of information flow to and from other functions

Uses systems engineering techniques to contribute to the definition of the enterprise

E11, E33 Can develop enterprise models, architecture and frameworks

Identifies the constraints placed on the systems engineering process by the enterprise

E06, E20, E21, E22, E23

List of constraints (non-functional requirements on a system)






Guides supervised practitioner E08, E10 Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in enterprise integration Evidence of assignment as a mentor

Table 63 - Evidence criteria for enterprise integration function – Proficiency level: Lead practitioner



Acts as a consultant on business organisations

E33 Can develop enterprise model Minutes of meetings creating the model

Advises on the effectiveness of the enterprise as a system

E09, E18, E33

Process improvement plans and quantitative results

Reviews the impact of systems engineering capability within a business context

E05, E09, E17, E18

Metrics trends and evidence of review Incorporation of any review changes Continuity

Reviews the impact of inputs from other functions on the systems engineering process

E05, E09, E12, E13, E18, E19, E35

Authored impact analysis Report showing uptake of recommendations of analysis Time saving and quality of the delivery


E08, E10, E35

Provides examples of coaching activities and the outcome of the process Conducted formal training courses and authored training material supported by successful post-training evaluation data Listed as an approved trainer in the organisation


E35 Documented examples of the introduction of novel enterprise integration techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model







6.2.3. Integration of specialisms

The evidence required for the function of integration of specialisms is explained in Table 64

through to Table 67. This function involves the coherent integration of specialisms into the

project or program at the right time. Specialisms can include reliability, maintainability,

testability, integrated logistics support, producibility, electromagnetic compatibility, human

factors and safety.

Table 64 - Evidence criteria for integration of specialisms function – Proficiency level: Awareness



Aware of different specialisms E01, E02 Defines a specialism Gives examples of specialisms

Aware of the importance of integrating specialisms into the project or program and that this is a potential source of conflict

E01, E02 Explains what is meant by integration of specialisms Explains the types of conflict that may occur Identifies the practical implementation of the concept of left shift with regard to resources

Aware that the specialisms can affect the cost of ownership

E01, E02 Explains that different implementation levels of some specialisms such as availability and reliability may affect the cost of ownership (total costs of delivered solution)

Table 65 - Evidence criteria for integration of specialisms function – Proficiency level: Supervised practitioner



Understands the role and purpose of the specialisms

E01, E02, E08

Has worked in an interdisciplinary team including specialisms

Able to work with appropriate specialists to support trade-offs

E01, E02, E08, E20, E21, E22, E23

Has worked in an interdisciplinary team including specialisms




Table 66 - Evidence criteria for integration of specialisms function – Proficiency level:

Practitioner



Manages the integration of specialisms within a project or program

E07, E11, E12

Can develop SEMP, work breakdown structure, task and resource map Project or program example

Conducts trade-offs involving conflicting demands from the specialisms

E11, E12, E13, E20, E21, E22, E23

Demonstrate use of trade-off tools Issue resolution Trade study

Understands how the specialisms affect the cost of ownership

E11, E12 Can develop whole-of-life cost model

Identifies the constraints placed on the system development by the needs of the specialisms

E11, E12 Limits imposed on system development

Guides supervised practitioner E08, E10 Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in integration of specialisms Evidence of assignment as a mentor

Table 67 - Evidence criteria for integration of specialisms function – Proficiency level: Lead practitioner



Understands primary tasks of each specialism

E11, E12 Project plans for specialist area

Successfully applied integration principles across a number of specialisms

E06, E07, E11, E12, E20, E21, E22, E23, E24, E25, E26, E27, E28

Project documentation in the specialist area

Resolves conflicts involving specialisms

E11, E12, E13

Minutes of meetings

Estimates the combined effect of the specialisms on the cost of ownership and the system development

E11, E12 Can develop whole-of-life cost models

Advise on the organisation of specialist functions

E11, E12 Correspondence containing advice









E35 Documented examples of the introduction of novel integration of specialisms techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model


6.2.4. Life cycle process definition The evidence required for the function of life cycle process definition is explained in Table 68

through to Table 71. Life cycle process definition establishes life cycle stages and their

relationships depending on the scope of the project or program. It also establishes super-

system characteristics, stakeholder requirements and the level of risk. Different system

elements can have different life cycles.

Table 68 - Evidence criteria for the life cycle process definition function – Proficiency level: Awareness



Aware of the different types of system life cycles

E01, E02 Typical system life cycles include: • acquisition (concept, assessment,

demonstration, manufacture, in-service and disposal)

• product (concept, development, production, utilisation, support, retirement)

Aware of the different types of life cycle models

E01, E02 Life cycle models include waterfall, spiral, iterative, incremental, evolutionary






Aware of the need to define an appropriate life cycle process model

E01, E02 Project or program vs product life cycle Characteristics that affect project or program life cycle models include size of project or program, experience of staff, cycle time, acceptable defect levels Appropriate life cycle processes can be defined by tailoring standard processes

Table 69 - Evidence criteria for the life cycle process definition function – Proficiency level: Supervised practitioner



Understands systems engineering life cycle processes


Worked on projects or programs that follow a typical systems engineering life cycle Carried out work on some of the system engineering life cycle processes (for example competencies within 'holistic life cycle view')

Supports life cycle definition activities

E01, E02, E05, E07, E11, E12, E32

Supports facilitation of process tailoring Documents agreed tailoring in project or program plans

Describes the systems engineering life cycle processes that are in place on their project or program


Carried out work on some of the system engineering life cycle processes, for example, competencies within 'holistic life cycle view'

Table 70 - Evidence criteria for the life cycle process definition function – Proficiency level: Practitioner



Identifies the project or program, enterprise and technology needs that affect the definition of the life cycle

E05, E07, E11, E12, E16, E32

Defines and advises on suitability of system and program life cycles Leads and facilitates process tailoring on projects and programs

Influences the life cycle of related super-system elements

E05, E07, E11, E12, E32

Minutes of meetings with the customer discussing super-system and system life cycles Documentation of dependencies, constraints and risks of differing life cycles

Identifies dependencies and aligns the life cycles of different system elements

E05, E07, E11, E12, E32

Documentation of the complete system life cycle






Guides supervised practitioner E08, E10 Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in life cycle process definition Evidence of assignment as a mentor

Table 71 - Evidence criteria for the life cycle process definition function – Proficiency level: Lead practitioner



Acts as an authority on life cycle definitions and the implication of life cycle on the project or program

E05, E07, E11, E12, E13, E32

Documentation of complete program life cycles

Resolves conflicts between life cycles

E05, E07, E11, E12, E13

Review comments Documented advice

Reviews and judges the suitability of the definition of multiple concurrent life cycles

E05, E07, E11, E12, E13, E35

Review comments

Advises program management on the implication of life cycle issues including project or program and commercial

E05, E07, E11, E12, E13, E35

Review comments Documented advice

Successfully determined and documented life cycles matched to the needs of the project or program

E05, E07, E11, E12, E13, E35

Documentation of SE life cycles on many occasions









E35 Documented examples of the introduction of novel life cycle process definition techniques and can provide evidence of the improvement made Published papers in refereed journals/ company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model


6.2.5. Planning, monitoring and controlling The evidence required for the function of planning, monitoring and controlling is explained in

Table 72 through to Table 75. This function establishes and maintains systems engineering

plan, which incorporates the tailoring of generic processes .This function also involves the

identification, assessment, analysis and control of systems engineering risks, and the

monitoring and control of progress.




Table 72 - Evidence criteria for planning, monitoring and controlling function –

Proficiency level: Awareness



Aware of the importance of planning, monitoring and controlling systems engineering activities

E01, E02 Aware that: • plans are required to define project or

program activities • plans are based on project or program

requirements, statements of work and estimates of effort and cost

• monitoring and controlling is required to provide an understanding of the project or program's progress so that appropriate corrective actions can be taken when progress deviates from the plan

• failure to plan, monitor and control significantly increases risk and will probably lead to schedule and cost overrun

• systems engineering planning is typically documented in a SEMP

• the relationship between the SEMP and the project or program management plan should be clearly understood

Aware that change is inevitable and so needs to be carefully managed

E01, E02 Describes where and when change can occur in the life cycle Describes the elements of change management

Table 73 - Evidence criteria for the planning, monitoring and controlling function – Proficiency level: Supervised practitioner



Understands the role of systems engineering planning as part of an overall project or program plan

E01, E02, E11, E12

Contributes towards writing a SEMP or relevant section of a project and program management plan

Monitors progress against the systems engineering plan

E01, E02, E11, E12, E17, E18

Collects and collates data on project or program performance measures (technical and programmatic)

Assists in the management of systems engineering risks

E01, E02, E14, E17, E18, E34

Assists in running a risk management activity

Assists in the management of systems engineering changes

E01, E02, E11, E12, E13, E15

Configuration and change records




Table 74 - Evidence criteria for planning, monitoring and controlling function –

Proficiency level: Practitioner



Plans systems engineering activities as part of an overall project and program plan

E11, E12 Can develop SEMP

Identify, assess, analyse and control systems engineering risks

E14, E34 Can develop systems engineering risk register Can develop risk management plan Minutes of risk review meetings

Anticipate, identify, assess, analyse and control systems engineering changes

E11, E12, E13, E15

Configuration and change records

Influences project or program management in order to secure the systems engineering needs of the project or program

E07, E11, E12

Before and after project or program plans and so on Minutes of project or program review meetings

Controls systems engineering activities by applying necessary corrective actions

E11, E12, E14, E17, E18

Project or program quantitative management plan Project or program performance measures (run charts, control charts, Pareto analysis, root cause analysis and so on) Evidence of project and program tracking, for example, updated Gantt chart, updated plans, and so on

Tailors systems engineering processes to meet the needs of a specific project or program

E11, E12, E32

Tailored systems engineering processes on a specific project or program

Guides supervised practitioner E08, E10 Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in planning, monitoring and controlling Evidence of assignment as a mentor

Table 75 - Evidence criteria for planning, monitoring and controlling function – Proficiency level: Lead practitioner



Successfully planned, monitored and controlled complex systems engineering activities

E07, E11, E12, E13

Authored SEMP or project or program management plan Project or program status reports Project or program graphs showing performance measures such as defect containment by phase, schedule performance index and so on






Reviews and judges the suitability of systems engineering plans

E11, E12 Review comments

Advises on systems engineering risks and their mitigation

E14, E34 Update to risk register showing changes to risks, mitigation actions or both Minutes of risk review meetings

Defines appropriate generic systems engineering processes for the enterprise

E05, E11, E33

Systems engineering processes that have been adopted by the enterprise

Influences the relationship between systems engineering and project or program management at the enterprise level

E07, E09, E11, E12, E13

Examples of improvements to project or program planning, monitoring and control due to intervention




E35 Documented examples of the introduction of novel planning, monitoring and controlling techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model





6.3. Systems thinking competencies The minimum competence requirements and minimum evidence requirements for systems

thinking competencies are explained in Section 6.3.1 through to Section 6.3.3.

6.3.1. System concepts The evidence required for the function of system concepts is explained in Table 76 through to

Table 79. This function establishes the application of the fundamental concepts of systems

thinking to systems engineering. This requires an understanding of what a system is, its context

within its environment, its boundaries and interfaces and its life cycle.

Table 76 - Evidence criteria for systems concept function – Proficiency level: Awareness



Aware of the need for systems concepts

E01, E02 Aware that systems are more than interfaced collections of parts Appreciates both static and dynamic properties of systems Aware of viewpoints – different perspectives on systems

Aware of the importance of system life cycle

E01, E02 Aware that a system has a life cycle from concept to retirement (ISO 15288) Knows a number of key life cycle stages Appreciates relationship between the stages and phases and the possibility of interaction, for example, basic trade-offs, such as first cost versus operating costs

Aware of the importance of hierarchy of systems

E01, E02 Knows that this includes but also means more than decomposition Aware of levels of detail Can relate this issue to those of context, super-system, system of interest, subsystems and beyond

Aware of the importance of system context

E01, E02 Appreciates the hierarchical view Aware that context is important when considering systems

Aware of the importance of interfaces

E01, E02 Aware a system has a boundary Aware that the system interacts across its boundary Aware that interfaces may be external or internal to the system

Aware of the importance of interactions amongst systems and their elements

E01, E02 Aware of the concepts of abstraction, interaction and emergence




Table 77 - Evidence criteria for systems concept function – Proficiency level: Supervised

practitioner



Understands systems concepts E01, E02, E06, E19, E20, E21, E22, E23

Has used a basic concept map or other model of a system at some stage of its development Has seen and appreciated the utility of system concepts prepared by others

Understands the system life cycle in which they are working

E01, E02, E05, E06, E11, E19, E20, E21, E22, E23, E24, E25, E26, E27, E28, E29, E30, E31, E32

Has participated in the life cycle aspects of a current or recently completed project or program

Understands system hierarchy and the principles of system partitioning in order to deal with complexity

E01, E02, E06, E20, E21, E22, E23

Has performed some form of decomposition – functional analysis or other modelling

Understands the concept of emergent properties

E01, E02, E06, E20, E21, E22, E23

Provides examples of emergent properties in their own or associated work

Identifies system boundaries and understands the need to define and manage the interfaces

E01, E02, E06, E19, E20, E21, E22, E23

Has carried out or been involved in partitioning or interface work

Understands how humans and systems interact and how humans can be elements of systems

E01, E02, E06, E19, E20, E21, E22, E23

Has contributed to analysis of human factors

Table 78 - Evidence criteria for systems concept function – Proficiency level: Practitioner



Identifies and manages complexity with appropriate techniques in order to reduce risk

E01, E02, E06, E17, E19, E20, E21, E22, E23, E34

System studies tackling the issues of complexity and recommending suitable approaches

Predicts resultant system behaviour

E01, E02, E06, E11, E19, E20, E21, E22, E23, E28

Requirements for system modelling and validation exercises Validated system analysis






Defines system boundaries and external interfaces

E01, E02, E06, E20, E21, E22, E23

Can develop system definition document Can develop system block diagram Can develop system interface control document

Assesses the interaction between humans and systems, systems and systems

E01, E02, E06, E19, E20, E21, E22, E23

Can develop human factors analysis reports Can develop human computer interaction (HCI) models Can develop system analysis reports Can develop system models

Guides supervised practitioner E01, E02, E08, E10

Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in system concepts Evidence of assignment as a mentor

Table 79 - Evidence criteria for systems concept function – Proficiency level: Lead practitioner



Reviews and judges the suitability of systems solutions and the planned approach

E01, E02, E06, E11, E19, E20, E21, E22, E23, E24, E25, E26, E27, E28, E29, E35



E01, E02, E08, E10



E01, E02, E35

Documented examples of the introduction of novel system concepts techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model






Contributes to best practice E01, E02, E35

Documented examples of the introduction of novel system concepts techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model

6.3.2. Super-system capability issues The evidence required for the function of super-system capability issues is explained in

Table 80 through to Table 83. This function involves an appreciation of the role the system

plays in the super-system of which it is a part.

Table 80 - Evidence criteria for super-system capability issues function – Proficiency level: Awareness



Aware of the concept of capability

E01, E02 Capability includes people, information, organisation, strategic goals and the technical systems needed to achieve the aims of the super-system owner Explains the concept of capability and its relationship to system requirements Appreciation of the hierarchy of systems

Aware that capability requirements can be satisfied by a system of systems approach

E01, E02 Explains the term systems of systems Aware that different organisations and teams may develop the individual systems

Aware that super-system capability needs impact on the system development

E01, E02 Appreciation that there is interaction as well as interface, that is, the system will affect the super-system and vice versa Aware that there are constraints or impacts on the system imposed by the super-system

Appreciates the difficulties of translating super-system capability needs into system requirements

E01, E02 Aware of basic conceptual mapping between capability and lower level requirements Appreciates the need for modelling or simulation in aiding the translation




Table 81 - Evidence criteria for super-system capability issues function – Proficiency

level: Supervised practitioner



Describes the environment and super-system into which the system under development is to be delivered

E01, E02, E06, E19, E20, E21, E22, E23

Worked on a project or program where the understanding of context is important

Identifies, with guidance, the super-system capability issues which will affect the design of a system

E01, E02, E06, E19, E20, E21, E22, E23

Participated in team reviews of systems context definition

Table 82 - Evidence criteria for super-system capability issues function – Proficiency level: Practitioner



Identifies the super-system capability issues which will affect the design of a system and translates these into system requirements

E01, E02, E06, E19, E20, E21, E22, E23

Can develop system requirements document Minutes of user or system requirements reviews Can develop technical reports

Assesses extent to which the proposed system solution meets the super-system capability, and provide advice on trade-offs

E01, E02, E13, E19, E20, E21, E22, E23

Can develop trade study reports Can develop review evidence Can develop technical reports

Guides supervised practitioner E01, E02, E08, E10,

Examples of on the job training objectives and guidance, and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in super-system capability issues Evidence of assignment as a mentor

Table 83 - Evidence criteria for super-system capability issues function – Proficiency level: Lead practitioner



Reviewed and advised on the suitability of systems solutions

E01, E02, E06, E19, E20, E21, E22, E23



E01, E02, E08, E10








E01, E02, E35

Documented examples of the introduction of novel super-system capability issues techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction with novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model

Contributes to best practice E01, E02, E35

Published papers in refereed journals/company literature Published articles or books and so on Ideas assimilated into international standards

6.3.3. Enterprise and technology environment The evidence required for the function of enterprise and technology environment is explained in

Table 84 through to Table 87. This function involves the definition, development and production

of systems within an enterprise and technological environment.

Table 84 - Evidence criteria for enterprise and technology environment function – Proficiency level: Awareness



Aware of the influence the enterprise (environment, objectives, social, political, financial, cultural, research) has on the definition and development of the system

E01, E02 Aware that influences may affect requirements Aware of the need to address influences with the agreement of stakeholders

Aware of the influence technology has on the definition and development of the system

E01, E02 Aware of the risk of mandating a technology Aware of the risk of relying on technology innovation to provide solutions Aware that technology availability and maturity affects system development

Aware of the influence the system has on the enterprise

E01, E02 Aware that: • the system may have an effect on the

enterprise such as facilities, number of staff and so on

• effects may not be apparent in the early stages of system development






Aware of the influence the system has on technology

E01, E02 Aware that new systems (projects or programs) either reinforce or broaden an enterprise’s understanding of its technology base when in-sourced or do that for someone else when outsourced. – enterprise level strategic issue

Table 85 - Evidence criteria for enterprise and technology environment function – Proficiency level: Supervised practitioner



Identifies, with guidance, the various enterprise issues (markets, products, policies, finance, technologies and so on) which interact with the system to be developed

E01, E02, E11, E12, E33

Contributed to analysis of one or more such issues as part of a project or program Used one or more methods

Contributes, with guidance, to the technology plan

E01, E02, E11, E12, E33

Read and understood a plan Contributed to a plan or taken part in analysis or other work contributing to a plan

Contributes, with guidance, to the enterprise improvement plan

E01, E02, E11, E12, E33

Read and understood a plan Contributed to a plan or taken part in analysis or other work contributing to a plan

Table 86 - Evidence criteria for enterprise and technology environment function – Proficiency level: Practitioner



Identifies the enterprise and technology issues which will affect the design of a system and translates these into system requirements

E01, E02, E06, E11, E19, E20, E21, E22, E23

Written or supervised the production of system requirements that take enterprise and technology considerations into account

Produces and implements a technology plan that includes technology innovation, risk, maturity, readiness levels and insertion points

E01, E02, E11, E12, E33

Can develop technology plan

Contributes to delivery of enterprise improvements to enable better system development

E01, E02, E11, E12, E13, E33

Can develop enterprise improvement plan Updated practices






Guides supervised practitioner E01, E02, E08, E10

Examples of on the job training objectives and guidance and so on Organisational breakdown structure for system development or project or program showing responsibility for managing those involved in enterprise and technology environment Evidence of assignment as a mentor

Table 87 - Evidence criteria for enterprise and technology environment function – Proficiency level: Lead practitioner



Influences and maintains the technical capability and strategy of their enterprise

E01, E02, E11, E12, E35

Successful projects or programs with technology advances either in depth or broader application

Recognised as an authority in technology planning and management

E01, E02, E11, E12, E35

Successful projects or programs with technology advances either in depth or broader application


E01, E02, E08, E10

Provides examples of the coaching activities and the outcome of the process Conducted formal training courses and authored training material supported by successful post-training evaluation data Listed as an approved trainer in the organisation


E01, E02, E10, E35

Documented examples of the introduction of novel enterprise and technology environment techniques and can provide evidence of the improvement made Published papers in refereed journals/company literature Evidence of development or introduction of novel facility supporting systems engineering technique such as simulated environment, concurrent design facility Published articles or books and so on Authored details of improvements to process and appraisal against a recognised process improvement model

Contributes to best practice E01, E02, E10, E35

Published papers in refereed journals/company literature Published articles or books and so on Ideas assimilated into international standards


Date post:	29-Jun-2020
Category:	Documents
Upload:	others
View:	9 times
Download:	0 times