July 2007, D1 1

Proposed Revision of ISO/IEC Proposed Revision of ISO/IEC 15026:15026:

Status ReportStatus Report

Jim MooreF-IEEE, CSDP

IEEE CS Liaison Representative to ISO/IEC JTC 1/SC 7

July 2007

July 2007, D1 2

Names are Important

445 Hoes Lane, Piscataway, NJ

Tract 15, Lot 25, Liber 227, Plat 22

Zip 08855- 1331

The portion of the 1677 royal grant of Charles II to Robert C. Smith, bounded by ...

Would you know that these are different names for the same thing? Would you know without the map?

We use names to localize the subject under discussion. But sometimes confusion results because we use different name spaces.

Driver’s view

Postal worker’s view

Town clerk’s view

Title researcher’s view

July 2007, D1 3

What are the names good for?

Jim Moore, 2004-03 CSEE&T Panel 7

A standard is a A standard is a NameName for an for an otherwise fuzzy conceptotherwise fuzzy concept

In a complex, multidimensional trade space of solutions ...

… a standard gives a name to a bounded region.

It defines some characteristics that a buyer can count on.

• Many software engineering standards assign names to practices or collections of practices.

• This enables communication between

• Buyer and seller

• Government and industry

• All IEEE and ISO/IEC standards are “voluntary”.

July 2007, D1 4

Some Basics

ISO IEC

JTC 1InformationTechnology

SC 7Software and Sys-tems Engineering

About 200 technical

committees

About 180 technical

committees and subcommittees

About two dozen subcommittees

July 2007, D1 5

US Relationship to SC 7

SC 7Software and Sys-tems Engineering

US TAGto SC 7

Members are a few dozen US-domiciled organizations, e.g.:

Computer Sciences Corporation

DoD

IEEE Computer Society

Members are “national bodies”, i.e., national standards organizations, e.g.,

ANSI (US)

AFNOR (France)

BSI (UK)

DIN (Germany)

American National

Standards Institute

The TAG recommends US positions and recommends the composition of the US delegation.

IEEE CS is also a liaison to SC 7, like INCOSE, ITSMF, PMI, and some other professional societies. Liaisons don’t vote.

July 2007, D1 6

Project to revise ISO/IEC 15026• Revision project was intended to replace and repurpose ISO/IEC

15026:1998 to become a comprehensive standard on software and systems assurance.

• 2004-02: WG 9 WD3 released. There is criticism of the approach.• 2004-05: SC 7 Resolution 770: ‘... change the name of its project IS

15026 from “System and software integrity levels” to “Systems Engineering - Software and system assurance” ...’

• 2004-05: Resolution 771: ‘... circulate a combined WD circulation, CD Registration and CD Ballot of ... 15026 ... when it is received...’

• 2004-05: Resolution 794: ‘... establish a Study Group to determine the derived system and software assurance requirements from ISO/IEC 15288, ISO/IEC 12207, and ISO/IEC 15026, and to recommend requirements for the development, modification, adoption, or reference of supporting standards...chaired by Mr. James Moore (IEEE-CS)...membership ... Paul Croll (USA)...’

July 2007, D1 7

More History

• 2005-02: Study group recommended:– “ISO/IEC 12207 and ISO/IEC 15288 (or their eventual

harmonized equivalents) will be umbrella documents for the software and system life cycle processes.

– “The current project to revise 15026 is intended to ... embed the process implications of system and software assurance in the life cycle processes provided by the two standards mentioned previously.

– “From this viewpoint, the process aspects of 15026 become a "delta" on the existing processes of 12207 and 15288.”

July 2007, D1 8

Intended Relationships of Key Software Engineering Process Standards

Revised 15288:Life cycle

processes for systems

Common vocabulary, process architecture, and process description conventions

Revised 12207:Life cycle

processes for SW

15026: Additional

practices for higher

assurance systems

Other standards providing details of

selected SW processes Interoperation

Revised 15939:

Measure-ment

Revised 16085:RiskMgmt

+

Other standards providing details of selected system

processes

24748: Guide to Life Cycle Management

Revised 16326:ProjectMgmt

Revised 15289:

Document-ation

July 2007, D1 9

More History• ca 2004-09: Editor (who wrote WD3) resigned.• 2004-09: IEEE CS offered to develop and contribute a

new draft. Offer is not accepted.• 2005-03 (?): Canada provides an editor• 2005-05: Resolution 832: ‘... issue a combined WD, CD

Registration and CD ballot of WD 15026.4 System and Software Engineering —Life Cycle Processes — Assurance when it is received ...’

• 2005-10: Convener briefs approach (see next page) to WG 7. WG 7 rejects the idea of a new process on the grounds that every discipline will want their own process.

Throughout this period, meetings of WG 9 were poorly attended.

July 2007, D1 10

Rev 3

AssurancePlan Im

pro

vem

en

t info

rma

tion

Establish & Maintain Assurance Argument

Ž

Plan Assurance Activities

Monitor & Control Assurance Activities &

Products

TECHNICAL & MANAGEMENT PROCESSES

Œ

Perform Measurement

Activities

Perform Risk Management

Activities

AssuranceNeeds

AssurancePlan

AssuranceMeasures

AssuranceMeasures

Assurance

Argument

AssuranceArgument

AssuranceIssues

CORE ASSURANCE PROCESS

Risk Information

July 2007, D1 11

More History• 2006-05: WG 9 convener loses funding and is unable to attend meeting.• 2006-05: Editor produces new draft that looks like a “management system” addressing QA

auditors. There is criticism of the approach.• 2006-05: Resolution 912: ‘Since project ISO 15026 has not reached Stage 3 in the two

years allowed by the JTC 1 directives, JTC 1/SC 7 note that the original NP is technically cancelled. JTC 1/SC 7 mandates its WG 09 to prepare a new NP that would ... adhere to the Study Group on Assurance recommendations (N3239). JTC 1/SC 7 instructs its Secretariat to ballot this NP once received and, if successful, assign it to its WG 7.’

• 2006-05: Resolution 929: ‘WG 9 has experienced declining and inconsistent participation and meeting attendance during the last four years. Consequently, insufficient resources are available to sustain a successful program of work. JTC 1 / SC 7 instruct its Secretariat to close its WG 9 ... WD 15026 is transferred to WG 7 for further development contingent upon successful balloting of the NWI in accordance with Resolution #912.’

• 2006-11: WG 7 rejects draft NP and WD and tells editor to go back to WD3 (2004-02).• 2006-11: WG 7 resolution: ‘In respect of the SQuaRE Study Group proposal on Specialty

Engineering, SC 7/WG 7 concludes that a Process View should be added to one of 12207/15288/24748.’

• 2006-12: FCD draft of 15288 revision includes the process view concept.

July 2007, D1 12

A New Concept: The “Process View”• Concept introduced in FCD 15288• Addresses a particular engineering interest• Gathers together process activities that address those

interests• Cuts across all or part of the life cycle• Like a process, a process view has:

– Purpose– Outcomes

• Unlike a process, it does not have activities and tasks. Instead, it musters activities and tasks from existing processes and adds guidance on how to use them to achieve the purpose and outcomes.

The term “view” comes from IEEE Std 1471, aka ISO/IEC 42010.

July 2007, D1 13

IEEE Computer Society Proposal• ISO/IEC 15026 should:

– Describe an “assurance case” as a central artifact for creating confidence that the system does, in fact, achieve desired properties

– Provide a process view for software and systems assurance.

• Sources of process provisions:– The example process view for specialty engineering.– Some necessary additions from 12207 and 15288,

e.g. risk management.– Hooks for regulatory activity, suggested by comments

received in the balloting of 12207 and 15288.– IEEE Std 1228, Software Safety Planning

July 2007, D1 14

IEEE CS Prepared an “Alternative Proposal” for SC 7

• IEEE CS developed a draft New Work Item proposal and a draft document.

• The proposal was reviewed and improved by a 14-member study group in IEEE CS.

• Because IEEE CS is a member of the US TAG to SC 7 (as well as a direct liaision to SC 7), the proposal was distributed to the US TAG.

• The proposal was tabled for the May 2007 plenary meeting of SC 7.

July 2007, D1 15

SC7 accepted the IEEE CS Proposal• After consideration, SC 7 approved the following resolution:

– ISO/IEC JTC 1/SC 7 thanks the IEEE Computer Society for the contribution of its "alternative proposal" (WG 7 N 1006) for the revision of ISO/IEC 15026 Systems and software assurance and requests the Convener of WG 7 to provide a draft New Work Item Proposal and Working Draft extracted from that proposal. SC 7 instructs its Secretariat to circulate for letter ballot, once available, the New Work Item Proposal (WG 7 N 1027) for the revision, and concurrently to circulate the working draft (WG 7 N1028) as a letter ballot for combined CD registration and CD approval. If and when the project is approved and in accordance with Resolution 912, SC 7 assigns the project to WG 7.

• Schedule: “Normal” process, executed quickly– Jun-Sep 2007: Ballot NP concurrently with CD registration and ballot of draft

– Oct 2007 interim meeting: Resolve comments

– Dec 2007-April 2008: FCD ballot

– May 2008 plenary meeting: Resolve comments

– Jul-Sep 2008: FDIS ballot

– Dec 2008: Publish

• The IEEE standardization process will operate concurrently so that both organizations arrive at the same standard.

July 2007, D1 16

Coordinated Development is used by S2ESC and JTC 1/SC 7

WG SC 7 JTC 1 ISO Staff S2ESC IEEE Staff

IEEE SA

Approve NP PAR

Draft

Form BG

CD Ballot

FCD Recirc

ProofProof

FDIS

Approve

IEEE StdISO/IEC

Prepare MS

Publishidentical

standards

May 2007

Oct 2007

May 2008

Oct 2008

Dec 2008

Recirc

DraftContribute Draft

July 2007, D1 17

Current Situation• A 65-page draft has been prepared.

– It incorporates material from FCD 12207, FCD 15288, ISO/IEC 15289, IEEE Std 1228 and the safety and security extensions to CMMI.

• The draft contains requirements and guidance for:– Assurance cases– Associated documents, e.g. assurance plan, reports, analyses

• The process view is comprehensive—it touches every process of 15288 and 12207 (except for the enterprise processes).

• Current balloting:– In SC 7: The New Work Item Proposal and the Committee Draft

are under ballot.– In IEEE: The balloting group is being formed. Immediately

thereafter, the draft will be balloted.

July 2007, D1 18

Outline of Current Draft• Clause 1 describes the scope and purpose of this International Standard and

provides a summary of its contents and their relationships to other standards. • Clause 2 describes how conformance may be claimed. • Clauses 3 and 4 provide normative references and definitions of terms. • Clause 5 is informative material introducing the reader to important concepts used

throughout the international standard. • Clause 6 provides normative requirements upon the assurance case. • Clause 7 provides normative requirements on the contents of an assurance plan. • Clause 8 provides a normative interpretation of many of the life cycle processes of

ISO/IEC 15288 and 12207. • Annex A is normative, providing requirements on various forms of analyses that can

be applied to software elements of the system in support of its assurance case. • Annex B is informative; it gives additional information regarding the software

analyses, specifically related to safety. • Annex C is informative, listing additional standards that may be helpful in addressing

the assurance of security properties.

July 2007, D1 19

Current Draft of Scope Clause• This International Standard provides requirements for the life cycle

including development, operation, maintenance, and disposal of systems and software products that are critically required to exhibit and be shown to possess properties related to safety, security, dependability, or other characteristics.

• It defines an assurance case as the central artefact for planning, monitoring, achieving and showing the achievement and sustainment of the properties and for related support of other decision making.

• The interaction of the requirements for the assurance case with life cycle processes implies a normative interpretation of the processes from ISO/IEC 15288 ISO/IEC 12207.

• Finally, the standard provides requirements, in addition to those of ISO/IEC 15289, for information artefacts that result from those processes.

July 2007, D1 20

Relationships to Other Standards• The provisions regarding process in this international standard make extensive

normative references to ISO/IEC 12207:2007 and ISO/IEC 15288:2007, the international standards for software and system life cycle processes..

• Users of this international standard will probably require risk management and measurement processes that are more fully detailed than the treatment provided in ISO/IEC 15288. Two international standards, ISO/IEC 16085 and ISO/IEC 15939 are useful in this regard.

• The provisions regarding the assurance plan and assurance case are intended to be compatible with the provisions of ISO/IEC 15289:2006 for information items resulting from life cycle processes.

• Some material regarding assurance planning and its supporting analyses has been adapted from IEEE Std 1228:1994.

• The provisions regarding product characteristics are intended to be generally consistent with those of the ISO/IEC 25000 series of standards related to product quality, the ISO/IEC 27000 series of standards related to information security management systems, the IEC 61508 standard on functional safety, and various standards of IEC TC 56 related to dependability.

• ISO/IEC TR 15443, Information technology--Security techniques--A framework for IT security assurance, discusses the need for arguments and evidence in the IT context.

July 2007, D1 21

Current Draft Conformance Clause• This international standard is intended to be used in conjunction with

ISO/IEC 12207 and ISO/IEC 15288. This standard provides requirements and guidance in addition to that of the referenced standards. To conform to this international standard, one conforms to the referenced standards and conforms to the additional requirements of this international standard.

• It is permitted to assert conformance for specified properties of the system or to specified portions of the system if the assertion is accompanied by a clear statement of the limitations ...

• One may assert conformance to this standard for specific product claims related to stated critical properties or characteristics for specifically identified versions of products or portions of products under specified conditions. ... Assertions of conformance that relate only to the lack of limited categories of faults or weaknesses must not be stated as claims for more general properties such as security or safety. ...

July 2007, D1 22

Role of Assurance CaseLife Cycle Processes

Requirements Analysis

Risk Management

Measurement

Project Assessment and Control

Assurance Case

Assurance Issues

Assurance Risks, Threats,

Hazards, etc

Assurance Measurements

Assurance Requirements

Project Planning

Assurance Plan

Transition

Operation

Maintenance

Changes in Operational Characteristics

Maintenance Updates

Operational Constraints

Life Cycle Processes

Requirements Analysis

Risk Management

Measurement

Project Assessment and Control

Assurance Case

Assurance Issues

Assurance Risks, Threats,

Hazards, etc

Assurance Measurements

Assurance Requirements

Project Planning

Assurance Plan

Transition

Operation

Maintenance

Changes in Operational Characteristics

Maintenance Updates

Operational Constraints

July 2007, D1 23

General Requirements on Assurance Cases• The project shall establish and maintain an assurance case.• The project shall ensure that:

– Goals and objectives for safety, security, dependability and any other designated critical properties are formulated.

– Product assurance-related objectives, properties, or characteristics are explicitly selected for special attention and application of this standard to address the goals and objectives.

– Requirements for the achievement of these objectives, properties, or characteristics are defined.

– Measures for the requirements are selected and related to the desired characteristics.– Criteria for the achievement or degree or achievement of these objectives, properties, or

characteristics are selected and traced to requirements.– Approaches for achieving the objectives, properties, or characteristics are planned,

designed, and implemented, as well as demonstrating and documenting that achievement.– The extent of achievement is continuously monitored, documented, and communicated to

stakeholders and managers.– An assurance case documenting and communicating the extent of achievement is specified,

developed, and maintained as an element of the system.– The artefacts for documenting, analyzing, and communicating the required or claimed

properties and characteristics and the extent of achievement are specified, developed, and maintained.

– Requirements of the approval authority are satisfied and necessary licenses or certifications are received.

July 2007, D1 24

Components of an Assurance Case

Evidence

Arguments

Claimssupports

justify belief inQuality / Assurance Case

Make the case for adequate quality/ assurance of the

System, Software, or Work Product

Quality / AssuranceFactor

Quality / AssuranceSubfactor

is developed for

Evidence

Arguments

Claims

Evidence

Arguments

Claimssupports

justify belief inQuality / Assurance Case

Make the case for adequate quality/ assurance of the

System, Software, or Work Product

Quality / AssuranceFactor

Quality / AssuranceSubfactor

is developed for

Adapted from a slide by Joe Jarzombek who, in turn, credited IEEE CS alternative proposal for 15026 and CMU SEI QUASAR tutorial by Donald Firesmith, March 2007