Enabling Success via Applying Modern Software Engineering...

© 2014 Carnegie Mellon University

Enabling Success via Applying Modern

Software Engineering Processes, Methods and

Technologies in the Rapid Acquisition of

Operational Capabilities

Dr. Kenneth E. Nidiffer

Software Engineering Institute

Carnegie Mellon University

Pittsburgh, PA 15213

703-908-1117

26 Annual Software Technology Conference 2014

Meeting Real World Challenges through Software Technology

29 March – 3 April 2013

Long Beach, California

Logogram: Symbol developed by OSD (DCMO) for the DoD IT Acquisition Reform Task Force

2

Meeting Real World Challenges through

Software Technology


2

Why Are IT Software-Intensive Projects Hard to Manage and Lead?

1939’s Science Fiction World of 2000 Actual World of 2000

Software is the building block for modern society – in

fact, the world runs on software!

3


Software Technology


Overview, Scope and Motivation The military will be smaller and leaner, but it will be agile, flexible, ready and technologically advanced*

• Perspective

• The Problem Space

• The Solution Space (Pre-Decisional)

• What Success Looks Like

Sources: Keynote Address, Mr. Alan R. Shaffer, A, SERC, Feb 2014

H.R.1232 - Federal Information Technology Acquisition Reform Act

(passed House – Congress*Government, 28 Feb 2014)

H.R.1232 (2014) - Federal Information

Technology Acquisition Reform Act*

Global Rapid Acquisition

of IT Operational

Capabilities

4


Software Technology


4

Perspective: IT Software Landscape

What are the opportunities?

+ + + + …

Transportation

Infrastructure

Healthcare

Infrastructure

Banking & Financial

Infrastructure

Energy & Utilities

Infrastructure

Communications

Infrastructure

Includes all:

• System of Systems

• Architecture

• Services

• Networked Hardware/ Platforms

• People who digitally connect to

cyberspace

Source: SEI

5


Software Technology


Perspective: Improving Efficiency and Effectiveness in IT/Cyber Acquisitions in DoD

Source: Director, Command and Control, Programs & Policy (OSD) - Pre-Decisional

6


Software Technology


Perspective: Fiscal IT Budget and Reliance

• The federal government reportedly plans to spend at least $82

billion on IT in fiscal year 2014.*

• Defense plans to spend over $39 billion—$5.5 billion on

classified systems, $9 billion on acquisitions, and $25 billion on

operations and maintenance.*

• Deep reliance on commercial infrastructure, services, and

products will grow and is a double-edged sword

Reference: Leveraging Best Practices and Reform Initiatives Can Help Defense Manage Major

Investments. GAO-14-400T: Published: Feb 26, 2014. Publicly Released: Feb 26, 2014.

7


Software Technology


Perspective: A New Reality Global Dimensions Affect IT Science and Technology

Pace of Technology

Rise of the Commons

Expanding Global

Knowledge Base

Information Agility

Mass Collaboration

Economic and S&T Mega-

Trends

Technology Commercialization

Black Swan Syndrome

Source: Dr Reginald Brothers’ chart

Deputy Assistant Secretary of Defense for Research

http://images.google.com/imgres?imgurl=http://www.theage.com.au/ffximage/2007/07/09/JET_narrowweb__300x388,0.jpg&imgrefurl=http://www.theage.com.au/news/national/f35-dearer-than-admitted/2007/07/08/1183833344654.html&usg=__FV2_vq3BLcTTyD3z4bXqp95hObA=&h=388&w=300&sz=18&hl=en&start=29&tbnid=Xukq4Y91Tle0KM:&tbnh=123&tbnw=95&prev=/images?q=air+force+jets&start=20&gbv=2&ndsp=20&hl=en&sa=N

8


Software Technology


Problem Space: Cyber Compared with Other Sciences

PHYSICAL SCIENCE BIOSCIENCE COMPUTER/SOFTWARE/CYBER

SCIENCE

Origins/History Begun in antiquity Begun in antiquity Mid-20th Century

Enduring Laws Laws are foundational to

furthering exploration in

the science

Laws are foundational to

furthering exploration in the

science

Only mathematical laws have proven

foundational to computation

Framework of

Scientific Study

Four main areas:

astronomy, physics,

chemistry, and earth

sciences

Science of dealing with

health maintenance and

disease

prevention/treatment

Several areas of study:

computer science, software/

systems engineering, IT, HCI,

social dynamics, AI

All nodes attached to/relying on

netted system

R&D and Launch

Cycle

10-20 years 10-20 years Significantly compressed; solution

time to market needs to happen

very quickly

Source: SEI

HCI: Human Computer Interaction; AI: Artificial intelligence

9


Software Technology


Problem Space: Developing the Workforce: Recruiting, Training, Education, Retention

The development of cybersecurity professionals is not keeping pace with

the exponential growth of cybersecurity challenges faced by the DoD

and all critical infrastructure sectors.*

Source: SEI

10


Software Technology


Building out capabilities to manage large

information technology projects has been a

sore spot for the Air Force.

Specifically, the service has been challenged

with developing IT acquisition talent among its

ranks, adopting and maintaining processes that

foster best practices and aligning acquisition

and cybersecurity strategies.

Lt. Gen. Charles Davis, the military deputy in the office of the

secretary of the Air Force for acquisition

Problem Space: Human Capital*

Source: C4ISR & Networks, Feb 2014

11


Software Technology


Problem Space: DoD IT Acquisition Cycle-Time - 32

MAIS*

Initial

Operational

Capability Planning Phase

Analysis of

Alternatives

Economic

Analysis

Milestone B

MS C

40

48

5

Test

Build Phase

Development

Cycle-Time Driven by Processes Developed to Counter a Cold War Adversary In Industrial Age Society

43

91

*Source: Defense Science Board Report, March 2009

** Source: Dr. William Scherlis, CTO, CEO

A Modality of Warfare – Software is the Material**

12


Software Technology


Problem Space: The Call For Change

Acquisition

• Long acquisition cycle-times

• Successive layers … built over years

• Limited flexibility and agility

Requirements

• Understanding and prioritizing requirements

• Ineffective role and communications in acquisitions

Test/Evaluation

• Testing is integrated too late and serially

• Lack of automated testing

Funding & Governance

• Program-centric, not capability-centric

• Overlapping decision layers

(e.g., multiple review processes)

• Lack of customer-driven metrics

• Funding inflexibility & negative incentives

13


Software Technology


Problem Space: An Effective Process for Major Defense Systems – But Not Very Agile for IT Systems*

Source: Defense Acquisition University

* Major Defense Systems Life Cycle Management System has Been Updated

to Address IT System Acquisitions Among Other Changes

a

14


Software Technology


Problem Space: Software-Reliant Acquisitions Can Be Difficult to Manage

According to Fred Brooks* software projects are difficult because of

accidental and essential difficulties

• Accidental difficulties are caused by the current state of our

understanding

— of methods, tools, and techniques

— of the underlying technology base

• Essential difficulties are caused by the inherent nature of software

— invisibility - lack of physical properties

— conformity

— changeability

— complexity

Dr. Fred Brooks

* Source: The Mythical Man-Month by Fred Brooks, Addison Wesley, 1995

15


Software Technology


Problem Space: Rate of Technology Development and Adoption Is Growing

15 UNCLASSIFIED 15

High

Low

1980 1985 1990 1995

Sophistication

Required of Actors

Declining

So

ph

isti

cati

on

cross site scripting

password guessing

self-replicating code

password cracking

exploiting known vulnerabilities

disabling audits

back doors

hijacking

sessions

sweepers

sniffers

packet spoofing

GUI

automated probes/scans

denial of

service

www attacks

“stealth” / advanced

scanning

techniques

burglaries

network mgmt. diagnostics

distributed

attack tools

Staging

sophisticated C2

…next?

Increased GIG Complexity

and dependence equates to

lower entry barriers and

potential for increased

number of malicious actors

Sophistication

Of Available Tools

Growing

Defensive measures are outpaced by the well resourced sophisticated threat . . .

Source: DoD

16


Software Technology


Solution Space: Software Engineering Processes, Methods and Technologies – Partial List

•Interim DoD Instruction 5000.02,

Operation of the Defense Acquisition

System, Nov 2013

•IT Body of Knowledge (ITBOK)

•Software Extension to Project

Managers BOK (SWX PMBOK)

•Software Engineering Body of

Knowledge (SWEBOK)

•Helix - Investigating the DNA of the

Systems Engineering Workforce

•Risk Management Framework (RMF)

for DoD Information Technology (DoD

Directives 8500/8510)

•IT Box

•Program Protection Plan (PPP)

•Graduate Software Engineering

Reference Curriculum (GSwERC)

•Body of Knowledge and Curriculum to

Advance Systems Engineering

(BKCASE)

•Software Assurance Community of

Practice (SwA COP)

•Software Engineering Competency

Model (SECOM)

•SE Role-Based Competency

•Joint Competency Experience

Accelerator

•Skills for the Information Age (SFIA)

17


Software Technology


Solution Space: IT is Different from a Weapon System - and Critical to Enable a more Resilient Cyber Environment

•Weapon platform centric

•Military unique requirements

•Development of military-

unique, breakthrough

technologies

•Development cycle of decade

or more

•Production decisions for

unique HW

•Service lives extending into

decades

•Enterprise network

centric

•Adapt commercial

capabilities for military needs

•Leverage commercial

technologies

•Technology cycle 12-18 months

•Procure commodity HW

•Periodic technology refresh to

avoid obsolescence

Weapon Systems IT& Business Systems

DOD Instruction 5000.02

Provides Different Acquisition Processes Sources: IT Acquisition Reform Task Force/MITRE Corporation

18


Software Technology


18

Solution Space: DoDI 5000.02 - 26 Nov 2013

Source: Defense Acquisition University &

DEPSECDEF Interim Policy on 26 NOV

2013

19


Software Technology


Solution Space: Value Recognition of Software Engineering: Top-Paying Majors for New College Graduates in 2012

http://www.naceweb.org/s01232013/top-majors-salary-survey.aspx

20


Software Technology


Solution Space: Refocusing University Curriculums: Alignment of Software and Systems Engineering

System Design

System Analysis

Software (SW) Requirements Analysis

Architectural SW Design

SW Subsystem Testing

Code and Unit Test

Detailed SW Design

System Testing

System Integrated Testing

SW System Testing

SW Integration Testing

SW Engineering SW Engineering

SW Systems

Engineering

SW Systems Engr.

Systems Engr.

SW Systems Engr.

Systems Engr.

Systems

Engineering

Three OSD Initiatives: Graduate Software Engineering Reference Curriculum (GSwERC)

& Body of Knowledge and Curriculum to Advance Systems Engineering (BKCASE)

SW = Software

21


Software Technology


Solution Space: Recognizing the Breath of IT: A New Reality – IT BOK

Scope - Dimensions of the IT Acquisition Space

Source: IEEE, 2014 IT BOK = IT Body of Knowledge

22


Software Technology


Solution Space: Information Technology (IT) Box

Applications &

System Software

Development &

Acquisition

Requirements Organization &

Oversight

Capabilities

Required $

$

Hardware

Refresh &

System

Enhancements

& Integration

JROC

Approved

IS ICD*

• Information Systems Initial Capabilities Document (ICD)

• Requirements Definition Package Sources:

Katrina McFarland, DoD ASD, C4ISR, 28 Feb 2014

•CJCSI 3170.01H, 10 Jan 2012; JCIDS Manual, 19 Jan

2012, DAU

Joint Concepts

Capabilities Based

Assessment

Strategic Guidance

MS A/B

O&S Engineering Analysis/ Design

ICD

(NF) Rapid Delivery

Full Deployment Decisions

IOC

CD

Agile Development

CD CD CD CD CD CD CD CD

RDP*

23


Software Technology


Solution Space: Software Project Management Software Extension to the Project Management BOK

Source: Software Extension to the PMBOK® Guide Fifth Edition

24


Software Technology


Solution Space: Improvements in Human Capital

•Software Engineering

Competency Model (SECOM)

•SE Role-Based Competency

Helix - Investigating the DNA of the Systems Engineering Workforce

Skills for the

Information Age (SFIA)

25


Software Technology


Solution Space Example: Focus on Software Assurance

Image Source: www. technobuffalo.com

The level of confidence that software

functions as intended (and only as

intended) and is free of vulnerabilities,

either intentionally or unintentionally

designed or inserted as part of the software

throughout the lifecycle*.

* Source: DoDI 5200.44 Protection of Mission Critical Systems to Achieve Trusted Systems and Networks

(TSN), November 5, 2012

26


Software Technology


Solution Space: Cybersecurity Policy Alignment

CNSSP 22

IA Risk Management Policy for

NSS

Knowledge Service

DoDI 8500.01 “ Cybersecurity”

IT Definitions

Security Controls Guidance

Enterprise Governance

DoDI 8510.01

“Risk Management Framework

for DoD IT”

NIST SP 800-39

Managing Information Security

Risk

NIST SP 800-37

Risk Management Framework

NIST SP 800-30

Risk Assessment

NIST SP 800-53

Cybersecurity Controls and

Enhancements

CNSSI 1253

Categorization

Baselines

NSS Assignment Values

CNSSI 1253A

Implementation and

Assessment Procedures

CNSS 4009

Information

Assurance/Cybersecurity

Definitions

NIST SP 800-53A

Cybersecurity Control

Assessment Procedures

NIST SP 800-137

Continuous Monitoring

NIST SP 800-60

Mapping Types of Information

to Security Categories

NIST SP 800-160 (DRAFT)

Security Engineering Guideline

DoD NSS NIST

26

27


Software Technology


Solution Space: DoDI 8510.01* “Risk Management Framework for DoD IT” - Adopts NIST’s Risk Management Framework, Used by Civil and Intelligence Communities (* Target Publish Date: 2Q FY14)

27

28


Software Technology


Real-time

Modification of

Systems

2: Methods of Secure

Systems Development

1: Foundations for Software

Assurance

3: SwA Management &

Operation

Tailored

Trustworthy

Spaces

“Science of

Security”

Software

Composability

Digital Curation and

Forensics

Modeling,

Simulation, Testing &

Certification

Architecture for

Secure Systems

Domain Specific

Assurance

Mitigations

SwA for Agile

Software

Methodologies

Metrics

Using Big Data

Analysis to

Advance

Software

Assurance

Techniques

SwA

Economic

Incentives

SwA Core

Competencies,

Education &

Training

SwA

Workforce

Development

SwA in Highly Parallel,

High-Performance

Computing Environments Security in Socio-

technical

Computing Security of Mobile

Applications &

Platforms

Designing Secure

Cyber-Physical

Systems Critical Infrastructure

Resiliency & Catastrophic

Recovery

Electronic

Effects in SwA

Effective

Acquisition Policy

& Guidance

Vulnerability

Prevention and

Detection Tools &

Techniques

Supply Chain

Visibility

Cultivating SwA

Maturity

5: Critical Infrastructure

4: Emerging & Disruptive

Technology

Scaling of

Assurance

Techniques

Solution Space: Defining Areas of Research – Focus on Software Assurance

Intrinsic Internet

Infrastructure

Security

29


Software Technology


Solution Space Example: SEI’s Software Assurance Capabilities

Secure Coding

• Coding standards in Java, C,

and C++

• Source Code Analysis

Laboratory (SCALe) to test

software applications for

conformance

30


Software Technology


What Success Looks Like The military , although smaller and leaner, will be agile, flexible, ready and technologically advanced

Source: Director, Command and Control, Programs & Policy (OSD) – Pre-Decisional

31


Software Technology


Questions?

32


Software Technology


Contact Information

Dr. Kenneth E. Nidiffer, Director of Strategic Plans for

Government Programs

Software Engineering Institute, Carnegie Mellon University

Office: + 1 703-908-1117

Fax: + 1 703-908-9317

Email: [email protected]

33


Software Technology


NO WARRANTY

THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE

MATERIAL IS FURNISHED ON AN “AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY

MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO

ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR

PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM

USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY

WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT,

TRADEMARK, OR COPYRIGHT INFRINGEMENT.

Use of any trademarks in this presentation is not intended in any way to infringe on the

rights of the trademark holder.

Requests for permission to use or reproduce should be directed to the Software

Engineering Institute at [email protected].

This work was created in the performance of Federal Government Contract Number

FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software

Engineering Institute, a federally funded research and development center. The

Government of the United States has a royalty-free government-purpose license to use,

duplicate, or disclose the work, in whole or in part and in any manner, and to have or

permit others to do so, for government purposes pursuant to the copyright license under

the clause at 252.227-7013.

Date post:	16-Dec-2018
Category:	Documents
Upload:	vucong
View:	224 times
Download:	0 times

Enabling Success via Applying Modern Software Engineering...

Documents