System Engineering Next Generation T&E

Jack RingEduce LLC

jring7@gmail.com

Attributed reuse permitted

TheFit For Purpose

Paradigm

ITEA SoS Workshop: The NIE Experience

El Paso, TX, July 17, 2013

Intended Takeaways

The tenets of Test & Evaluation currently taught and practiced are not sufficient for net-centric SoS, especially at brigade scale.

Key gaps are:a) In-field modeling of the anticipated engagement and

formulating a ConOps for the warfighter’s intervention system.

b) System Design and Engineering of a responsive T&E system.

c) Inadequate SoSSE and Acquisition policies and practices.

Next Step: Develop a cadre of T&E Systemists.

04/08/23 jring7@gmail.com, attributed reuse permitted 2

You don’t know JackYou don’t know Jack

1957 – Present. GE 20, Honeywell 10, Edelbrock 3, Ascent Logic 2, IBM OTP 1, Entrepreneur 20. Kennen Technologies LLC, OntoPilot LLC, Educe LLC.

System Test & Evaluation (Atlas ICBM Radio Guidance System) System Engineering (State-determined Stochastic Non-deterministic Systems).

Inaugural chair, 1970, GE-wide workshop on Software Engineering. Led SAFE system concept definition for multi-agency federation of

intelligence collection, analysis and production. Product Manager, Distributed Transaction Processing products suite. Led Computer Integrated Manufacturing shop floor control network demo

involving 20 vendors using GM MAP protocol.

?

SME: Autonomous System T&E technologies. Tutorials & Papers; ITEA, INCOSE, INCOSE IL, ISSS,

IEEE-SMC, IEEE SysCon, ICSEng, NIST. Patent co-author, General Purpose Set Theoretic

Processor Architecture and Method.04/08/23 3

The Net-Centric Paradigm

Anticipated Engagement

Warfighter’s Engagement

Model

Net-centricSoS

SoSRealized & Deployed

SoSDesign

WarfighterAction

Is this a system?Will it be Fit For Purpose?Is it still Fit For Purpose?Conduct

T&E Sessions

SysEng T&E System

Realize T&E System

Red Force, Blue Force, Green Force, Grey Force

Predictable + Adaptive + Autonomous assets

SoSSEPraxis(i)

ActionableKnowledge

Lead Time?

Adaptation

Cycle Time

04/08/23 jring7@gmail.com, attributed reuse permitted 4

Warfighters Deserve To Know

Is This System FIT FOR PURPOSE, F4P?

Is This System Still FIT FOR PURPOSE, F4P?

POSIWID: the purpose of a system is what it does, regardless of designer or operator intent.

The Mean Time to Configuration Change. MTTCC, of a brigade-scale system

may be < 15 minutes. The F4P test must be run at MTTCC/2

04/08/23 jring7@gmail.com, attributed reuse permitted 5

Current T&E Situation

Gap: Inadequate warfighter-trusted knowledge regarding the dynamic and integrity limits of multi-node networks of heterogeneous, autonomous systems.

Status: Current T&E, e.g., NIE, is a) 10X too expensive and time consuming and b) Does not determine Fit For Purpose.

Talent: T&E community competencies have dwindled toward instrumentation and data technicians. Does not leverage NCO knowledge and learning.

Remediation: The current resurgence of system engineering and system of systems engineering does not provide the necessary facts.

04/08/23 jring7@gmail.com, attributed reuse permitted 6

Why Beyond T&E?

Current T&E < <Factor > > Beyond

System of Interest Focus Effect of SOI on larger system

Does system meet Requirements?

Question Will SoS be Fit for Purpose(s)?

DataTENA

Key Capability System modelingDesign of Experiment

Errors, Operational Capabilities

Findings Loci of dynamic and integrity limits

Data Work Product Actionable Knowledge

Acquisition Serves Warfighter04/08/23 jring7@gmail.com, attributed reuse permitted 7

WarfighterConOps

AnticipatedEngagement

6SystemRealization

& ResilienceInventory of Components

54

2Effects

CapabilitiesMOE’s

OperationalSystem

Engineering

3

Acquisition

ApplicationKnowledge

Base

ProblematicSituation

1

Operatorse.g.,

Warfighters

F4P?7Y

N

04/08/23 jring7@gmail.com, attributed reuse permitted 8

SoS Interoperability Principles

• A system exists only when deployed, activated and responding to stimuli.

• Entails orchestration of “N” self-adapting systems that exhibit the desired effects.

• The dynamic and integrity limits of any system are determined bya) Progress properties: starting from some state reach a

desired state in a finite number of steps.b) Safety/Integrity properties: maintain certain states that

always insure correct progress.

04/08/23 jring7@gmail.com, attributed reuse permitted 9

Key Gap Closing Actions

1. Engagement Modeling

2. SE of T&E of SoS F4P

3. Develop a cadre of T&E Systemists

04/08/23 jring7@gmail.com, attributed reuse permitted 10

1. Engagement Modeling• Warfighters, envision their intervention system by deciding

a) the intended Effects, Measures of Effectiveness and Capabilities of their intended SoS and b) the likely evolution as interaction progresses and c) update their SoS model at the pace of engagement or resource change.

• Warfighters need an modeling method and tool to clarify the explicit and implicit effects of the problem system and intervention system on each other, notably [Input/Output, Performance, Technology, Tradeoff Gradients, Testability, and Cost]*

• Warfighters need an asset knowledge base provided by the Acquisition community.

* Model-based System Engineering by A. Wayne Wymore, CRC Press, 199504/08/23 jring7@gmail.com, attributed reuse permitted 11

1.1 The Warfighter SoS Model1.1 The Warfighter SoS Model

Intended: Emergence & Prevention of Emergence

The truth, the whole truth,

and nothing but the truth.

InformaticsThermodynamics

BiomaticsTeleonomics

Social DynamicsEconomicsEcologics

Model must be directly executable. based on a formal ontology.reflexive.

Minimal Implicate Order04/08/23 jring7@gmail.com, attributed reuse permitted 12

WHENWHO

WHERE

WHAT

HOW

“Day 2”of SoS Projects Until End of Life At speed of SoS evolution

Actionable Knowledge

DevelopmentField Ops

Training Exercises in situ (Battlefield)

Whole system span, e.g., DOTMLPFThird Generation SENon-deterministic SoS’s

With WarfightersBy Purple-suiters

And Civilian SME’S

Intended Outcomes: A new T&E paradigm. Enthusiasm to Transition. Justifiable budgets.

Joint OperationsBrigade-scaleWarfighters

WHY

More with less

Knowledge Discovery/Usage

New User Paradigms

Adequate, Accurate, Timely Trusted, Cyber-AssuredIndependent & ObjectiveSimple Autonomous

Engagement-specific Physical & Cyber T&E enterprise

Many Kinds of T&EFamily of T&E systems

PEOsPMs

Test Range Executives

FAA, FCC, DHSAllies, Congress

2. SE of T&E

Of F4P

2. SE of T&E

Of F4P

04/08/23 jring7@gmail.com, attributed reuse permitted 13

2.1 Is Each Stage of the Warfighter SoS viable?

1) Ensure Acquisition SoSSE a) describes progress properties and safety/integrity properties and b) Includes sufficient self-test in components.

2) Leverage new technology that automates System Readiness Assessment: a) Software problems: Reduced to ≈ 1% of current.b) System Integration cost/time: Reduced to ≈ 20% of

current experience.c) Test abort delays: Reduced by ≈ 40% of current T&E

experience.d) Cybersecurity: Reduced vulnerability.

04/08/23 jring7@gmail.com, attributed reuse permitted 14

2.2 Is the Warfighter SoS Fit For Purpose?

• Confirm that anticipated and demonstrated dynamic and integrity limits of warfighter’s SoS are necessary, sufficient and efficient.

• Apply from Day 2 of a new engagement project through Year N of the SoS usage/evolution cycle --- as often as MTTCC/2.

04/08/23 jring7@gmail.com, attributed reuse permitted 15

2.3.a Is the SoS Personnel Subsystem Systemic?

= U * P * C where; U = degree of understandingP = level of role proficiencyC = level of collaboration ability

Beneficiaries

Supply

Operators

DB Admin

Beneficiaries

Supply

Operators

DB Admin

Z = model of the local system of interest = fidelity of Z to S

Config Admin

In-service Eng.

Dev.Eng.

Systems Eng.Z(S1)

Config Admin

In-service Eng.

Dev. Eng.

Systems Eng. Z(S2) J. Ring & A. Madni, “Key Challenges in SoS Engineering” 2005 IEEE International

Conference on Systems, Man and Cybernetics, Waikoloa, Hawaii October 10-12, 2005

Node 1 Node (i)

04/08/23 16

2.3.b Are SoS-wide Changes Coherent?2.3.b Are SoS-wide Changes Coherent?

X, d(X)/dt, d2(X)/dt2

Thermodynamics: mass, momentum and energyInformatics: data, information and knowledge Teleonomics: skills, rate of learning, and rate of invention Human social dynamics: trust, enthusiasm, co-evolutionEconomic: Investment, ROI, Liquidity Ecology: Waste, Fads, Unintended Consequences

Consistent with Conservation Laws?Consistent with Conservation Laws?

Adjust: Gradients on relationshipsArrange: Pattern of relationshipsCo-align: Content of system with context and resources.

1704/08/23 jring7@gmail.com, attributed reuse permitted

3. Develop a cadre of T&E Systemists

c) Executable models enable system composition. d) Ensures requisite information from acquisition programs.e) Accelerates co-learning of all involved.

a) Conduct 12-15 person expeditions that implement ConOps. 3X100 day cycles. 20 teams in five years.

b) Adept at all degrees of Extent, Variety, Ambiguity.

04/08/23 jring7@gmail.com, attributed reuse permitted 18

3.1 Use Work Program of Complexity

Discovery:

Resolution:

Diagnosis is done by an experienced individual professional, who iterates with the group until the description is fully understood and accepted.

UNDERSTANDING COMPLEXITY: THOUGHT & BEHAVIOR, 2002, www.jnwarfield.comA HANDBOOK OF INTERACTIVE MANAGEMENT, 1990/1994, www.jnwarfield.com

Implementation is carried out by whatever means the design specifies.

Description is done in a group process. focus on problematic situation and underlying problem system.

Design is done in a group process. involves both formal logic and behavioral pathologies.

04/08/23 jring7@gmail.com, attributed reuse permitted

19

3.2 Create Self-synergizing SystemistsRelationship Meaning Mediators

Co-evolve Morphing toward

Win-Win-Win

Joy-enabled Level of Consciousness

Co-facilitate Value Out/Value In ≈ eN Stewardship by N participants

Co-learn Meaningful reflection Shared knowledge claims

Collaborate Help one another Desire to serve

Co-celebrate En-joying one another Time & Space, F2F

Cooperate Compatible Actions Willing to wait

Commit Principled relationship Courage to plan

Converge Common compelling purpose

Shared self-respect

Communicate Share interests and values Common language

Connect Two discover one another Accessible attributes

04/08/23 jring7@gmail.com, attributed reuse permitted 20

3.3 Encourage Reflective Practitioners3.3 Encourage Reflective Practitioners

Four ascending levels of behavior: •Know how.

•Reflection -- on how 'know how' was applied.

•Knowing-in-action (devising while doing)

•Reflection-in-action.

A Practitioner must have two kinds of knowing:• Objectivist - descriptive• Constructivist – prescriptive - world making

Designing cannot be taught -- but can be coached• Joint experimentation. • Follow Me!• Hall of Mirrors.

Educating the Reflective Practitioner, Donald Schon, Jossey Bass, 1987

04/08/23 jring7@gmail.com, attributed reuse permitted 21

Recap: F4P Paradigm Benefits

1) Free. Return >> Investment.2) Consistent with DoD 5000.2 (effects, capabilities).3) Consistent with Swarms of Autonomous Systems.4) Benefits Acquisition community as well as warfighters.5) Dynamic and integrity limits inform Design of Experiments.6) Effective in at least defense, aviation, homeland security,

industry supply chains, knowledge discovery/vetting networks, and human activity systems.

7) Re-orients and prepares system engineering personnel to leverage next generation technologies.

04/08/23 jring7@gmail.com, attributed reuse permitted 22

04/08/23 jring7@gmail.com, attributed reuse permitted 23

Clarifications?Questions?Comments?

The castle, Jack,

besiege the CASTLE!

Thank you!