APPROVED FOR PUBLIC RELEASE APPROVED FOR PUBLIC RELEASE 1 APPROVED FOR PUBLIC RELEASE APPROVED FOR PUBLIC RELEASE Utilizing Network Emulation to Provide Cost Effective Large Scale System of Systems Test and Evaluation 20 Jan 17 CERDEC S&TCD MODESTA Team Data Collection, Analysis, Modeling and Simulation Branch System Engineering, Architectures, Modeling and Simulation Division DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Transcript
APPROVED FOR PUBLIC RELEASE
APPROVED FOR PUBLIC RELEASE
1APPROVED FOR PUBLIC RELEASE
APPROVED FOR PUBLIC RELEASE
Utilizing Network Emulation to Provide Cost Effective Large Scale System of Systems Test
and Evaluation
20 Jan 17
CERDEC S&TCD MODESTA TeamData Collection, Analysis, Modeling and Simulation Branch
System Engineering, Architectures, Modeling and Simulation Division
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
APPROVED FOR PUBLIC RELEASE
APPROVED FOR PUBLIC RELEASE
2
Note: notional architecture and systems displayed
As each system contains multiple sub-systems, including Comms, Computers, and EW, the task of engineering, integration and testing can be very complex
Integration of desperate systems Interleaving systems engineering tasks Many voluminous data sources and sinks Accomplishing the required testing and
Challenges to SoS Engineering, Integration, and Testing
APPROVED FOR PUBLIC RELEASE
APPROVED FOR PUBLIC RELEASE
3
Network Architecture
Traffic Profiles
Operational Scenarios
Force Structure / Unit Roles
Environment / Terrain
Data Collection /Metrics
System Under Test / Evaluation
User Actions / Interaction
COMPLEXITY
+
RESOURCES
• Expertise needed in the areas of• Tactical Radios• Networks• Applications
• Integration of complex data from multiple sources
• Complex tools, each needing their own subject matter experts
• Unique configuration requirements• Non-trivial maintenance and
operation
High Degree of Difficulty
For each experiment, a complex working group dynamic must be in place:
APPROVED FOR PUBLIC RELEASE
APPROVED FOR PUBLIC RELEASE
4
SoS Evaluation Trade Space
• The desired SoS Evaluation is high fidelity and scalable.• Traditionally takes a considerable amount of time and money.
– Time to setup the simulations (develop/configure inputs) and execute scenarios– Traditionally, high fidelity models tend to run slower than real-time.
Behavioral Discreet Event Simulation:-Highly abstracted system models-Models attempt to capture notional behavior-Detailed models require excessive time/cost-Notional traffic-Higher fidelity des models typically don’t scale well-Difficult to interface with live systems
Shared Code Model / Real-time Emulation-Actual code take from live systems-Cross-compiled into lightweight VMs-True ethernet frame/IP packet processing-Locked to real-time execution-Highly scalable on commodity hardware-Interfaces with system at IP layer
SDR, Multifunction Waveforms, Embedded Security,
QoSMechanisms,
Complex Routing
Algorithms
Large Tactical Networks, Soldier to Brigade,
Thousands of Nodes,
Enterprise to Tactical,
Many to Many Interfaces
Live Testing-Live systems provide true performance characteristics-Lives systems allow full interaction with other systems-High costs of personnel, ranges, resources-Challenging data collection-Typically all-manual configuration
APPROVED FOR PUBLIC RELEASE
APPROVED FOR PUBLIC RELEASE
5
Experimentation Fidelity
• Simulation provides high scalability and moderate costs while typically sacrificing realism due to a high level of abstraction
• Live Testing provides true performance and usability characteristics, but does not scale well and can be very costly
• Real-time Emulation provides a middle ground where system software (waveform, router, and application shared code) is executed in virtual machines to allow for scalability and high fidelity
Realism Scalability Abstraction Cost
Simulation Low / Med High High Low
Emulation Med / High High Med Low
Test (Live) High Low Low High
Realism
Scal
abili
ty
Test (Live)
Emulation
Simulation
APPROVED FOR PUBLIC RELEASE
APPROVED FOR PUBLIC RELEASE
6
System of Systems Emulation:MODESTA
• The Modeling, Emulation, Simulation, Tool for Analysis (MODESTA) is a CERDEC developed government purpose rights framework for conducting systems of systems network analysis.
– Allows for disparate models, such as the shared code emulation models ported into EMANE and the behavior models developed in JNE, to be deployed in large scale for realistic network effects analysis.
– High Fidelity Representation of Multiple Layers Including Physical Connectivity, Network, Transport, and Application
– Integrates live hardware when available (e.g. HMS, WIN-T, MNVR systems).– Allows stimulation of network with live C4ISR systems (e.g. JBC-P, CPOF) and/or generated traffic.– Provides the ability to deploy low fidelity surrogate networks when high fidelity is not available or
unnecessary.
• MODESTA allows high fidelity analysis within realistic timeframes
– Reduces data import times up to 50% through advanced data parsers– Reduces model configuration up to 50% by automatically configuring models based on imported
BOIs and standard waveform parameters across disparate emulation environments– Provides an ability to quickly iterate through different courses of actions/network configurations
MODESTA provides capability to construct large scale, high fidelity scenarios to provide detailed and scalable analysis.
The Extendable Mobile Ad-hoc Network Emulator (EMANE) models the radio and SATCOM networks
EMANE’s PHY/MAC model Models path loss with antenna pointing Allocates timeslots to emitters Connects to preferred neighbors first Varies data rate and latency per neighbor
EMANE’s Comm Effects model imposes fixed data rate/delay for satellite
EMANE exposes virtual L2 “Tap” network interfaces that behave as the Ethernet port on an radio or SATCOM modem
MAC
PHY
MAC
PHY
APPROVED FOR PUBLIC RELEASE
APPROVED FOR PUBLIC RELEASE
8
Network Emulation Testbed –Linux Containers
Linux containers Process groups isolated from the rest of
the system to some degree (lightweight virtualization)
Takes a different approach than system virtualization (VMWare)
A container (optionally) uses a network namespace – an isolated network stack containing its own interfaces, routing table, neighbor table, netfilter rules, etc
Containers in the emulation allow multiple ciphertext and plaintext routers to co-exist in the same operating system instance
* Lightweight Virtual Linux Container
MAC
PHY
MAC
PHY
*
APPROVED FOR PUBLIC RELEASE
APPROVED FOR PUBLIC RELEASE
9
Tactical Network Emulation Deployment on HPC Compute Nodes
• To achieve Brigade level scalability of live/virtual tactical nodes:– Define Scenario– Allocate DoD Supercomputer Resources at Army Research Lab partner– Configure Hardware and Software on the Fly– Execute Test in Real Time– Reduce, Analyze Collected Data– Tear Down and Sanitize Resources Making Available for Next Queued Test
Define Test / Scenario
Configure Hardware ExecuteTest
Sanitize Resources
Allocate Resources Configure Software Reduce & Analyze Data
