TENA in a Resource Constrained
Environment (TRCE) - Enabling
Distributed Testing Across Wireless
and Embedded Computing Domains
ITEA T&E System of System Conference El Paso, TX
January 24-47, 2011
Thomas Treakle – Principal Investigator [email protected]
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
The authors would like to thank the Test Resource
Management Center (TRMC) Test and
Evaluation/Science and Technology (T&E/S&T) Program
for their support. This work is funded by the T&E/S&T
Program through the Netcentric System Test (NST) focus
area under contract with PEO STRI, Orlando, FL, contract
W900KK-09-C-0013.
Any opinions, findings and conclusions or
recommendations expressed in this material are those of
the author(s) and do not necessarily reflect the views of
the Test Resource Management Center (TRMC) Test and
Evaluation/Science & Technology (T&E/S&T) Program
and/or the U.S. Army Program Executive Office for
Simulation, Training & Instrumentation (PEO STRI)
Acknowledgement
2 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
TRCE Project Overview
• The Test and Training Enabling Architecture
(TENA) is the interoperability standard for the
DoD test and training ranges
• TRCE is developing technologies to extend the
applicability of the TENA Middleware to support
enterprise use cases
– Support a broader range of network types/topologies
including wireless networks
– Support for small size, weight, and power (SWaP)
computational devices and smartphones
• Currently in the 3rd year (Phase 4) of project
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. 3
TRCE OV-1
4
TRCE is extending TENA support for real-time connectivity to all T&E systems
and instrumentation throughout the enterprise. DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
TENA in Resource Constrained Environments
(TRCE) Project Addresses These Issues
Non-TENA Applications
RangeResource
Application
Reusable
Applications
Reusable
Applications
Non-TENA Communications
TENATENA
Range ResourceApplication
Data
Collectors
HWILHWIL
RangeResource
Application
Repository
Utilities
TENAObject
TENAObjectTENA
Object
Infrastructure
Management and
Planning Utilities
Object Model
Utilities
TENA Utilities
TENA Common Infrastructure
TENA Applications
Non-TENA System
Non-TENA System
I S R F o r c e M i x S t u d y
S h a d i n g i s : P h a s e
TENA Tools
GatewayGateway
TENA MiddlewareTENARepository
TENA MiddlewareLogicalRangeData
Archive
5
- Improve TENA’s
support for
variable quality
and low data rate
networks
including wireless
- Expand TENA’s
support for
handheld and
embedded
instrumentation
computational
platforms
• Low Data Rate Networks – TENA must establish and maintain data
connections on low data rate networks
– Need to optimize use of low data rate networks to support operational scenarios
• Wireless Networks – Current range environments use
wireless links extensively for various systems under test
• Variable Quality Networks – T&E systems poorly tolerate high
loss, link failure, or heterogeneous links
– Need to provide data continuity for degraded or heterogeneous networks
• Hardware constrained devices – Low power, reduced CPU, reduced
memory
– Embedded instrumentation support
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
TRCE Project Overview (cont.)
• Phase 1: Develop use cases and prototype technologies
– Link Node Controller (LNC) prototype to investigate constrained link capabilities
– Develop TENA performance characterization testbed with network emulation capability
• Phase 2: Mature link optimization/management technologies & develop
TENA mobile ports
– Initiate development of the RelayNode as part of the TENA Middleware
– Prototype mobile device TENA ports (iOS, Android, Gumstix Overo)
– Initiate transition of non-intrusive hardware/software sensor data acquisition system to
TENA
• Phase 3: Test technologies in relevant TRL-6 environment at a single range
– Build TENA Middleware on required devices (iOS & Overo)
– Build auto-generated RelayNode 1.0 capability
– Execute test at Aberdeen Test Center
• Phase 4: Work to mature technologies and transition to the TENA program
– Build framework for full TENA RelayNode capability within the middleware
– Transition mobile device ports with sample applications (iOS, Overo, & Android)
– Demonstrate non-intrusive TENA enabled embedded hardware / software data acquisition
system
– Perform multi-range test with developed technologies in TRL-6 environment
6 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
RelayNode 1.0
• Auto-generated application that will support a wide range of object models
• Can be deployed at strategic points geographically on the LAN/WAN
• Supports each device connection in separate thread
• Replicates TENA API as closely as possible
7 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Relay Node 2.0 (in progress)
• Will be Fully integrated capability within the TENA Middleware – Limited capabilities initially such as eliminating redundant
transmissions and potentially store-and-forward
– Object Model agnostic capabilities implemented first
• Proper implementation requires restructuring and enhancement of core TENA Middleware components (ongoing and significant effort)
• Will provide a framework to support a wide range of new TENA capabilities – User authentication / access control
– Local data archiving and store/forward queuing
– Heterogeneous network link monitoring and management
– Encryption and data protection functionality
• Incorporate functionality of the Execution Manager at each RelayNode – Single EM model becomes obsolete
– Approach provides improved event management possibilities
8 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Sample Relay Node Deployment
9 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Platform Ports of TENA
Middleware to Hardware
Constrained Devices
iOS TENA Port • Full Native TENA Port Complete
– Platform and all its dependent OMs & PRITEC Measurand completed
– Tested with Fedora 14, OS-X, Overo, and Windows 7 connectivity
• Beta Group Registration required for access (Dec. 2011)
– No automatic OM build support
– Not integrated with TENA testing harness (not fully tested)
• Support
– iOS 4.x and 5.x support with Xcode 4.2
– Universal armv6/armv7, Simulator x86
• Current Platform Designators
– ios40-xcode32-x86-d, ios40-xcode32-x86
– ios40-xcode32-armv6-d, ios40-xcode32-armv6
• TENA Standard OM Support initially
– Provide example application with Platform Publish/Subscribe
capabilities
– Users interested in additional OM support may request through
Helpdesk (support is very limited)
11 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
iOS Platform Demo App
• Thin layer in application allows for runtime selectable support of TENA or RelayNode wire protocols – RelayNode wire protocol necessary for 3G/4G connections
without VPN due to cellular network topology
– TENA wire protocol supported on 3G/4G with VPN
• Supported Hardware – iOS 4.x - iOS 5.+
– iPhone (3gs, 4, 4s), iPod, iPad (1 & 2)
• Can support static and dynamic mapping tiles
• Support subscription to TENA Video Distribution System (TVDS) feeds – Live video displayed within App
– Control of Pan/Tilt Cameras
12 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
iPad Demo App • Mapping and display of TENA
Platforms (static and dynamic
maps)
• Supports Pub/Sub of TENA-
Platform and EventTeam
AVStream, CameraControl,
RangeSafetyWarning OMs
• Display of TVDS Video Streams
• WiFi Connectivity
– Directly to TENA Execution
– Via RelayNodes
• 3G/4G Connectivity
– Via RelayNodes using reduced
bandwidth protocol
– When connected through VPN • Directly to TENA Execution
• Via RelayNodes
13 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
iPad Demo App
14
• Native iPad / iPhone
look & feel
– Pinch zoom
– Pin dialogues
– Google mapping
support
• Platform Listing with
zoom-on-select
• Popup dialogues
show detailed
Platform specifics
• Indicators for
“attached” video
feeds from TVDS
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Gumstix Overo
• Computer on Module
– Arm Cortex A-8 CPU @ 600MHz
– 256 MB memory, MicroSD card, Bluetooth
and WiFi on the module
– Expansion boards:
• Summit: USB master, wired 100MB Ethernet,
HDMI out (can do 1024x768 60 HZ), stereo sound
in/out, GPS, 4.3 inch touch display
• Pinto: USB OTG port, solder connections for
GPIO and ADC input/output
15 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Gumstix Overo • OS/compiler
– Openembedded Linux, kernel 2.6.35, • full X11, ssh, media encoding/decoding
– Compiler gcc 4.3.3 based cross compiler
• Full native TENA port complete
– Platform and all its dependent OMs & PRITEC Measurand completed
– Tested with Fedora 14, OS-X, iOS, and Windows 7 connectivity
• Beta Group Registration required for access (Dec. 2011)
– No automatic OM build support
– Not integrated with TENA testing harness (not fully tested)
• Bluetooth and wired sensor support
– Connected to GPS & multiple IMUs via Bluetooth link or serial connections
– Published values via TSPI and Measurand OMs
• TENA Standard OM Support initially
– Provide example application with Platform Publish/Subscribe capabilities
– Users interested in additional OM support may request through Helpdesk (support
is limited)
16 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Android • NDK Version: android-ndk-5b
• API Level: 2.2+ (Froyo)
• Full C++ support for:
– TENA Middleware v6.0.2
– All Standard Implementations
– All TENA Object Models
– Execution Manager
• Java Binding (in progress)
– Working in partnership with TENA project
– Using SWIG to auto generate bindings
– Partial coverage of TENA Middleware API
– Partial coverage of Standard TENA Object Models
• Demo App
– Tested with API 2.2 (Foyo) and 2.3.4 (Gingerbread)
– Uses prototype Java Binding
– Sends position via TENA-Platform OM
– Display list and map of other Platforms
– Fully TENA connections with WiFi or 3G with VPN
– RelayNode support in process 17 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Mobile Device Path Forward • iOS and Overo Ports will be released to TENA community as 6.0.2 Beta
platforms
– Limited testing of ports has been conducted (not integrated into the TENA
comprehensive build/test process)
– Only provide support for TENA standard Object Models initially
– Example applications and documentation will be provided
– Anticipate release December 2011
• Android Port will be delayed until the formal TENA Java Binding is completed
(estimate early-mid 2012)
– Java Binding is required to provide comprehensive User Interface capabilities on
Android
– Example applications and user documentation are being developed
• Work with interested ranges to develop personnel tracking capabilities to
support range situational awareness
– Inclusion of safety fan or “keep out” regions within smartphone mapping displays
– GPS only trackers with ~1Hz update rates
• Fully integrate mobile platforms ports into TENA build/test harness
– Auto-generate OMs and execute full regression & performance tests
• Full support for OASIS / EISA capabilities
18 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Migration of Capabilities to TENA
Embedded Instrumentation System Architecture (EISA)
and
Open-Source Architecture for Software Instrumentation
of Systems (OASIS)
EISA / OASIS Migration to TENA
• Bring Previously Prototyped T&S/S&T Project
Capabilities to TENA Community
• Goals are to Provide:
– Non-intrusive sensor data acquisition architecture
– Provide real-time sensor data and sensor-specific
metadata to analysts
– Support for hardware and software sensor probes
– Support for hierarchical sensor groups (derived data)
– Localized data storage with remote query
– Standardized sensor data models for community use
– Centralized or distributed management of data
acquisition process
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. 20
OASIS / EISA System-of-Systems
Case Study
21
T
E
N
A
T
E
N
A
T
E
N
A
T
E
N
A
TENA
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Objectives of Critical Actors in OASIS/EISA
SW
Sensor
SW
Sensor
HW
Sensor
HW
Sensor
Software Probes: Collects application and system metrics
Hardware Probes: Collects physical sensor data (temp, pressure, accelerations, etc.)
DAC
Commodity HW Resource Constrained HW
Data Extraction
TENA <Private>
OASIS OM
EINode HW
Probe
SW
Probe
SW
Sensor
HW
Sensor
EINode Manages Probes
TENA Sensor
OMs
Commands
Commands
EINode:
• Sensor/transducer interface support
• Physical data aggregation
• Synthetic data aggregation
• Data preprocessing and buffering
• Regulates network transmission of
collected metrics
Embedded Instrumentation External Connections
Network Boundary
22 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Data Acquisition Controller (DAC):
• Data aggregation from multiple EINodes
• Static probe meta-information (model, SN,
manufacturer, calibration, etc.)
• Data access control and local data storage
• System configuration management
• Interface with the outside world
EISA / OASIS:
Data Acquisition Approach
23
Layering of DACs & EINodes allows for improved sensor management
by facilitating logical sensor groupings to include virtual composite sensors.
Propagates Appropriate Probe Manufacture Specs to Analysts
DAC-0
DAC-4 DAC-3 DAC-2 DAC-1
EISA / Oasis Infrastructure
Real-time
Management &
Monitoring
Offline
Analysis
EIN-4 EIN-3 EIN-2 EIN-1 EIN-4 EIN-3 EIN-2 EIN-1
EIN-4 EIN-3 EIN-2 EIN-1 EIN-4 EIN-3 EIN-2 EIN-1
User Monitoring, Management, & Analysis Applications
Systems Under Test
Hardware
Probes
Software / OS
Probes
TENA Cloud
Private Object Model
Public Object Models
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Flexible & Hierarchical Data Extraction
• The TENA Bridge is responsible for combining real-time OASIS OM data from EINode with static sensor meta-information and republish as SpecificProbe OMs (Thermostat, Accel, Strain, etc.)
• The flexibility and configurability of OASIS’s data collection architecture helps to promote domain-specific hierarchical instrumentation needs of testers.
Objective: Develop a TENA-enable HW/SW sensor data collection and management capability that is flexible, scalable, and non-intrusive
DAC
Commodity HW Resource Constrained HW
TENA
Bridge
Sensor
Config. &
Extracted
Data
Data Extraction
TENA <Private>
OASIS OM
TENA Sensor
OMs
Commands
Commands
Embedded Instrumentation
Network Boundary
External Connections
EINode HW
Probe
SW
Probe SW
Sensor
HW
Sensor
24 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
TRL-6 Technology Tests
Aberdeen Test Center
October 20-21, 2011
TRL-6 Test at Aberdeen Test Center
• Test Conditions – Performed at Aberdeen Test Center in actual test range
environment
– Instrument the Test Track Survey Vehicle that is used to measure variation in track conditions
• Goal – Collect relevant TSPI metrics using TRCE technology
in actual range environment • GPS: time, position
– Compare current ADMAS data (real-time via WiFi and locally recorded) with TRCE technology collected data
– Prove TRCE data collection through 3G links with RelayNode and direct TENA VPN connections
– Verify improved data collection with mobile device and RelayNode enhancements
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. 26
TRL-6 Test: Aberdeen Test Center Current Aberdeen Test Configuration
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. 27
TRL-6 Test: Aberdeen Test Center TRCE Test Configuration
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. 28
Aberdeen Test Center Map Churchville
(Track 2)
Perryman
(Track 1_
~11.6
mile
s
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. 29
Test Track 1 Comparison
Green - iPhone
Red - Overo
Yellow - ADMAS (TENA)
Cyan - ADMAS (Recorded) DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. 30
Test Track 1 Accuracy
Green - iPhone
Red - Overo
Yellow - ADMAS (TENA)
Cyan - ADMAS (Recorded) DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. 31
Test Track 1 Accuracy
Green - iPhone
Red - Overo
Yellow - ADMAS (TENA)
Cyan - ADMAS (Recorded) DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. 32
Travelling to the Churchville Track
Overo 3G data collection
during entire transit.
Green - iPhone
Red - Overo
Yellow - ADMAS (TENA)
Cyan - ADMAS (Recorded) DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. 33
Tracking Accuracy
Green - iPhone
Red - Overo
Yellow - ADMAS (TENA)
Cyan - ADMAS (Recorded) DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. 35
Results • Data publication established and maintained from iPhone
and Overo using Verizon and AT&T 3G
• iPads used to monitor real-time test progress and vehicle
positions
– Capability worked better than anticipated with improved AT&T
coverage since previous test in May
– Provided great insight into the conduct of the test
• Collected native TENA data from iPhone over VPN
connection with AES 128bit encryption
– Automatic recovery from link outages proved difficult with current
iPhone & iPad VPN capability
– Direct RelayNode connections with modifications better able to
handle link dropouts but provided no encryption capability
• 3G Link encountered less drop-outs than WiFi connections
• Overo data collection via RelayNode was very robust
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. 36
Lessons Learned
• 3G links may be a viable option for sending T&E data sets
– External antennas essential for good GPS and 3G connectivity
– 3G bandwidth sufficient for publication of significant vehicle data
– Occasional data gaps using WiFi connectivity compared to 3G links
– New VPN and encryption capabilities are emerging CERDEC that
would protect data to SECRET over 3G/4G links
• Mobile device and RelayNode automatic re-connect
modifications worked very well
– Devices able to quickly re-establish links when coverage lost
(apparent more in AT&T link vs. Verizon links)
• iPads proved to be a great real-time to monitoring capability
to track test progress and verify collected data
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. 37
HW/SW
Probes TENA
Wireless
LAN
TENA /JMETC Booth Demonstrations
3 Axis IMU
GPS
Accelerometer
Thermometer
GPS
Gumstix
Overo
38
6DOF IMU
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
• All devices battery powered with exception of WiFi access point
• “Instrumented” small R/C quad-copter vehicle
39
EINode
(Thermostat)
EINode
(CPU,Memory,
and Network
Probes)
DAC and
Webserver
Personnel Tracker
(GPS, IMU, Thermostat)
OASIS
Metrics
Display
TENA Console
& TVDS
FlightGear
(Live
GPS/IMU
Platform
Subscription)
TENA /JMETC Booth Demonstrations
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Continuing Efforts
• Complete development of the TENA RelayNode 2.0 wireless
system and transition to the TENA Program and user
community
• Complete the migration of all Smartphone and embedded
computation devices to the TENA program including Apple
iOS, Overo Open Embedded Linux, and Android
• Complete the migration of the Embedded Instrumentation
Systems Architecture (EISA) and Open-Source Architecture
for Software Instrumentation of Systems (OASIS) capabilities
to the TENA Middleware
• Perform a multi-range demonstration of all the TRCE
technologies as part of a JMETC or similar test event
40 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Wrap Up
• TRCE will provide technologies for use in the TENA Middleware to support low data rate, variable quality, and wireless networks – The TENA RelayNode 1.0 application provides an interim solution that
can be leveraged by the T&E community.
– Fully integrated TENA RelayNode 2.0 as a core component of the Middleware will provide a long term capability with integrated link management functionality
• TRCE is developing a TENA Middleware ports that will operate natively on small form factor devices like smartphones and embedded instrumentation computers – Apple iPhone OS, Android, and OpenEmbedded Linux support
– The goal is to support all required platforms on single TENA code base
• TRCE Smartphone support fills T&E need for event observation range situational awareness use case (personnel tracking and safety fan violation) at ranges
• TRCE technologies will enable full TENA interoperability throughout the entire mission based testing domain
41 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.