© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448© 2014 The MITRE Corporation. All Rights Reserved.
Applications of Spring Integration to
Near-Real Time Radar Data ProcessingG. Wampole – MITRE Corporation
Approved for Public Release; Distribution Unlimited. Case Number 14-2448
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
NOTICE
This technical data was produced for the U.S. Government
under Contract No. W15P7T-13-C-A802, and is subject to
the Rights in Technical Data-Noncommercial Items clause
at DFARS 252.227-7013 (FEB 2012)
© 2014 The MITRE Corporation. All Rights Reserved.
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Your Presenter
• Garrett Wampole– Hometown: Allentown, PA
– Alum: Rochester Inst. Tech.
– Been Doing This: ~15 years
• Currently– Prin. Engineer @ MITRE
– Radars, Battle Management, Remote Sensing, Imagery Exploitation, U-2, JSTARS
• Spare Time– iPhone & Windows Phone
(VoiceNotes anyone?)
– Root for New York Mets
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
What is MITRE?
• A not-for-profit company that operates FFRDCs(Federally Funded Research and Development Center)
• Presently administers six centers
– National Security Engineering Center (DoD)
– Center for Advanced Aviation System Development (FAA)
– Center for Enterprise Modernization (IRS)
– Homeland Security Systems Eng. and Development Institute (DHS)
– Judiciary Engineering and Modernization Center (Federal Judiciary)
– CMS Alliance to Modernize Healthcare (DHHS)
• Approximately 7000 employees (80.4% engineers)
• Two main headquarters: Bedford, MA and McLean, VA
– ~50 domestic sites, 10 international
• http://www.mitre.org
– Fun Fact: First name registered in the .org TLD (July 1985)
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
What is Radar?
Image source: NOAA
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Simplest Case – Monostatic Radar
Mover
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Fusion of Multiple Monostatic Radars
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Trade-Offs
• Multistatic
+ Less complicated hardware, smaller sensor footprint
+ Cheaper to Build
– Need multiple sensor locations
– Need coordinated central processing
• Phased Array
+ Everything in-situ, one (bigger) box
+ Potential for greater accuracy
– More complex sensor setup
– Development costs have three commas
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
It’s a Data Reduction Problem
Mover Reports
Secondary
Processing
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Pipeline Processing
> wget -qO - www.cnn.com | grep -i spring | wc -l
1. HTTP GET
2. Grep
3. Count
1. Coordinate Timing
2. Find Movers
3. Track Movers over Time
4. Geolocate and Report
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Technical Challenges
• Matching up data from multiple sensors that may arrive at different times
• Coordinating stateful pipeline steps that depend on prior data from each sensor
• Keeping up with near-real-time processing when unforeseen processing hiccups occur
• Supporting multiple types of deployment
– One machine, multiple laptops, servers on different networks
• Support different types of sensor input and product output
– Sensors hardwired or wireless, provide reports as KML or STANAG 4607 to file or network or …
• Provision for a not overly burdensome development environment
• Save intermediate processing products for later analysis
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Pipeline Processing Steps
• Service Activator which manages a set of MATLAB objects
maintained in a Matlab Compiler Runtime instance
MCR
service()
onNewData()
blockonProccessingComplete()
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Multiple Input/Output Types
• Channel Adapter abstraction is key
– Doesn’t change the rest of the processing pipeline
PbufFileIn/OutboundChannelAdapter
CgmtiFile/UDPOutboundChannelAdapter
PbufTCPIn/OutboundChannelAdapter
PbufUDPIn/OutboundChannelAdapter
PbufUDTIn/OutboundChannelAdapter
ReversePbufFileInboundChannelAdapter
Processing
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Development Environment
• Accurately recreating what happened during near-real-
time processing is difficult
– But key for debugging problems and for testing new
algorithms against a representative data set
setRealTimePlaybackEnabled( true )
Recorded
Data
Poller blocked
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Flexible Deployments
• One Spring Context / Matlab Runtime combo per JVM
• Use network-based Channel Adapters as IPC
Generate Products Process
Generate Products
Both deployments supported, no recompile
– Pick an XML context file to start
Process
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Deployments
Receive Data From Sensors
Coordinate Pre-Processing
Find and Track Movers
Geolocate and Create Products
Do It All At Once
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Coordinated Processing
• How to coordinate processing steps that have to occur in proximity, but might be on different processors according to one of many deployments?
– Decorate the service activator with a inter-process cyclic barrier, or “Network Phaser”
Time
arriveAtPhase( 0 )
arriveAtPhase( 0 )
waitForPhase( 1 )
arriveAtPhase( 1 )
blocked
JVM1
JVM2
Phase
Server
Phase
Announcements
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Keeping Up
• Fixed-size queue of data inside a message channel
– How to handle out-of-order data over unreliable networks?
Sort according to when sensor produced data
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
This Stuff Works
http://www.mitre.org/publications/project-stories/bringing-covert-aircraft-out-of-the-shadows
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448
Thoughts
• Enterprise Integration is not the same as B2BIt can work within near-real-time constraints
Spring Integration == ESB == Slow
• SI’s lightweight implementation enables flexibility by not forcing you into too many paradigms
– It also enables performance
• The Spring IoC container eases management and deployment
– Easy deployment is a driver for quick-response ISR systems
• You guys made this possible, thanks!
Not True
© 2014 The MITRE Corporation. All rights reserved. Approved for Public Release; Distribution Unlimited. Case Number 14-2448