UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

Advantage Through TechnologyAdvantage Through TechnologyAdvantage Through TechnologyAdvantage Through Technology

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

Information Fusion Functional Model(Jt. Directors of Laboratories (JDL), 1993)

Level 0 — Sub-Object Data Association & Estimation: pixel/signal level data association and characterization

Level 1 — Object Refinement: observation-to-track association, continuous state estimation (e.g. kinematics) and discrete state estimation (e.g. target type and ID) and prediction

Level 2 — Situation Refinement: object clustering and relational analysis, to include force structure and cross force relations, communications, physical context, etc.

Level 3 — Impact Assessment: [Threat Refinement]: threat intent estimation, [event prediction], consequence prediction, susceptibility and vulnerability assessment

Level 4: Process Refinement: adaptive search and processing (an element of resource management)

Level 0Processing

Sub-object DataAssociation &

Estimation

Level 1Processing

Single-ObjectEstimation

Level 2Processing

SituationAssessment

Level 3Processing

Mission ImpactAssessment

Level 4Processing

ProcessAssessment

Data BaseManagement System

SupportDatabase

FusionDatabase

INFORMATION FUSION PROTOTYPEJEM

JWARN3GCCS• Point and

Standoff Sensors• Data Sources• Intel Sources• Air Surveillance• Surface Sensors• Standoff Sensors• Space

Surveillance

Methods:--Combinatorial Optimization

--Linear/NL Estimation--Statistical

--Knowledge-based--Control Theoretic

TrackingAttributesID/Events

RelationshipsAggregation

Intent

LethalityCOA

Opportunity

PerformanceContext

Consistency

DetectionReports

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

State University of New York

UniversityCenterBuffalo

Engrg & Applied Sciences

Industrial & Systems Engrg CMIF

……….

CUBRC

Intl Security

Hyper-sonics

ChemBioDefense

MedicalBiotech

PublicSafety

InfoFusion

Major Research University Multidisciplinary Not-for-Profit

Shared Technical and Administrative StaffsJointly Managed

Jim Llinas, Executive DirectorMoises Sudit, Managing DirectorRakesh Nagi, ISE ChairJohn Crassidis, Associate Director

Mike Moskal, Vice President of CUBRC

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

• Applied RDT&E

Focus

• Cleared Personnel

and Facilities

• Systems

Engineering

• SW/HW

Development

• World Class Research

Personnel and

Facilities

• Focus on Basic

Research

Application Driven Research and Development for Defense, Intelligence Application Driven Research and Development for Defense, Intelligence

and Homeland Securityand Homeland SecurityGovernment Agencies (25+) Government Agencies (25+) and Industrial Partners (50+)and Industrial Partners (50+)

Development & Transition Engineering

Applications Engineering,

Fielding & Support

6.1 6.2 6.3 6.4 6.5

TRL1 TRL4 TRL5 TRL6 TRL7 TRL8 TRL9TRL2 TRL3

Universities (~30+)Universities (~30+)

• Specific Technologies

Under Development

Small Businesses (10+)Small Businesses (10+)

USAMRIID

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

• Capability to represent interactions of players at a high level of fidelity including

• Real world capability to model tracker and correlation capabilities of current and future systems

• Demonstrated interfaces to many other high fidelity models for aircraft platforms under test

• Interfaces to many other specific threat system models

• Description/Customer Needs: DIADS is an emulation fidelity model of the integrated threat air defense system. Important component in many test and training activities

• Customer: Air Force• Importance: Key simulation in the evaluation

of aircraft effectiveness• Comment: Used at training ranges and key

airborne weapons platforms evaluations

• Continuous development since 1996• Derived from a Hardware-in-the-loop simulation• Demonstrated performance in Live, Virtual and

Constructive Applications (LVC)• Used by both the test and training community • Capability to run as a mission level model but

used more frequently in a multi-model environment

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

• Mission: Information Fusion and related areas primarily but not exclusively for defense and homeland security applications

• Basic and Applied Research in:– Multiple-sensor and instrumented systems– Synergistic Human-Multisensor systems – Real-time Decision-making using Hierarchical Fusion– Graph Theory and Optimization for Level 2/3 Fusion– Multi-modal information environments (speech+text+imagery+RF sensor+human input)

• Applications:– Defense: Intelligence/Surveillance/Reconnaissance; Tactical Applications; Homeland Security– Non-Defense: Robotics; Conditioned-Based Maintenance; Medical; Transportation; Geology; Natural

Disasters/Crisis Mgmt• History and Funding:

– Started in 1996 with Air Force Research Lab Contract– Funding activity evolving; currently ~$10M/year

• Scholarly: – Long-standing member of “JDL” fusion group and First President of Intl Society for Info Fusion– Extensive publishing by CMIF PI Team including books, Jl papers, conference papers and review boards– “Critical Issues” Workshops—5 years– CMIF is unique in American Universities as a research activity focused on IF technology for DHS/DoD– Consortium development to include other universities (SU, RIT and PSU) and industrial partners and

development of a Graduate-level program in Data Fusion– Currently working on developing a consortium with TAMU and VPI as well

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

• Only Integrated Information Fusion (IF) Research Center in US Academia

• In existence 15 years• Broad range of DoD/Agency

sponsored research programs• ~15 Professor/Stakeholders• Systemic approach to IF capability

development• Unclassified to Classified, 6.1 to

Transition• Collaborations with

– PSU– RIT, Syracuse– TAMU, VPI– Buffalo:

• Ctr for Unified Biometrics• National Center for Geographic

Information and Analysis• Center for Information Systems

Assurance • Center for Document Analysis and

Recognition• Wireless and Networking Systems Lab• Semantic Network Processing Systems

Research Group

Scientific Foundations of the Data Fusion Process

RealStates in the

World

ObservationalMeans

Info. FusionProcesses

EstimatesOf World States

Dec-MkgAnalysis

etc

Data Association

EvaluationActions

Process Refinement

A Process to ESTIMATE conditions in the Real World from Observational Data

Modeling Tactical

Phenomena

SensingTechnologies

SignalPropagation

MathematicalAnd Symbolic

EstimationTechniques

SignalProcessing Human

ComputerInterfacing

HumanFactors and

HumanEngineering

DecisionScience

VisualizationVirtualReality

SensorNetworks

ControlTheory

CombinatoricOptimization

Broadly Multidisciplinary

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

• Capability to accept data from different domains

• Capability to accept potentially double counted data

• Cross-correlate different domains to objects under surveillance

• Build “Big Picture” with reduced workload for operators

• Description/Customer Needs: Evolving set of algorithms to process data of unknown provenance in a Net-Centric SIGINT Focused Information System (NCSFIS)

• Customer: Sierra Nevada Corporation• Importance: Process large volumes of data to

assist operators in developing situation awareness

• Comment: First integration performed. Additional capabilities in the “pipeline”

• Recent effort for CUBRC – Sept 2007• First integration involves ELINT and COMINT

pre-processed reports on static objects• Pre-integration demonstration of

measurement-level COMINT, MTI and IMINT• Interplay with Coarse-of-Action Mission-

Planning to provide real-time optimization of surveillance asset usage

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

• Problem to be solved– Fully derive relative air vehicle navigation equations

• Includes both relative position and attitude• Used quaternion for relative attitude

– Used relative LOS observations only• Texas A&M VISNAV sensor

– Applications include UAV relative navigation without GPS and fully autonomous refueling

tan tan

tan ( ),

FHG

IKJ

ffct v

F v

fct v

v ki i

i i

linear ( )

nonlin. calibration ( ) where

scaled current imbalance

1 2

1 2

• Energy centroid located more accurately than 1 part in 5,000 with proper design, calibration & signal processing, 1 part in 2,000 routinely achieved

• 1 to 5 μs rise time Can be sampled at very high frequency• With proper choice of optics, accuracy of energy centroid is a

weak function of the depth of field

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

• Problem to be solved– Consider only one set of LOS vectors between

spacecraft– Rotation around LOS vector is not known– Under-deterministic case (standard attitude

determination issue)– Can overcome problem by running filter with

motion• Convergence and observability problems need to

be overcome

– Our goal is to exploit formation information to find a deterministic solution Derived a method that handles

nonparallel beams, but also includes range errors. Less of a problem as distance increases

Deputy 2Aircraft

Deputy 1Aircraft

ChiefAircraft

Total of 2 “eqns” and3 unknowns

Total of 4 “eqns” and6 unknowns

Total of 6 “eqns” and6 unknowns

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

Posterior PDF

KF 1

KF 2

KF M

Unknown SystemReal System

MMAE Filter

• Bank of parallel filters, multiple estimates• State estimate is a weighted sum of each

filter’s estimate– Measurement residuals are “drivers” of

adaptive process– Weights derived from Bayes rule– Weights are probabilities

• Can work with nonlinear systems– EKF assumptions must be valid– Measurement residual must be Gaussian

• CMIF Extension– Generalized MMAE (GMMAE) uses a window

of residuals– Combines autocorrelation for i steps back

with MMAE– When i = 0 standard MMAE is achieved – Goal is to improve convergence– Many applications

• Current research involves L1/L2 integration

– Currently extending to UKF GMMAE• Collaborations with Simon Julier

Tracking Example with i = 4

Proc

ess

Noi

se V

aria

nce

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

MMAE

Health Monitoring and Fault Detection

Parameter

Identification

Filter Tuning

• Robust Target Tracking• Adaptive Signal Processing• Navigation applications

• Adaptive Control• Communication Systems• Tactical Assessment

• Sensor and actuator degradation faults• Structural and mechanical health monitoring• Reconfigurable (intelligent) systems• Higher level fusion applications

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

15 20 25 30 35 40 45 50 55 600

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

Time (mins)

X-P

ositi

on (

3

Bou

nds)

(m

)

Fusion Node vs. Local Nodes

• Problem to be solved– Reduce single point failures in estimation system

architectures– Fuse multiple filters (nodes) in optimal manner– Use optimal platform (sensor) motion to lower

estimation errors– Fusion node (FN) uses only an estimate and

covariance from each Local node (LN)– Our objective is to provide a robust estimation

system architecture for dynamic problems• e.g. Target Tracking

FN – red (solid)LNN – all others (dashed)

15 20 25 30 35 40 45 50 55 60-2000

-1000

0

1000

2000

X -

Err

ors

(m)

Fusion Node 3 Bounds

15 20 25 30 35 40 45 50 55 60-2000

-1000

0

1000

2000

Time (mins)

Y -

Err

ors

(m)

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

Project Description Discrete Optimization Models are designed to solve the “right”

problems considering spatial as well as temporal constraints with the objective of maximizing the delivery of services by the entire fleet of UAVs:

● Maximize the servicing of certain predefined “targets”● UAVs are constrained by the amount of time they can fly before

refueling, the amount of service they can provide and a lower/upper amount of service the entire fleet should provide to a specific “target”.

● Consideration of threat, enemy attack, or natural hazards ● Novel Discrete Optimization techniques• Management of Emerging Targets:● Targets need to be classified into an appropriate priority● UAVs scheduling and Sensor-Platform Assignment

Significance

Implementation of the OPTIMAS architecture into the overall Command and Control and Combat Systems (C2 and CS) Program will greatly enhance the decision-making effectiveness of maritime operators by increasing overall situation awareness and presenting optimal or high quality solutions to a suite of strategic and tactical decision making problems. The OPTIMAS suite of tools provides feedback to the operators and to the tools themselves both within and between the strategic and tactical level issues/problems. OPTIMAS facilitates the complex decisions to be made in an atmosphere of tight resources and dynamically changing environments taking advantage of the computational strength of computers in the human-computer interaction paradigm.

Flight Dynamics – ‘Real’ and Approximated by Dubins Vehicles

Dubins Vehicles: route tracking by a kinematic vehicle moving forward only with a lower bounded turning radius

Dubins Vehicles: route tracking by a kinematic vehicle moving forward only with a lower bounded turning radius

A

B

C C

A

BBetter

Tracking

UNCLASSIFIED//CUBRC PROPRIETARY

UNCLASSIFIED//CUBRC PROPRIETARY

• Approximate the conditional pdf as a mixture of Gaussian components

– The mean and covariance of each of the Gaussian component is propagated by using the extended Kalman filter or unscented Kalman filter

• Two update schemes for the forecast weights– Continuous Dynamical Systems: minimize the

Fokker-Planck-Kolmogorov Equation (FPKE) error– Discrete Dynamical Systems: minimize the integral

square difference between the true pdf and its approximation

– During the measurement update, Bayes rule is used to update the weights

– Unique solution for weights is guaranteed

• Future Work– Automatic selection of number of minimum

number of Gaussian components required– Improve computational cost through

parallelization– Many applications

• Orbit determination, Attitude Estimation• Asteroid Collision Probability• GPS-less Localization• Plum Tracking

– Currently extending to UKF Gaussian Sum Filter

• Collaborations with Simon Julier