Basic/Futures ResearchFODAVA and Homeland Security Joseph KielmanDirector, Basic and Futures ResearchCommand, Control and Interoperability Divisionand Manager, Center of Excellence for Command, Control, and Interoperability Office of University ProgramsScience and Technology [email protected] December 3, 2009
CNR Annual Program Status Review 8 February 2005
Command, Control and Interoperability
Vision Stakeholders have comprehensive, real-time, and relevant information to create and maintain a secure and safe Nation
MissionThrough a practitioner-driven approach, the Command, Control and Interoperability Division (CCI) creates and deploys information resources to enable seamless and secure interactions among homeland security stakeholders
CCI Mission Areas and Challenges
• Information analysis• Knowledge management• Threat assessment• Situational awareness• Decision support• Information sharing• Interoperable communications• Surveillance and investigative operations• Cyberinfrastructure protection
Digital Data – the WHY
• In 2002, recorded media and electronic information flows generated about 22 exabytes (1018) of information
• In 2006, the amount of digital information created, captured, and replicated was 161 EB
• In 2010, the amount of information added annually to the digital universe will be about 988 EB (almost 1 ZB)
• About 2 GB of digital information is being produced per person per year
• 95% of the Digital Universe’s information is unstructured
– 25% of the digital information produced by 2010 will be images
– 30% will be produced by organizations
The Homeland Security Environment – the WHO• Department of Homeland Security
– 22 components, 7 operational agencies– 215,000 personnel
• Federal Government Partners– 15 Intelligence Community agencies– 11 Law Enforcement Community agencies– 350,000 personnel
• Tribal, Local, and State Partners– 80,000 public safety, public health, emergency response,
law enforcement agencies– 750,000 homeland security practitioners
New Dimensions of Homeland Security – the WHATTime Scale – short to long-lived
incident to development to crisisattack to epidemic to global change
Consequence - limited to extensivelocal to regional to national to globalhuman to group to society
Action – insight to decisionpreparation to prevention to response
to remediationprediction to anticipation to warning to detection
The Information Challenge – the HOW
• CCI is developing a toolkit - the tools, technologies, knowledge, and resources to enable Homeland Security practitioners to acquire, manage, analyze, share, and secure information
• Each of CCI’s thrust areas plays an important role in creating this toolkit
Basic/Futures Research
• Conducts research on information and intelligent systems to address problems associated with synthesizing information and deriving insight from massive, dynamic, ambiguous, and diffuse data sets
• The objective is support for an effective risk management approach to homeland security based on comprehensive, timely threat awareness and decision making informed by accurate consequence analyses
•Cluster and Thematic views• Evidence Evaluation• Triage Networks• Visual analysis for multiple languages• Multi-Viewpoint Support• Affect and Emotion Measures• Correlation Analysis• Streaming Data•Collaborative Team Analytics
Research Interests (1 of 2)• Dynamic, on-Demand Data Processing and Visualization: Capability
for real-time management, analysis, and visualization of selected data in multiple forms and from multiple, diverse sources. These techniques would automatically select, rank, and correlate only those data relevant for purpose-driven decision-making.
• Hypothesis-driven Analysis: This capability would include three elements: automated retrospective analysis of collected or extant data using pre-selected hypotheses; automated generation of alternative hypotheses by constant updating of data; and prospective analysis of potential risks and threats using data-derived hypotheses.
• Visualization of Structured, Unstructured, and Streaming Data: Capability for integrated visual analysis of free text, database records, audio, video, imagery, transactional data, geographical data, and sensor information. The focus on this effort is twofold: development of a single, scalable framework for visual analytics and establishment and validation of reliable performance metrics for visual processing of data.
• Mathematics of Discrete and Visual Analytics: Development of the mathematical foundations for discrete processing and simulation and for visual analytics. This will provide a rigorous scientific basis for future algorithm development.
Research Interests (2 of 2)
• Scalable Filtering and Dissemination: Techniques for secure, privacy-aware identification and dissemination of information among international, federal, state, tribal, and local agencies. This includes advanced methods, processes, and procedures that ensure sharing of information for immediate decision-making by multiple partners under a range of technical, political, and organizational parameters
• Visualization and Simulation of Data: Application of visualization techniques, discrete mathematics methods, and game theory to diverse information, including development of new approaches to simulating multiple threats or disasters.
• Mobile and Light-Weight Information Analytics and Sharing: Information discovery, dissemination, and decision-making tools capable of being tailored for diverse homeland security applications and software architectures. These techniques need to focus on a range law enforcement, public safety, public health, and emergency response applications.
Basic/Futures Research Program Areas• Visual Analytics, Precision Information Environments
Visually based mathematical methods and computational algorithms for discovering, comprehending, and manipulating diverse data and applying the resulting knowledge to anticipate terrorist incidents or catastrophic events and guide response and recovery activities
• Data-intensive Computing, Privacy and ForensicsSimpler, more efficient software algorithms and hardware architectures for extracting
and managing data, assessing threats and consequences, ensuring information privacy, securing the cyber infrastructure, and ensuring telecommunications interoperability
HIGHLIGHTS • Canada-USA, UK-USA, and German-USA Collaborations - Collaborative Activity Agreements (CAA) under existing treaties between DHS S&T and Defense Research and Development Canada, UK Home Office, and German BMBF • Visualization and Analytics Complex – The National Visualization and Analytics Center (NVAC), the new CCI Center of Excellence, Joint NSF-DHS Foundations of Data and Visual Analytics (FODAVA) Program, and 10 industry partners• National Research and Development Agenda
Technologies and Tools
Country AFirm 1
Firm 2Firm 3
Firm 4Firm 5Firm 6
Firm 7Firm 8
Firm 9Firm 10
A Bank
Organization
Cognition
Prediction
Connect the Dots
Content Management
Data Information Knowledge
Aggregation
Integration
Extraction
Link Discovery
Pattern Analysis
Graph Matching
Evidence Extraction
Visual AnalyticsSynthesis
Analysis
Analysis - Retrospective
Organization
Cognition
Prediction
Connect the Dots
Content Management
Data Information Knowledge Wisdom
Aggregation
IntegrationExtraction
Link Discovery
Pattern Analysis
Graph Matching
Evidence Extraction
Visual AnalyticsSynthesis
Analysis
Predictive, Prospective Analytics
Discovery
Policy Context, Culture,GenesCognitive/Behavioral Analytics
The Top Ten – Framing the Discussion
• Conditions
– Untethered to Device/Network/Interaction
– Tethered to Data/Information
– Indefinite or Indeterminate Data
– Minimized Transaction Costs
– Trust
• Components
– Challenges
– Research Foundations
– Applications
The Top Ten Applications
• Synthetic Materials
• Disease Development
• Economic Stability
• Climate Change
• Environmental Stability
• “Slow” Catastrophes
• Genetics and Epigenetics
• Threat (All-Hazards) Awareness
• Human Learning
• Financial Policy
The Top Ten Challenges• Dynamic, on-Demand Data Processing and Visualization
• Hypothesis-driven Analysis
• Structured, Unstructured, and Streaming Data
• Mathematics of Discrete and Visual Analysis
• Scalable Filtering and Dissemination
• Common Modes for Models, Simulations, and Real-World Data
• Mobile and Light-Weight Information Analytics and Sharing
• Provenance, Privacy, and Security
• Synthetic Environments and Data
• Purpose-driven Interaction
The Top Ten Research Foundations
• Cognitive and Perceptual Science
• Data Transformation and Representation
• Dimensionality Reduction
• Smart Query
• Machine Learning
• Computing and Digital Architectures
• Games and Synthetic Environments
• Risk
• Evolutionary Biology
• Cultural Anthropology
Principles
1. Information -There are real-world problems for which the availability of information is problematic
2. Utility -The capabilities being developed must address at least four information-related issues
3. Outcome -The improvements being provided must have real-world outcomes or implications
The Principle of Information• The problem (or need) is actually two-fold: increasing amounts
of data along with increasing diversity (types, sources, modes) of data
• This is compounded by two additional factors: multiple, diffuse users* and the need for real-time analysis and decision-making
• In the future the indeterminate nature of the data, whose relevance is uncertain, transitory, or yet to be determined, will present more significant challenges
• Homeland security involves a multiplicity of potential threats: for instance, terrorist attacks, natural disasters, pandemics, and border security
*80,000 law enforcement, public safety, public health, or emergency response agencies forming the local, state, tribal, and federal homeland security environment
The Principle of Utility
• Analyze single types of data in a more immediate or real-time fashion – that is, address scalability
• Integrate multiple types of data – that is, address heterogeneity
• Combine streaming data or sensor measurements, information and intelligence, models, and analyses into a comprehensive assessment – that is, deal with complexity
• Create dynamic analyses of situations or incidents, which are simultaneously available in the field or in the laboratory, on an analysts’ desktops, in command centers or situation rooms, and to decision- or policy-makers – that is, enable decision-making on multiple levels
The Principle of Outcome• Understanding threats in sufficient time or to a sufficient
degree to prepare or respond
• Predict the scale or scope of individual homeland security problems, and the associated contributing or cascading secondary effects
• More accurately gauge the immediate as well as lingering, that is, the short-term and longer term, consequences of threat agents and develop effective response and remediation technologies and strategies
• Address complex issues or problems, such as climate change, economic stability, resource constraints, and population dynamics, which are interrelated and will increasingly affect our world in the future
Virtual USA (vUSA) • Support tactical, strategic and planning decisions for
homeland security practitioners by enabling the exchange of information in whatever form is needed, delivered to whatever device is available
• Seek to unify activities related to the gathering, analyzing, managing, sharing, and protecting of information
• Create a way to organize efforts at multiple levels (local, state, tribal, national)
• Advance application- and platform-agnostic technologies, standards, governance models, and other tools
Organizing Efforts at the National Level
Virtual USA Lead Efforts
• Virtual City: A pilot demonstration as part of a national program that integrates multiple data sources and makes them more accessible for responders with and other city officials.
• Virtual State: Builds on existing investments in Virtual Alabama with advanced visualization, video streaming and other technologies.
• Regional Operations Platform Project: Expands the Virtual Alabama concept to a multi-state regional level and integrates existing platforms, enhanced visualization tools, and other data sets such as hurricane data to allow state systems to interoperate and exchange data with each other regardless of the particular platform or application in use.
New VAC
from Retrospective Analysis …
… to Prospective Analytics
Data Deluge
The man who acquires an encyclopedia does not thereby acquire every line, every paragraph, every page, and every Illustration; he acquires the possibility of becoming familiar with one and another of those things. If that is the case with a concrete, and relatively simple, entity, then what must happen with a thing which is abstract and variable – ondoyant et divers?
Jorge Luis BorgesShakespeare’s Memory
The Library of Babel
The Future World – Part 1
Necessary and Sufficient Conditions for Interaction:
1. No personal computers; no WIMPS; that is, single-layer or flat interactions prevail
2. Access to tailored information is complete on public screens or on individual displays
3. Communications, information networks automatically, dynamically establish and maintain communities of interest
4. Interaction with information, data is mediated not by bandwidth but by cognitive properties
5. When I do search for information, that search is enabled by images, graphs, multimodal forms
Knowledge acquired from the changeable and perishable objects of sense-perception is subject to doubt and error
St. Bonaventure
The Future World – Part 2
Necessary and Sufficient Conditions for Discovery:
1. data from all disciplines is available/accessible and indexed
2. Processes, controls are understood at a basic level (mature paradigm)
3. Modeling, simulation, gaming environments are more finely grained
4. Interactions become, replace experiments
5. Tailored collaborations are dynamically established, resourced
Knowledge – the agent intellect provides the light
Thomas Aquinas
The Future World – Part 3
Necessary and Sufficient Conditions for Precision Information Environments:
1. Synthetic world and real world co-exist, are co-equal
2. Physical and information spaces are co-existent
3. Modeling and simulation capabilities apply to both
4. Streaming, dynamic data are available, accessible in real-world tiem
5. Environment is tailored, suited for multiple user groups on a dynamic basis
6. Information spaces are multimodal
7. Multiple users and manifold uses are managed simultaneously
Knowledge involves the presence in the mind of a representation of its objects
Duns Scotus
Te
chn
olo
gy
Research Futures: T-Square Model
Threat
Extant Emerging
Extant
Emerging
Key
Yes
Maybe
Some-how
Who Knows
VAC Map
Detecting the Expected -- Discovering the UnexpectedTM
RVACUniversity of Washington
RVACPurdue UniversityIndiana Univ.Schoolof Medicine
RVACUniv. of North Carolina Charlotte,Georgia TechBank of America
RVACPenn. State
DHSGVACs
Scholars
Consortium
A Partnership with Academia,Industry, Government LaboratoriesAlaska
NewZealand
Australia
Hawaii
Europe
Canada
PacificRim
Drexel UniversityNY/NJ Port AuthorityEmergency Op Center
NSF
IVAC
RVACStanford
University
IDS-UACUniversity of
Southern California
IDS-UACUniv. of Illinois
IDS-UAC, Rutgers Univ.
IDS-UACUniversity of Pittsburgh