Post on 25-Feb-2016
description
transcript
Intelligence in National Security
Association of Former Intelligence OfficersBanquet 2 May 2014
Dr. John M. Poindexterjohn@jmpconsultant.com
2
Agenda
• My goal -- Improving product to help decision makers
• National Security Equation
• Big Data
• Cognitive Computing
• Privacy Problems and Potential Solution
3
National Security Equation
Where operators (functions) are: Collection = human and sensor (e.g. Internet) collection of data-> Analysis = selects data-in-context to produce information=> SenseMaking= understanding what the information meansp> PathFinding = deciding what to do about it in policy contexte> Execution = “operational forces” carry out decision Iteration = many steps are often repeated (e.g. Action
changes the world and thus new collection is required.)• Simplified but basic non-linear process that is essential to understand.• Analysis is an over-used term.• This provides a working definition of Sensemaking and Pathfinding.• Process carried out in a collaborative environment with relevant agencies.
• Collaboration is essential to bring diversity to problem of uncertain data.• Need competitive SenseMaking to give decision makers range of understanding.
• Great deal of confusion amongst the terms data, information and knowledge.• “Operational Forces” – military, diplomatic, economic, public diplomacy, law enforcement, covert.
Involves All of National Security Community Not Just Intelligence…
Data -> Information => Knowledge p> Options e> Action
Extensive Automation More Cognitive
c>
c>
Goal: Develop information technology components to aid process.
4
Improvement Requires Co-Evolution
One• Technology• Largely exists but needs system integration
Two• Process• Needs more automated systematic approach
Three• Policy• Needs to be clearly defined and supported
Four• Culture• Understanding of technology by all concerned
Incr
easi
ng D
iffic
ulty
Of:
5
Big Data – Major Problem but Opportunity
Characterized by: • Volume• Velocity• Variety• Veracity
----------------data
6
DataBases
• DataBases are designed for storage – not analysis– Great for storage of collection– Originally designed for back office operations
• Personnel, inventory and accounting– Ok if queries are of static form– Tables are designed to answer these queries promptly
• With intelligence, complex query forms are dynamic– Can’t predict a priori what needs to be asked– In this case table joins are usually required – With Big Data these joins are very time consuming
• Typically Hours to Days
• Often said about DataBases – “Write Once Read Never”
For analysis there are problems…
7
Person:
Poindexter
Matrix
Organization:
White House
Matrix
MemoryBase – A New Technology
Who/What is similar?How similar/different?
Who/What is related? How? Where? When?
What could happen? Where?
When?
What has been done before? Did it
work?
Sense-M
aking D
ecision Support
A matrix for every person, place, and thingA matrix for every situation, action, and outcome
Design influenced by analogy to human memory…
Multiple Contexts
8
MemoryBase Characteristics
• Does not replace databases, but is an adjunct• Ingests distributed data in heterogeneous formats
– Static and streaming – structured and unstructured text• Incoming schemas are translated to generic schema• Scales to Big Data• Standard off-the-shelf servers• Dynamic query response time in sub-second to seconds
independent of MemoryBase size• Now moving to more cognitive functions• Produced by Saffron Technology, Inc.• Intel has made a multi-million dollar investment recently
Works like the human brain, but never forgets…
9
Privacy Appliance Concept
• Recent revelations about access to Big Data by the USG have raised concerns again about privacy – Section 215.
• Government agencies in the national security domain work diligently IAW the law to protect the privacy of innocent individuals while protecting the US from various threats.– The people want this protection, but are concerned about privacy.
• The problem is the people don’t trust the government.• Maybe technology can help with this.
– Complicated, but possible.• When I was at DARPA after 911, we came up with a concept for a
Privacy Appliance and began research.– It was the only part of the TIA program that was not transferred to the IC and
work on it stopped.
Access to Big Data has privacy implications…
10
Concept for Controlled Data Access
Transactions
Collaborative, Multi-Agency
AnalyticalEnvironment
AutomatedData
RepositoriesWorld WideDistributedData Bases
PrivacyAppliance
Leave data distributed, identify critical data bases…
Red teams simulating threat organizations plan attacks and developpatterns of transactions that are indicative of attack planning.
Pattern-basedQuery
FilteredResults
Patterns are important to search for data-in-context to avoid 6-degrees of separation problem.
11
Finding Relevant Information -- AnalysisWhile protecting the privacy of innocents, sources & methods…
data source
data source
data source
privacy appliance
user query cross-
source privacy
appliance
privacy appliance
privacy appliance
Government owned
Commercial or Government owned
Independently operated
response
• Authentication • Authorization• Anonymization• Immutable audit trail• Inference checking• Selective revelation• Data transformation• Policy is embeded• Create MB Index
• Contains MemoryBase (MB) Index• Updated in real time
12
• Search Patterns are authorized by a judicial authority (e.g. FISA court).• Selective Revelation to limit response details depending on level of authorization.• Inference Control to identify queries that would allow defeat of anonymization.• Access Control to return identifying data only to appropriately authorized,
authenticated users.• Immutable Audit Trail for accountability – must have way of analyzing routinely.• Masking to hide analyst intent – especially for non-government data bases.• MemoryBase index created to home in on relevant data bases.
Authorization tables
Inference control knowledge base
Immutable audit trail
User query
Query blocked
or allowed
Masking
Selective Revelation&
Anonymization
Policy & BusinessRules Embedded (machine readable)
The Privacy Appliance ConceptAll functions highly automated to reduce time late…
Transparent, cryptographicprotected shell
(much like network guards)
MemoryBaseProcessing
Publish source code for appliance. Need to avoid Clipper Chip problem.