8/2/2019 Tristan Sternson & Jeff Jonas
1/67
2012 Infoready Pty Ltd1
Big Data,New Physics, and
Geospatial Super-FoodTristan Sternson, InfoReady Managing Director
Jeff Jonas, IBM Distinguished EngineerChief Scientist, IBM Entity Analytics
8/2/2019 Tristan Sternson & Jeff Jonas
2/67
2012 Infoready Pty Ltd2
Who is InfoReady?
?Pure-Play Information Management and Business IntelligenceConsulting firm
?Team InfoReady career IMand BI Experts
One of the fastest growingconsulting firms in
Australia. IM Focused Tier One
Consulting capability. Focus - people, process and
technology Assisting companies turn
valuable information intoactionable intelligence.
Strategy, Architecture,Solution Design & Delivery
8/2/2019 Tristan Sternson & Jeff Jonas
3/67
2012 Infoready Pty Ltd3
My Background Tristan Sternson
?Past 10 years focussed purely on IM / BI Solutions
?Started InfoReady in 2008
?Prior Roles Accenture Data Management & Architecture / IMLead, PWC Consulting / IBM
?Personally designed and deployed and led many large IM andDW application in Australia and UK
?Thought leader in Information Management in Australia andAPAC
?
Early adopter Data Governance, Big Data, Industry DataModels, Appliance DW solutions
8/2/2019 Tristan Sternson & Jeff Jonas
4/67
2012 Infoready Pty Ltd4
My Background Jeff Jonas
?Early 80s: Founded Systems Research & Development (SRD), acustom software consultancy
?Personally designed and deployed +/- 100 systems, a number ofwhich contained multi-billions of transactions describing 100sof millions of entities
?1989 2003: Built numerous systems for Las Vegas casinosincluding a technology known as Non-Obvious RelationshipAwareness (NORA)
?2001: Funded by In-Q-Tel, the venture capital arm of the CIA
?
2005: IBM acquires SRD?Today: Primarily focused on sensemaking on streams with
special attention towards privacy and civil liberties protections
8/2/2019 Tristan Sternson & Jeff Jonas
5/67
2012 Infoready Pty Ltd5
Big Data Definition
Datasets that grow so large that they become difficult to work with,including; capture, storage, search, sharing, analytics, and visualization.
Benefits of working with larger and larger datasets allowinganalysts to "spot business trends, prevent diseases, combat crime.
We havent seen anything yet, as more devices come online, eg;mobile, airborn, logs, cameras, microphones etc
Wikipedia - 2012
8/2/2019 Tristan Sternson & Jeff Jonas
6/67
2012 Infoready Pty Ltd6
The Big Data Opportunity
V3
8/2/2019 Tristan Sternson & Jeff Jonas
7/67
2012 Infoready Pty Ltd7
Big Data Why the hype?
?By 2015, nearly 3B people will be online, pushing the datacreated and shared to nearly 8 zettabytes.
?30 billion pieces of content were added to Facebook this pastmonth by 600M plus users.
?More than 2B videos were watched on YouTube yesterday.
?In the US mobile phone users between the ages of 18 and 24send an incredible 110 text messages per day.
?32B searches were performed last month on Twitter.
?Worldwide IP traffic will quadruple by 2015.
8/2/2019 Tristan Sternson & Jeff Jonas
8/67
2012 Infoready Pty Ltd8 8
Business leaders frequently makedecisions based on information theydont trust, or dont have
1in3
83%of CIOs cited Business intelligence &Analytics as part of their visionaryplans to enhance competitiveness
Business leaders say they dont haveaccess to the information they need to
do their jobs
1in2
of CEOs recognise they need to betterunderstand information more rapidlyin order to make swift decisions
60%
Business Value
8/2/2019 Tristan Sternson & Jeff Jonas
9/67
2012 Infoready Pty Ltd9
Big Data Trends
80%20%
8/2/2019 Tristan Sternson & Jeff Jonas
10/67
2012 Infoready Pty Ltd10
What the Industry Analysts say
Gartner predicts Big Data to beone of the top-10 strategic initiatives
for 2012
8/2/2019 Tristan Sternson & Jeff Jonas
11/67
2012 Infoready Pty Ltd11
What the Industry Analysts say
Key take-aways from Analyst perspectives Gartner TDWI
Data will grow exponentially
Fusion of structured and unstructured data
The connection between big data and advancedanalytics will get even stronger
Future users will not be able to put all useful
information into a single data warehouse
BigDatausedtob
ea
technicalprob
lem -nowitsa
businessoppo
rtunity.
8/2/2019 Tristan Sternson & Jeff Jonas
12/67
2012 Infoready Pty Ltd12
Enterprise Intelligencevs. Enterprise Amnesia
Jeff Jonas, IBM Distinguished Engineer
Chief Scientist, IBM Entity Analytics
8/2/2019 Tristan Sternson & Jeff Jonas
13/67
2012 Infoready Pty Ltd13
Time
C
omputingPow
erGrowth
Sensemaking
Algorithms
AvailableObservation
Space
Context
Trend: Organizations Are Getting Dumber
EnterpriseAmnesia
8/2/2019 Tristan Sternson & Jeff Jonas
14/67
2012 Infoready Pty Ltd14
Time
Sensemaking
Algorithms
AvailableObservation
Space
ContextWHY?
Trend: Organizations Are Getting Dumber
C
omputingPow
erGrowth
8/2/2019 Tristan Sternson & Jeff Jonas
15/67
2012 Infoready Pty Ltd15
Algorithms at Dead End.
You CantSqueeze KnowledgeOut of a Pixel.
8/2/2019 Tristan Sternson & Jeff Jonas
16/67
2012 Infoready Pty Ltd16
No Context
8/2/2019 Tristan Sternson & Jeff Jonas
17/67
2012 Infoready Pty Ltd17
Context, definition
Better understandingsomething by taking intoaccount the things around it.
8/2/2019 Tristan Sternson & Jeff Jonas
18/67
2012 Infoready Pty Ltd18
Information in Context and Accumulating
Top 200Customer
JobApplicant
IdentityThief
CriminalInvestigation
8/2/2019 Tristan Sternson & Jeff Jonas
19/67
2012 Infoready Pty Ltd19
The Puzzle Metaphor
?Imagine an ever-growing pile of puzzle pieces of varying sizes,shapes and colors
?What it represents is unknown there is no picture on hand
?Is it one puzzle, 15 puzzles, or 1,500 different puzzles?
?Some pieces are duplicates, missing, incomplete, low quality, orhave been misinterpreted
?Some pieces may even be professionally fabricated lies
?Until you take the pieces to the table and attempt assembly,you dont know what you are dealing with
8/2/2019 Tristan Sternson & Jeff Jonas
20/67
2012 Infoready Pty Ltd20
Puzzling
200 pieces
66%
150 pieces
50%
270 pieces90%
6 pieces
2%(pure noise)
30 pieces10%(duplicates)
8/2/2019 Tristan Sternson & Jeff Jonas
21/67
2012 Infoready Pty Ltd21
8/2/2019 Tristan Sternson & Jeff Jonas
22/67
2012 Infoready Pty Ltd22
8/2/2019 Tristan Sternson & Jeff Jonas
23/67
2012 Infoready Pty Ltd23
First Discovery
8/2/2019 Tristan Sternson & Jeff Jonas
24/67
2012 Infoready Pty Ltd24
More Data Finds Data
8/2/2019 Tristan Sternson & Jeff Jonas
25/67
2012 Infoready Pty Ltd25
Duplicates in Front Of Your Eyes
8/2/2019 Tristan Sternson & Jeff Jonas
26/67
2012 Infoready Pty Ltd26
First Duplicate Found Here
8/2/2019 Tristan Sternson & Jeff Jonas
27/67
2012 Infoready Pty Ltd27
8/2/2019 Tristan Sternson & Jeff Jonas
28/67
2012 Infoready Pty Ltd28
8/2/2019 Tristan Sternson & Jeff Jonas
29/67
2012 Infoready Pty Ltd29
Incremental Context Incremental Discovery
6:40pm START
22min Hey, this one is a duplicate!
35min I think some pieces are missing.
37min Looks like a bunch of hillbillies ona porch.
44min Hillbillies, playing guitars, sitting
on a porch, near a barber sign and a banjo!
8/2/2019 Tristan Sternson & Jeff Jonas
30/67
2012 Infoready Pty Ltd30
150 pieces
50%
8/2/2019 Tristan Sternson & Jeff Jonas
31/67
2012 Infoready Pty Ltd31
Incremental Context Incremental Discovery
47min We should take the sky and grassoff the table.
2hr Lets switch sides, and see if wecan make sense of this fromdifferent perspectives.
2hr10m Wait, there are three no, fourpuzzles.
2hr17m We need a bigger table.
2hr18m I think you threw in a few randompieces.
8/2/2019 Tristan Sternson & Jeff Jonas
32/67
2012 Infoready Pty Ltd32
8/2/2019 Tristan Sternson & Jeff Jonas
33/67
2012 Infoready Pty Ltd33
8/2/2019 Tristan Sternson & Jeff Jonas
34/67
2012 Infoready Pty Ltd34
8/2/2019 Tristan Sternson & Jeff Jonas
35/67
2012 Infoready Pty Ltd35
How Context Accumulates
?With each new observation one of three assertions are made:1) Un-associated; 2) placed near like neighbors; or 3) connected
?Must favor the false negative
?New observations sometimes reverse earlier assertions
?Some observations produce novel discovery
?As the working space expands, computational effort increases
?Given sufficient observations, there can come a tipping point
?Thereafter, confidence improves while computational effortdecreases!
8/2/2019 Tristan Sternson & Jeff Jonas
36/67
2012 Infoready Pty Ltd36
Observations
UniqueIde
ntities
True Population
Overstated Population
8/2/2019 Tristan Sternson & Jeff Jonas
37/67
2012 Infoready Pty Ltd37
Counting Is Difficult
Mark Smith
6/12/1978443-43-0000
Mark R Smith(707) 433-0000DL: 00001234
File 1
File 2
8/2/2019 Tristan Sternson & Jeff Jonas
38/67
8/2/2019 Tristan Sternson & Jeff Jonas
39/67
2012 Infoready Pty Ltd39
Data Triangulation
Mark Smith
6/12/1978443-43-0000
Mark R Smith(707) 433-0000DL: 00001234
File 1
File 2
Mark Randy Smith443-43-0000DL: 00001234
New Record
8/2/2019 Tristan Sternson & Jeff Jonas
40/67
2012 Infoready Pty Ltd40
Big Data [in context]. New Physics.
?More data: better the predictions Lower false positives
Lower false negatives
?More data: bad data good Suddenly glad your data is not perfect
?More data: less compute
8/2/2019 Tristan Sternson & Jeff Jonas
41/67
2012 Infoready Pty Ltd41
Big Data
Pile of ____ In Context
8/2/2019 Tristan Sternson & Jeff Jonas
42/67
2012 Infoready Pty Ltd42
One Form of Context: Expert Counting
?Is it 5 people each with 1 account or is it 1person with 5 accounts?
?Is it 20 cases of H1N1 in 20 cities or one
case reported 20 times?
?If one cannot count one cannot estimatevector or velocity (direction and speed).
?Without vector and velocity prediction isnearly impossible.
8/2/2019 Tristan Sternson & Jeff Jonas
43/67
2012 Infoready Pty Ltd43
Expert Counting: Degrees of Difficulty
Exactly
Same
Fuzzy
IncompatibleFeatures
Deceit
Bob Jones123455
Bob Jones123455
Bob Jones123455
Robert T Jonnes000123455
Bob Jones123455
bjones@hotmail
Bob Jones123455
Ken Wells550119
8/2/2019 Tristan Sternson & Jeff Jonas
44/67
2012 Infoready Pty Ltd44
Key Features Enable Expert Counting
People Cars Router
Name Make Device IDAddress Model MakeDate of Birth Year ModelPhone License Plate No. Firmware Vers.
Passport VIN Asset IDNationality Owner Etc.Biometric Etc.Etc.
8/2/2019 Tristan Sternson & Jeff Jonas
45/67
2012 Infoready Pty Ltd45
Consider Lying Identical Twins
#123Sue3/3/84UberstanExp 2011
PASSPORT#123Sue3/3/84UberstanExp 2011
PASSPORT
Fingerprint
DNA
Most TrustedAuthority
Sameperson
trust me.
Most TrustedAuthority
8/2/2019 Tristan Sternson & Jeff Jonas
46/67
8/2/2019 Tristan Sternson & Jeff Jonas
47/67
8/2/2019 Tristan Sternson & Jeff Jonas
48/67
2012 Infoready Pty Ltd48
Life Arcs Are Also Telling
Bill Smith4/13/67
Salem, Oregon
Bill Smith4/13/67
Seattle, Washington
Address History
Tampa, FL 2008-2008
Biloxi, MS 2005-2008
NY, NY 1996-2005
Tampa, FL 1984-1996
Address History
San Diego, CA 2005-2009
San Fran, CA 2005-2005
Phoenix, AZ 1990-2005
San Jose, CA 1982-1990
8/2/2019 Tristan Sternson & Jeff Jonas
49/67
2012 Infoready Pty Ltd49
OMG
8/2/2019 Tristan Sternson & Jeff Jonas
50/67
8/2/2019 Tristan Sternson & Jeff Jonas
51/67
2012 Infoready Pty Ltd51
Powerful Predictions
?Prediction with 87% certainty where you will benext Thursday at 5:35pm
?Names of the top 10 people you co-locate with,not at home and not at work
?The Uberstan intelligence service preempts thenext mass protest in real-time
?
A political opponent is crushed and resigns twodays after announcing their candidacy
8/2/2019 Tristan Sternson & Jeff Jonas
52/67
8/2/2019 Tristan Sternson & Jeff Jonas
53/67
2012 Infoready Pty Ltd53
Macro Trends
8/2/2019 Tristan Sternson & Jeff Jonas
54/67
2012 Infoready Pty Ltd54
ValueofData
The Greater the Context, the Greater the Value
Pile of Data
Records Managed(Big) (Ludicrous Big)
Data
in Context
8/2/2019 Tristan Sternson & Jeff Jonas
55/67
2012 Infoready Pty Ltd55
Willingness
toWait
The better thepredictions the
faster they will bewanted.
Why did we haveto wait until the
end of the day forthe smart answer?
Time Is Of The Essence
Relevance(Iffy) (Totally)
Day
Hour
200ms
Batch
Real-Time
8/2/2019 Tristan Sternson & Jeff Jonas
56/67
2012 Infoready Pty Ltd56
Enterprise IntelligenceOne Plausible Journey
Enterprise IntelligenceOne Plausible Journey
8/2/2019 Tristan Sternson & Jeff Jonas
57/67
2012 Infoready Pty Ltd57
ObservationSpace
Sense and Respond
What you know
NewObservations
8/2/2019 Tristan Sternson & Jeff Jonas
58/67
2012 Infoready Pty Ltd58
ObservationSpace
Decide
?Relevance
Finds the Sensor(
8/2/2019 Tristan Sternson & Jeff Jonas
59/67
2012 Infoready Pty Ltd59
Explore and Reflect
ObservationSpace
Decide
?
DirectedAttention
RelevanceFind You
DeepReflection
CuratedData
Pattern
Discovery
RelevanceFinds the Sensor
(
8/2/2019 Tristan Sternson & Jeff Jonas
60/67
2012 Infoready Pty Ltd60
ObservationSpace
Decide
?
DirectedAttention
NEWINTERESTS
DeepReflection
CuratedData
Pattern
Discovery
RelevanceFinds the Sensor
(
8/2/2019 Tristan Sternson & Jeff Jonas
61/67
2012 Infoready Pty Ltd61
ObservationSpace
Decide
?
DirectedAttention
NEWINTERESTS
DeepReflection
CuratedData
Pattern
Discovery
RelevanceFinds the Sensor
(
8/2/2019 Tristan Sternson & Jeff Jonas
62/67
2012 Infoready Pty Ltd62
Closing Thoughts
8/2/2019 Tristan Sternson & Jeff Jonas
63/67
2012 Infoready Pty Ltd63
The most competitive organizations
are going to make sense of what they are observing
fast enough to do something about it
while they are observing it.
8/2/2019 Tristan Sternson & Jeff Jonas
64/67
2012 Infoready Pty Ltd64
Time
Sensemaking
Algorithms
AvailableObservationSpace
Context
Wish This On The Enemy
EnterpriseAmnesia
ComputingPowerGrowth
8/2/2019 Tristan Sternson & Jeff Jonas
65/67
2012 Infoready Pty Ltd65
Time
The Way Forward: Enterprise Intelligence
Sensemaking
Algorithms
AvailableObservationSpace
Context
ComputingPowerGrowth
8/2/2019 Tristan Sternson & Jeff Jonas
66/67
2012 Infoready Pty Ltd66
Related Blog Posts
Data Finds Data
Algorithms At Dead-End: Cannot Squeeze Knowledge Out Of APixel
Puzzling: How Observations Are Accumulated Into Context
Big Data. New Physics.
On A Smarter Planet Some Organizations Will Be Smarter-erThan Others
Your Movements Speak for Themselves: Space-Time Travel Datais Analytic Super-Food!
8/2/2019 Tristan Sternson & Jeff Jonas
67/67
Email: [email protected] [email protected]
Twitter: http://www.twitter.com/jeffjonas http://www.twitter.com/tsternson
Blog: www.jeffjonas.typepad.com www.infoready.com.au
Questions?