+ All Categories
Home > Data & Analytics > Using Chaos to Disentangle an ISIS-Related Twitter Network

Using Chaos to Disentangle an ISIS-Related Twitter Network

Date post: 20-Feb-2017
Category:
Upload: steve-kramer
View: 1,891 times
Download: 1 times
Share this document with a friend
36
Finding Key Influencers and Viral Topics in an ISIS- Related Twitter Network Steve Kramer, Ph.D. President & Chief Scientist Paragon Science, Inc. December 2015 Copyright © 2006-2015 Paragon Science, Inc. All rights reserved.
Transcript
Page 1: Using Chaos to Disentangle an ISIS-Related Twitter Network

Finding Key Influencers and Viral Topics in an ISIS-Related

Twitter Network

Steve Kramer, Ph.D.President & Chief Scientist

Paragon Science, Inc.December 2015

Copyright © 2006-2015 Paragon Science, Inc. All rights reserved.

Page 2: Using Chaos to Disentangle an ISIS-Related Twitter Network

Overview

• Research background• Sample Twitter data set related to ISIS/ISIL• K-Core Decomposition– Central URLs– Central Users

• Topic Detection• Sentiment Analysis and Anomaly Detection

2

Page 3: Using Chaos to Disentangle an ISIS-Related Twitter Network

What Are We Doing?Provide valuable intelligence results to clients using our

dynamic anomaly detection software and data mining tools

Many possible application areas: Social media alerting and sentiment change detectionAnalysis of web trends and user activitiesPricing and market trend analysis and alertingNetwork defense against cyberattacksInsider threat detectionFraud prevention (banking, insurance, online auctions,…)Healthcare data mining

Paragon Science, Inc. 3

Page 4: Using Chaos to Disentangle an ISIS-Related Twitter Network

How Is It Done Today?Existing approaches

Standard SNA metricsRule-based systems (transaction profiling, etc.)Bayesian and other statistical/probabilistic modelsMachine learning tools (neural nets, HMMs, etc.)

Some limitations of existing methodsTraining requirements can be large for neural nets.For rule-based systems, it is difficult to effectively predict or

define new “bad” anomalies or patterns in advance. Many current methods are not scalable to real-world

operational requirements.

Paragon Science, Inc. 4

Page 5: Using Chaos to Disentangle an ISIS-Related Twitter Network

What Is New in Our Patented Approach?

• A powerful anomaly detection approach that incorporates nonlinear time series analysis methods– US Patent #8738652 (1.usa.gov/1kkyVD9)

“Systems and Methods for Dynamic Anomaly Detection”

• Key questions answered:– Which entities behave or evolve differently than others in the

data set?– Which entities have shifted their behavior unexpectedly?

Paragon Science, Inc. 5

Page 6: Using Chaos to Disentangle an ISIS-Related Twitter Network

What Is New in Our Patented Approach? (Cont’d.)

Our framework inherently captures the dynamics of the entities under study, without having to specify in advance normal vs. abnormal behavior.

We can simultaneously analyze the time evolution of Network structures Any associated attributes (text terms, geospatial position, etc.)

Our technique is robust with respect to missing or erroneous data.

As result, we can Find key players in rapidly changing networks Provide early warning of viral videos and online documents Focus attention on the most-anomalous events or transactions

Paragon Science, Inc. 6

Page 7: Using Chaos to Disentangle an ISIS-Related Twitter Network

Dynamic Anomaly Detection OverviewA general approach that incorporates nonlinear time

series analysis methodsComplexity measuresFinite-time Lyapunov exponents (FTLEs)

Input dataCommunications or transactional data streamsGeneral time-dependent data sets

Key questionsWhich entities behave or evolve differently than others in the

data set?Which entities have shifted their behavior unexpectedly?

Paragon Science, Inc. 7

Page 8: Using Chaos to Disentangle an ISIS-Related Twitter Network

Finite-Time Lyapunov Exponents (FTLEs)

• General dynamical system

• Flow map– Advects points in the state

space– Describes the time

evolution of the system

Paragon Science, Inc. 8

Page 9: Using Chaos to Disentangle an ISIS-Related Twitter Network

Finite-Time Lyapunov Exponents (FTLEs)

• FTLEs characterize the amount of stretching or contraction about a point x0 during a time interval T– Stability– Predictability

• Definition

Paragon Science, Inc. 9

Page 10: Using Chaos to Disentangle an ISIS-Related Twitter Network

Derived Jacobian Vectors• Similarly, characteristic vectors derived from the flow

map’s Jacobian can describe the generalized directions of the local stretching or contraction.

• Possible derivation approaches:– Weight-based column sampling– Singular value decomposition (SVD)– Principal component analysis (PCA)

Paragon Science, Inc. 10

Page 11: Using Chaos to Disentangle an ISIS-Related Twitter Network

Paragon Science Dynamic Anomaly Detection

Paragon Science, Inc. 11

Representation of Data at t=ti

Cluster Resolution

Feature Vector Encoding

Outlier Detectionat t=ti

3+ Time Intervals?

No

Yes

Clustering /Segmentation

Dynamic Anomaly Detection

Nonlinear Time Series AnalysisFTLEs, Dynamic Thresholds, etc.

PatternClassification

Outlier Detection

Domain-Specific FilteringThreat Signatures,Risk Profiles, etc.

Page 12: Using Chaos to Disentangle an ISIS-Related Twitter Network

ISIS-Related Twitter AnalysisSample data set from Twitter API collected using

twittertap:Date range: 11/30/2015 – 12/10/20152,541,812 tweets7,802,210 generated links with hashtags, URLs, and user replies

Research planPerform k-core decompositionRun anomaly detection software on sub-networks of nodes in the

central core to find the most influential users and most viral URLsCarry out community detection, topic detection, and sentiment

analysis

Paragon Science, Inc. 12

Page 13: Using Chaos to Disentangle an ISIS-Related Twitter Network

ISIS-Related Twitter Network

Paragon Science, Inc. 13

User A User B

User C

replies to

mentions

URL 1 URL 2

Hash Tag 1

Hash Tag 2

references

uses

uses

references

Link Type # LinksUser links to URL 2,014,572User mentions user 2,867,633User references hashtag 2,699,875User references symbol 2,636User replies to user 215,343

Page 14: Using Chaos to Disentangle an ISIS-Related Twitter Network

K-core DecompositionThe k-core of a graph is a maximal subgraph in which

each vertex has at least degree k. The coreness of a vertex is k if it belongs to the k-core but not

to the (k+1)-core. The k-core decomposition is performing by recursively

removing all the vertices (along with their respective edges) that have degrees less than k.

The k-core decomposition of a network can be very effective in identifying the individuals within a network who are best positioned to spread or share information. M. Kitska, et al., “Identifying influential spreaders in complex networks,”

arXiv:1001.5285v1 [physics.soc-ph] (2010).

14

Page 15: Using Chaos to Disentangle an ISIS-Related Twitter Network

K-Core Decomposition of the ISIS Network

Paragon Science, Inc. 15

http://sourceforge.net/projects/lanet-vi/

Page 16: Using Chaos to Disentangle an ISIS-Related Twitter Network

Central Core of the ISIS Network

Paragon Science, Inc. 16

Users at the center of the k-core decomposition are positioned well to spread information and influence the network.

Page 17: Using Chaos to Disentangle an ISIS-Related Twitter Network

Top URLs in the Central Core

Paragon Science, Inc. 17

URL Web Page Title Coreness # Linkshttp://www.mirror.co.uk/news/uk-news/isis-would-love-you-bomb-6941441#ICID=sharebar_twitter

ISIS would love you to bomb them to bring about apocalyptic final fight, says journalist who lived among terrorists - Jurgen Todenhofer - Mirror Online

89 398

https://www.youtube.com/watch?v=nVDiK3J9PKQ

How to Paralyse & Eliminate ISIS in Less Than 24 Hours - Younus AlGohar - YouTube

89 384

http://shr.gs/Um8lnCZ Jihadi BILLIONAIRES: ISIS top terror rich list“ but how are they blowing all the dough?

89 349

https://www.youtube.com/watch?v=FS9iPz-cPlY Humanity Under Attack! What Must Be Done Now? - Younus AlGohar - YouTube

89 331

http://is.gd/txNkng How to Paralyse & Eliminate ISIS in Less Than 24 Hours - Younus AlGohar

89 327

http://bbc.in/aggad Paris attacks: Bataclan third attacker identified - BBC News

89 317

http://ti.me/1XPKXcx London Subway Attacker Had ISIS Images on Phone: Officials

89 317

http://dailym.ai/1NFIp5L ISIS releases its latest video as they execute two ˜sorcerers” in Libya | Daily Mail Online

89 298

http://youtu.be/mXOSQj4xjPY Fitna-e-Khwarij - YouTube 89 259http://www.telegraph.co.uk/news/worldnews/northamerica/usa/12037849/Majority-of-Americans-support-sending-ground-troops-to-fight

Majority of Americans support sending ground troops to fight Isil

89 255

Page 18: Using Chaos to Disentangle an ISIS-Related Twitter Network

Top 5 URLs in the Central Core

18

Page 19: Using Chaos to Disentangle an ISIS-Related Twitter Network

Top Users in the Central Core

Paragon Science, Inc. 19

User Coreness # LinksMailOnline 89 6255David_Cameron 89 3330Telegraph 89 2072TarekFatah 89 1907BBCWorld 89 992younusalgohar 89 977mehdifoundation 89 830rafu007 89 791TIMEWorld 89 700niallboylan4fm 89 667

Page 20: Using Chaos to Disentangle an ISIS-Related Twitter Network

Topic Detection in the ISIS Twitter Network

Paragon Science, Inc. 20

User A User B

User C

replies to

mentions

URL 1 URL 2

references

Term 1

Term 2

Term N

Term 3

Topic 1

Topic 2

Topic M

146 Topics Detected

Page 21: Using Chaos to Disentangle an ISIS-Related Twitter Network

Title-to-Term Network for Topic Detection

21

Page 22: Using Chaos to Disentangle an ISIS-Related Twitter Network

Title-to-Term Network for Topic Detection

22

Page 23: Using Chaos to Disentangle an ISIS-Related Twitter Network

Topic 3 Communities of Users

23

Page 24: Using Chaos to Disentangle an ISIS-Related Twitter Network

Topic 3 Top 10 Web Sites

24

Page 25: Using Chaos to Disentangle an ISIS-Related Twitter Network

Topic 3 Selected Users

25

Page 26: Using Chaos to Disentangle an ISIS-Related Twitter Network

Topic 4 Top Web Sites

26

Page 27: Using Chaos to Disentangle an ISIS-Related Twitter Network

Incorporating Sentiment Analysis• Incorporate sentiment analysis scores as an input to

dynamic anomaly detection in order to track the propagation of references to websites with particular emotions.

• Use the LIWC (Linguistic Inquiry and Word Count) tool to calculate the sentiment scores of the web pages.– Prof. James Pennebaker from UT Austin (

http://liwc.wpengine.com/) – Sample categories

• Positive emotion• Negative emotion• Anger• Anxiety

Page 28: Using Chaos to Disentangle an ISIS-Related Twitter Network

Top Web Pages by AnxietyWeb Page Title URL Anxiety ScoreWatch Daniel Scavino Jr.'s Vine "POTUS on terrorism."

https://vine.co/v/i71FvOKlYgv 11.11

*WARNING: New ISIS VIDEO: Muslim Children Execute Captives, Obama, we will behead you, as we will do to all the Jews | Pamela Geller

http://bit.ly/1TMcgif 6.51

The Mastermind Of The San Bernardino Massacre Has All The Hallmarks Of An ISIS Terrorist Attack... - Linkis.com

http://ln.is/shoebat.com/2015/12/PGcNB

5.56

The Far-Reaching Effects of Global Terrorism - YouTube

http://youtu.be/L_qr01yHoQs 4.85

Terrorism isn't scaring Americans; Obama is by Andrew Malcolm - Investors.com

http://news.investors.com/politics-andrew-malcolm/120715-784023-obama-isis-speech-no-new-strategy.htm

4.03

57 Paris airport workers on terror watch list, “Allahu akbar” scrawled on fuel tank

http://www.jihadwatch.org/2015/12/57-paris-airport-workers-on-terror-watch-list-allahu-akbar-scrawled-on-fuel-tank

3.03

Ian56 https://twitter.com/Ian56789: DIA Emails: ISIS was deliberately armed and funded by Obama & Hillary Clinton

http://ian56.blogspot.com/2015/06/the-terrorist-threat-has-been.html?m=1

2.94

Page 29: Using Chaos to Disentangle an ISIS-Related Twitter Network

Top Web Pages by Negative Emotion RatioWeb Page Title URL Negative/

Positive Emotion Score

Russian airstrike 'kills family in their car' as bombs obliterate ISIS oil convoy | Daily Mail Online

http://dailym.ai/1IIU2Yz 21.9

Study: Unprecedented support for ISIS in the U.S. - CNNPolitics.com

http://cnn.it/1XF0p61 13.3

US-led coalition not striking ISIS oil trucks despite evidence – Russia’s General Staff” RT News

http://on.rt.com/6y9c 12.1

ISIS PARIS TERRORIST Recruited Fighters at Hungarian Refugee Camp - YouTube

https://www.youtube.com/watch?v=88TJBvH1zzg

11.9

U.S. rejects Russia’s claim of Turkey’s cooperation with ISIS

http://goo.gl/Q9MWGk 11.8

Islamic State's Sinai chief said in Gaza to coordinate with Hamas | The Times of Israel

http://bit.ly/1N6bqZa 10.0

Is ISIS Entering US Through Mexico? Amid Islamic State Fears, Border Patrol Captures Afghan, Pakistani Men Being Smuggled Into Country

http://bit.ly/1l9Mxo1 9.8

Why Can't White House Just Say ISIS Beheaded Christians? - Investors.com

http://ift.tt/1zMpWNz 8.6

For the Record: How Stubborn U.S. Leaders May Be Hurting the Fight Against ISIS on Vimeo

https://vimeo.com/147860012 8.4

Just 0.4 Percent of Syrian Refugees Admitted to U.S. Since Paris Attacks Are Christian - Breitbart

http://www.breitbart.com/big-government/2015/12/08/just-0-4-percent-syrian-refugees-admitted-u-s-since-paris-attacks-christian/

8.1

Page 30: Using Chaos to Disentangle an ISIS-Related Twitter Network

Paragon Science, Inc. 30

Mapping Anomalies to Source DataAnomalies Discrete/Continuous

Attribute DistributionsRelated Source

Data

Where and when are the hotspots of changes?

Which nodes and attributes were involved in each anomalous peak?

Page 31: Using Chaos to Disentangle an ISIS-Related Twitter Network

Anomaly Detection Results for Websites with Negative Emotions

Paragon Science, Inc. 31

Surge of Twitter user links to web page with high negative emotion score: “The ISIS Trail of Death - NBC News”

Page 32: Using Chaos to Disentangle an ISIS-Related Twitter Network

Summary of Top 50 Negative Emotion Anomalies

32

Web Page Title Peak Start Peak End Max Change Metric

# Anomalies

The ISIS Trail of Death - NBC News 2015-12-08 03:36:39

2015-12-09 13:36:39

3.01 24

Russia strikes ISIS targets in Syria from sub in Mediterranean for first time (VIDEO) RT News

2015-12-09 07:36:39

2015-12-09 16:36:39

2.33 8

US Air Force running out of bombs to fight ISIS | Fox News

2015-12-06 07:36:39

2015-12-06 21:36:39

2.10 2

If you keep saying Saudi Arabia is like ISIS, you might get sued - The Washington Post

2015-12-02 04:36:39

2015-12-07 09:36:39

2.01 11

Everyone knows what’s going on: Istanbul residents on Turkey-ISIS oil trade — RT News

2015-12-04 15:36:39

2015-12-04 16:36:39

1.96 2

Is ISIS Entering US Through Mexico? Amid Islamic State Fears, Border Patrol Captures Afghan, Pakistani Men Being Smuggled Into C

2015-12-03 15:36:39

2015-12-03 15:36:39

1.91 1

Iran news in brief, 30 November 2015 - YouTube

2015-12-01 17:36:39

2015-12-01 17:36:39

1.90 1

No Christians: All 132 Syrian Refugees Admitted to U.S. Since Paris Attacks Are Sunni Muslims

2015-12-01 19:36:39

2015-12-01 19:36:39

1.89 1

Page 33: Using Chaos to Disentangle an ISIS-Related Twitter Network

Most-Anomalous Negative Emotion ISIS Web Page Shared by Twitter Users

33

Page 34: Using Chaos to Disentangle an ISIS-Related Twitter Network

Twitter Network Related to Negative Peak

34

Page 35: Using Chaos to Disentangle an ISIS-Related Twitter Network

Paragon Science, Inc. 35Paragon Science, Inc. 35

What Are the Payoffs?• Quickly identify key influencers and trends in online

networks, incorporating sentiment analysis scores to track the viral spreading of emotions

• Provide early warning of viral videos, anomalous web events, or unusual network traffic

• Enable enhanced business intelligence without having to specify normal vs. abnormal behavior in advance

Page 36: Using Chaos to Disentangle an ISIS-Related Twitter Network

Third-Party Software Acknowledgements Paragon Science gratefully acknowledges the following researchers and software providers:

Cytoscape (http://www.cytoscape.org/) Lanet-vi (http://sourceforge.net/projects/lanet-vi/)

o J. Alvarez-Hamelin, et al., "Understanding Edge Connectivity in the Internet through Core Decomposition," Internet Mathematics 7 (1): 45–66, 2011.

LIWC (Linguistic Inquiry and Word Count) (http://liwc.wpengine.com/) o Y.R. Tausczik and J.W. Pennebaker, “The psychological meaning of words: LIWC and

computerized text analysis methods,” Journal of Language and Social Psychology, 29, 24-54.a, 2010.

Louvain community detection software (http://perso.crans.org/aynaud/communities/)o V. Blondel, et al., “Fast Unfolding of Communities in Large Networks,” Journal of

Statistical Mechanics: Theory and Experiment, 10, P10008, 2008. Networkx (https://networkx.github.io/)

o A Hagberg, D Conway, "Hacking social networks using the Python programming language (Module II - Why do SNA in NetworkX)", Sunbelt 2010: International Network for Social Network Analysis.

Paragon Science, Inc. 36


Recommended