Post on 05-Nov-2021
transcript
Graph Learning in Financial Networks
Jiaxuan YouStanford University
Collaboration with Tianyu Du, Fan-yun Sun, Jure Leskovec
Financial Networks
§ Financial Networks: Describe financial entities and their connections
Jiaxuan You, Stanford University 2Image credit: The Political Economy of Global Finance: A Network Model Image credit: https://dailyblockchain.github.io/
International banking• Nodes: Countries• Edges: Capital flows
Bitcoin transactions• Nodes: BTC wallets• Edges: Transactions
Graph Learning in Financial Networks
§ Goal: A graph learning framework for financial networks§ Applications: Fraud detection, Anti-money laundering, Anomaly detection§ Solution: Graph representation learning!
Jiaxuan You, Stanford University 3
Input: Financial networks
Node-level: Fraudsters, …
Edge-level: fraudulent/anomalous transactions, …
Output: Predictions
Subgraph-level: Money laundering subnetworks, …
Graph Neural Network
Why Graph Representation?
§ Transaction-based approach§ “On 01/03, Client 𝐴 sends Company 𝐵 $500”§ Build models based on transaction attributes§ Issues: ignore the context of a transaction
§ Graph-based approach§ Represent transactions as a dynamic graph§ Predictions are made based on the entire graph§ Benefits:
§ Represents a transaction with a broader context§ Requires fewer feature-engineering
Jiaxuan You, Stanford University 4
$400, 01/05
bankclient company
$100, 01/01
$500, 01/03
$200, 01/02
$200, 01/02
$100, 01/06
Overview of This Talk
ROLAND: GNN for Financial
Networks
Results & Insights from
ROLAND
Application: Anomaly Detection
5Jiaxuan You, Stanford UniversityGraph Neural Networks
bankclient company
NN
NN
NN
NN
NN
NN
$400, 01/05
ROLAND: GNN for Financial Networks
6
bankclient company
$100, 01/01
$500, 01/03
$200, 01/02
$200, 01/02
Will there be a transaction? YesWhat is the amount? $500When will it happen? 01/03…Is it a fraud? NoIs it money laundering? Yes…
Graph Neural NetworksFinancial networks Learning Signals & Application
$100, 01/06
bankclient company
Self-supervised(from raw data)
Supervised(from external sources)
bankclient company
NN
NN
NN
NN
NN
NN
?
§ ROLAND framework:§ Transform financial networks as GNN
computational graphs§ Learning from diverse signals
Jiaxuan You, Stanford University
J. You, T. Du, J. Leskovec. ROLAND: Graph Neural Networks for Dynamic Graphs, in submission
node embeddings
ROLAND Model: From Static to Dynamic GNN
Jiaxuan You, Stanford University 7
GNN Layer 1
Graph !
Pred "
GNN Layer 2
#(")
#($)
a) A static GNN with modern architectural design options
Skip connections
LayerconnectivityGNN Layer
GNN Layer
GNN Layer
Intra-layer Design: 4 dims
MLP Layer
MLP Layer
Messagepassinglayers
Post-processlayers
MLP Layer
MLP Layer
Pre-processlayers
Linear
BatchNorm
Dropout
Activation
Attention
Aggregation
Transformation
GNN Layer
GNN Layer
GNN Layer
MLP Layer
MLP Layer
Post-processlayers
MLP Layer
MLP Layer
Pre-processlayers
GNN Layer
GNN Layer
GNN Layer
Linear
BatchNorm
Dropout
Activation
Attention
Aggregation
Aggregation
Attention
Activation
Dropout
BatchNorm
LinearGNN Layer 1
Snapshot !%&"
Pred "%
GNN Layer 2
Embedding update
Embedding update
#$%&"(")
#%&"(")
#$%&"($)
#%&"($)
GNN Layer 1
Snapshot !%
Pred "%'"
GNN Layer 2
Embedding update
Embedding update
#$%(")
#%(")
#$%($)
#%($)
#%&"(")
#%&"($)
b) Extend static GNNs to dynamic GNNs
……
Hierarchical Node state #% = {#%" , … , #%(()}
Question: Can we adapt a SOTA static GNN to dynamic prediction tasks?
t+1t+2 t+3
This transaction is fraudulent
SOTAStatic GNN
ROLAND Model: From Static to Dynamic GNN
Jiaxuan You, Stanford University 8
§ Idea: Recurrently update node embeddings at each layer
§ Introduce a new module to a static GNN:
§ Benefits:§ Simple and effective§ Benefit from the SOTA designs of a static GNN
Static GNN Dynamic GNN
Embedding update
Input:§ Previous embeddings from the
the same layer§ Current embeddings from the
previous layerOutput: Updated embeddings
GNN Layer 1
Graph !
Pred "
GNN Layer 2
GNN Layer 1
Snapshot !!"#
Pred "!
GNN Layer 2
Embedding update
Embedding update
GNN Layer 1
Snapshot !!
Pred "!$#
GNN Layer 2
Embedding update
Embedding update
……
ROLAND Training: Efficient Training
§ Incremental training: § Only keep these in GPU
§ GNN Model 𝐺𝑁𝑁!§ Historical node states 𝐻!"#§ Incoming new graph snapshot 𝐺!§ Efficient and work well in practice
§ Meta training:§ Train a meta-GNN that can quickly adapt
to new data§ Benefits: ROLAND does not need to be
frequently retrained9Jiaxuan You, Stanford University
Snapshot!!"#
"!
!##!"#
Evaluate
!##!
Train
Time !
"$!
State%!
Snapshot!!
"!$#
!##!
Evaluate
!##!$#
Train
"$!$#
…
Time ! + #
…
Live-update evaluation pipeline
Only this part in GPU
Overview of This Talk
ROLAND: GNN for Financial
Networks
Results & Insights from
ROLAND
Application: Anomaly Detection
10Jiaxuan You, Stanford UniversityGraph Neural Networks
bankclient company
NN
NN
NN
NN
NN
NN
ROLAND Implementation: GraphGym
§ GraphGym offers a standardized pipeline for graph learning§ Design Space of Graph Neural Networks (NeurIPS 2020 spotlight)§ Now officially part of PyG 2.0! (Stay tuned for my next talk :)
11
General GNN design space
Clean interface for comparing designs
Jiaxuan You, Stanford University
J. You, R. Ying, J. Leskovec. Design Space of Graph Neural Networks, NeurIPS 2020
§ We implement the financial prediction pipeline with GraphGym§ We adopt many optimal design choices from GraphGym
An Example Financial Dataset
Jiaxuan You, Stanford University 12
§ Transactions between companies captured by a bank clearing system§ 336 million transactions, 6 million accounts, 6 years of daily data
§ Task setting: Supervised learning§ Unbalanced labels: 2% of all the transactions are fraudulent§ Random dataset split: 80% training, 10% validation, 10% testing§ Metric: AUC on test set (random guess has AUC 0.5, higher is better)
§ Best non-deep learning baseline§ AUC: ~0.8
§ Simple GNN implementation using out-of-the-box PyG§ AUC: 0.81
§ ROLAND framework works much better!§ AUC: 0.94
Task 1: Classify Fraud Transactions
13Jiaxuan You, Stanford University
Task 2: Forecast Future Transactions
§ Task setting: Self-supervised/Unsupervised learning§ Rolling prediction: On each day, use all the historical information to predict the
transactions on the next day§ Metric: Mean reciprocal ranking (MMR) of ground-truth future transactions
Jiaxuan You, Stanford University 14
ROLAND significantly outperforms SOTA baselines
Analysis: ROLAND’s Performance over Time
Jiaxuan You, Stanford University 15
ROLAND maintains a robust performance over transaction pattern changes
# of retraining epochs
Almost no retraining needed!
Test MRRRecall@1
(week)day of the year
Transaction amount
Analysis: Interpreting ROLAND’s Predictions
§ Question: What kind of transactions the model paid attention to?
§ Method: Inspect edges' attention scores of the last GNN layer.§ Select edges with low/high attention scores.§ Get the feature value distribution of these
edges
Jiaxuan You, Stanford University 16
Domestic -> Abroad
Non-financial àHousehold
Higher attention to large amount transactions
Overview of This Talk
ROLAND: GNN for Financial
Networks
Results & Insights from
ROLAND
Application: Anomaly Detection
17Jiaxuan You, Stanford UniversityGraph Neural Networks
bankclient company
NN
NN
NN
NN
NN
NN
§ Goal: Detect anomalous transactions without external supervision§ Combine Graph Representation Learning with Outlier Detection
Application: Anomaly Detection
18
Input: Financial networks
node embeddings
edge embeddings
Jiaxuan You, Stanford University
Self-supervised objectives
19
Real world Indicators (from baselgovernance.org)
Objectives for GNN
investment funds sent to countries of concern Predict country
Large cash deposits into (company) accounts Predict amount
unusual subject of transaction Predict subject of transaction(link prediction)
high volume of transactions within a short period predict acculumated amount
multiple individuals sending funds to the one beneficiary predict # transactions
Jiaxuan You, Stanford University
“Indicators and patterns of money laundering or terrorist financing”
Objectives for Anomaly Detection
20
t+1t+2 t+3
Self-supervised Objectives● Objective 1: predict amount● Objective 2: predict recipient● Objective 3: predict country● …
dim
|N|
Outlier Detection Objectives
Jiaxuan You, Stanford University
§ Solution: ROLAND framework + diverse objectives
Quantitative Evaluation for Anomaly Detection
● Task: Detect anomalous transactions without supervision● Goal: Simulate real-world anomaly detection use cases○ Corrupt the dataset to see if ROLAND can extract them○ Report precision/recall of corrupted edges detected by ROLAND
○ Ways of corrupting the dataset:■ Amount corruption (~1% transactions)■ Country corruption (~1% transactions)■ Recipient corruption (~1% transactions) → Rewire !
21
A B
C D
A B
C D
Jiaxuan You, Stanford University
Quantitative Evaluation for Anomaly Detection+14.7% Precision +14.2% Recall
Jiaxuan You, Stanford University
Deep learning baseline:
Rule-based baseline: +172% Precision +171% Recall
Jiaxuan You, Stanford University 23
Normal accountsPotential anomalous accounts
Overview of This Talk
ROLAND: GNN for Financial
Networks
Results & Insights from
ROLAND
Application: Anomaly Detection
24Jiaxuan You, Stanford University
Graph Neural Networks
bankclient company
NN
NN
NN
NN
NN
NN
Resources: ROLAND (in submission)Design Space for Graph Neural NetworksLearn more about GNN: PyG.org