The role of NoSQL in the Next Generation of Financial Informatics

The Role of NoSQLIn Next Generation Financial Informatics

December 10, 2015

2Proprietary & Confidential | © 2015 Aerospike Inc. All rights reserved.

■Market Overview ■The role of NoSQL in Financial Services■Customer Use Cases

■Fraud

■Order Management

■How you can achieve Speed at Scale

Agenda

Steve YatkoCEO, Oktay Technology

Financial ServicesMarket Overview


Hours - Days

Analytics Store

Operational Store (Landing Zone – Fast Ingest)

MarkLogicElasticSearch

Deep Archive

Apache StormApache SparkApache IgniteProject Heron

Ingest/GatewayTibco EMSJMS 60 East29 WestTCPFTPIBM MQ

Data Warehouse

(Legacy)TeradataDB2 WH

Actian(ParAccel)PivotalAstor Data/TDNeteeza/DB2 WHExadataMark LogicVeritca

Data Lake(100+ Petabyte)

Tiered External StorageFlash Arrays to

Warm/Cold/Frozen

HadoopMapReduce

RedisIMDGridHazelcastAerospike

Graph Db

Large Footprint

Front Office

Real-Time ProcessingOn The Wire “Fast Data’

Hours Days-Years

No SQLHBase

MongoDBCassandraAerospike

Time Series

ETL

In Memory Cache

Data Operating System – YARN, MESOS

Deeper Analysis

World Class Big Data Ecosystem: “Service Oriented Information”

HDFS Interface

Small Footprint

Surveillance TradingAnalytics

Metrics, Monitoring and Alerts

Search ComplianceRT Discovery & Visualization

Oktay Technology LLC Confidential

RAM-first Storage

Flash-enabled Storage

SQL-BasedMemSQLVolt DB

Kafka

In Memory Fil;esystem

TachyonIgniteFS

Brian BulkowskiFounder and CTO, Aerospike

The Role of NoSQl in Next Generation Financial Informatics


2010Aerospike 2.0 deploys with first at-scale customers

2011

Funding, company launch

2013Aerospike 3.0 adds indexes, analytics integration

2014

Open Source

2015Deployments in Financial Services, Telecom, etc

Aerospike History


Modern Scale Out Architecture

Research Warehouse

Long-term cold storage

HDFS BASED

App ServersFast, stateless

Load BalancerSimple, stateless

High Performance NoSQLOperational Key ValueSession, authentication, account status, cookies, deviceID, IP address, location, segments, trades, debits, billing, prices...

Real-Time DecisionsBest sellers, top scores, trending tweets


Powering Pioneers Across the Internet

MobileAdvertising

Omni-Channel Marketing

Search, Video, Social, Gaming

Web Advertising


1 Million Writes/Sec on Google Compute

New results: 20 nodes, and 4M reads per second

• Aerospike hits 1M writes/sec with 6x fewer servers than Cassandra

• Delivers consistent low latency with no jitter for both read and write workloads


Shared-Nothing System, High Availability

• Every node in a cluster is identical, handles both transactions and long running tasks

• Clients give rich semantics in multiple languages and connect over the network

• Data is replicated synchronously with within the local cluster

• Data is replicated asynchronously across data centers

• Primary key hash RIPE MD160 ( 20 byte ) for extreme collision resistance ( DHT ), Red black tree for records within hash bucket

• Scatter-gather B+ secondary index


The NoSQL Pattern Applied to Inline Fraud Intervention


Fraud Prevention4% of online transactions are fraudulent

False positives cost millions in lost sales

Global criminal agents

New opportunities: Data sourcesAlgorithms

Speed at Scale

Proprietary & Confidential | © 2015 Aerospike Inc. All rights reserved.


Operational Scale Problem

LEGACY DATABASE(Mainframe)

XDR

Decision Engine

DATA WAREHOUSE/DATA LAKE

LEGACY RDBMSHDFS BASED

BUSINESSTRANSACTIONS

Requestfor

Payment

( Mobile Queries ) ( Recommendation )

( And More )

High Performance NoSQL

500Business Trans per sec

5000 Calculations per sec

X =

“REAL-TIME BIG DATA”“DECISIONING”

2.5 M Database Transactions per sec


Advanced Fraud Scoring

Anything can be a “feature”POS type, item sold, price of item, time of day for given itemsBrowser type, screen resolution, item + browser

But you’ll need recent data

And cutting edge libraries

( your private algorithms )


Example IP Addresses

2B to 4B addresses

Unknown number of networks

Different risk models for different network ranges

Recent behavior required


Example – IP based scoring

ID START_IP END_IP Score

1 192.168.4.1 192.168.4.26 20

2 192.168.4.7 192.168.17.0 2

SELECT id, score WHERE START_IP >= this_ip AND END_IP <= this_ip

Not so bad!Too slow, and

too simple


Using SQL

ID IP TIMESTAMP BEHAVIOR

1 192.168.4.1 2015-06-02 12:00 transaction

2 192.168.4.7 2015-06-02 12:01 tweet

Add the most recent 5 minutes of traffic for each IP address and lookup by previous network

INSERT ip, time, behaviorFOR ( network ) SELECT behavior WHERE ip <= network.start_id AND

ip >= network.end_id AND timestamp > now - 600

You can do it in SQL!

Too slow, call the DBAs.


With a NoSQL KVS

key Network list

192.168.4.0 [ net1, net2, net3 ]

192.168.5.0 null

ID START_IP END_IP Score

net1 192.168.4.1 192.168.4.26 20

net2 192.168.4.7 192.168.17.0 2

Key on Class C address List of networks

Batch lookup networks Filter on client

That’s messy.

Two network round trips, it’ll go fast.


With a NoSQL KVS – add behavior

Already have filtered list of networksFor each network, insert into capped or time limited collection

key Network list

192.168.4.0 [ net1, net2, net3 ]

192.168.5.0 null

ID START_IP END_IP Score behavior

net1 192.168.4.1 192.168.4.26 20 { collection }

net2 192.168.4.7 192.168.17.0 2 { collection }

That’s really messy! It’s fast and easy

to explain.


Use Cutting Edge Libraries in Your Favorite App Environment

If it’s in SQL, it’s too late


Modern Scale Out Architecture

App ServersFast, stateless

Load BalancerSimple, stateless

High Performance NoSQLOperational Key ValueSession, authentication, account status, cookies, deviceID, IP address, location, segments, trades, debits, billing, prices...

Real-Time DecisionsBest sellers, top scores, trending tweets

Scale and Compute Data

Enabled by Fast Networks

Least Vendor Lock-in


The NoSQL Pattern Applied to Order Management


Financial Services – Order Management Systems

Challenge• Must update stock prices, show balances on 300

positions, process 250M transactions, 2M updates/day

• Scale-out requirement

• Multi-asset aggregation for combined:TradingRiskSurveillanceCustomer

• Real-time data view and computation

Speed at Scale



Individual data silos• No global, integrated data view

• No simple post-order visibility

• Often built on non-scalable technologies

• Little shared operational infrastructure

DB1 DB2 DB3

Order ManagementSystem



FIXED INCOMEEQUITIES DERIVATIVES



Shared Data OMS• True multi-asset aggregation

• Supports real-time risk models

• Immediate customer visibility of order state

• Natural NoSQL data modelincluding parent-child and fills

• Microsecond performance withpersistence

• Scale-out architecture







Hybrid order management• Existing order management code unchanged

• Order states placed onto message bus

• Recorded in unified NoSQL storage

• Achieve unified multi-asset aggregated view

• Storm and Spark integration

• Beyond RAM-based processing solutions

DB1 DB2 DB3





MESSAGE BUS (KAFKA, etc)COMPUTE

DATA


How Much Data Can I Use?


Multiple Engagements

10B to 50B objects

300 T before HA replication

2 M TPS

50 milliseconds per 5,000 queries

Local and remote DR


1 Million TPS on Flash per server

Options for storage on a database before Aerospike:

RAM, which was fast, but allowed very limited storage Disk, which allowed for a lot of storage, but was limited in

speed

Intel achieved 1M TPS using 4 Intel P3700 SDs with 1.6 TB capacity on a single Aerospike server. The cost per GB is a fraction of the cost of RAM, while still having very high performance.


Aerospike Has the Experience

Flash optimized from the beginning

5 9’s reliability( longest running cluster: 5 minutes of outage in 4.5 years )

Multi-million TPS clusters in production today

Enterprise Support of your POC & NoSQL design

Used in financial services, telecom, gaming( and adtech of course )


Questions?

@aerospikedb

[email protected]@bbulkow

mailto:[email protected]


High PerformanceNoSQL Database

Powering NewOpportunities at Scale

@aerospikedb

NEXT STEPS:

See how much you can save with Aerospike:http://www.aerospike.com/tco-calculator/

Ready to get started?http://www.aerospike.com/quick-start/

If you have any questions or want to further explore if Aerospike is right for you, contact us:[email protected]

http://www.aerospike.com/tco-calculator/






http://www.aerospike.com/quick-start/

http://www.aerospike.com/quick-start/

mailto:[email protected]

Date post:	16-Apr-2017
Category:	Technology
Upload:	aerospike-inc
View:	570 times
Download:	1 times