NeoTechnologyPolyglotArchitectures

September2016StefanKolmarDirectorFieldEngineering

Agenda

•  Driversforchangeinenterprises•  Requirementscreatedforpolyglotarchitecture•  Tradi<onalapproaches•  Benefitsofusinggraphtechnology•  Neo4jasna<veGraphDB•  ArchitecturesandEcosystems•  Casestudies•  Conclusions

Admin

Driversforchangeinenterprises

• Dataexplosion

Admin

Driversforchangeinenterprises

• Dataexplosion• IoT/Industrie4.0

Admin

Driversforchangeinenterprises

• Dataexplosion• IoT/Industrie4.0• CloudCompu<ng

Admin

Driversforchangeinenterprises

• Dataexplosion• IoT/Industrie4.0• CloudCompu<ng• NeedtouseConnectedData Admin

TheneedofconnecteddataintheEnterprise

ConnectedEnterpriseEmployees,Customers

andPartners

DigitalMesh

InternetofConnected

Things

KnowledgeNetwork

7

Requirementscreatedforpolyglotarchitecture

•  Abilitytohandlelargescaledatavolumes•  Highavailability24x7•  Openinterfaces•  Structuredatatotakeadvantageofconnec<ons•  Abilitytoqueryconnecteddataondemand•  Flexibilitytoconnectandquerydatawithdifferentschema->Usingthetechnologywhichfitsbestdoingthejob

Admin

TradiBonalapproaches

• UsingRDBMSsystems:•  En<tyrela<onshipmodels•  LogicalandphysicalSchema•  SQLqueries

Admin

DatabaseLandscape

1990

RDBMS

2000

RDBMS(Opera<onal)

DataWarehouse

2010

RDBMS

Hadoop

NoSQL

OLAP/DW

Today

RDBMS

Hadoop

NoSQL

OLAP/DW

GraphDB

Spark

Today:sBlltherealitystoringdata

Whatistheproblemwiththat?

Whatistheproblemwiththat?

•  Understandableforbusinessusers?

•  Agility?•  Needsefforttomaterializerela<onships

•  Logicalversusphysicaldatamodel?

•  Op<miza<onneededtoperformqueries?

CaseStudies

What is the most powerful database in the world?

AlternaBveDatabase:GraphDataModel

AdvantagesusingGraphDB

Timetomarket

FlexibilityOpera<onalEfficiency

TheWhiteboardModelIsthePhysicalModel

CAR

name:“Dan”born:May29,1960twifer:“@dan”

name:“Ann”born:Dec5,1995

since:Jan10,2011

brand:“Volvo”model:“V70”

PropertyGraphModelComponents

Nodes•  Theobjectsinthegraph•  Canhavename-valueproper@es•  CanbelabeledRelaBonships•  Relatenodesbytypeanddirec<on•  Canhavename-valueproper@es

FATHEROF

LIVESWITH

LIVESWITHPERSON PERSON

RelaBonalVersusGraphModels

RelaBonalModel GraphModel

KNOWSANDREAS

TOBIAS

MICA

DELIA

Person FriendPerson-Friend

ANDREASDELIA

TOBIAS

MICA

Cypher:PowerfulandExpressiveQueryLanguage

KNOWS

Dan Ann

MATCH(:Person{name:“Dan”})-[:KNOWS]->(:Person{name:“Ann”})

LABEL PROPERTY

NODE NODE

LABEL PROPERTY

MATCH(boss)-[:MANAGES*0..3]->(sub),(sub)-[:MANAGES*1..3]->(report)WHEREboss.name=“JohnDoe”RETURNsub.nameASSubordinate,count(report)ASTotal

ExpressComplexQueriesEasilywithCypher

Findalldirectreportsandhowmanypeopletheymanage,upto3levelsdown

CypherQuery

SQLQuery

ConnectednessandSizeofDataSet

Respon

seTim

e

0to2hops0to3degreesThousandsofconnec<ons

TenstohundredsofhopsThousandsofdegreesBillionsofconnec<ons

Rela<onalandOtherNoSQLDatabases

Neo4j

Neo4jis1000xfasterReducesminutestomilliseconds

Benchmarkreal-BmequeryperformanceNeo4jVsRela<onalandotherNoSQLDatabases

AdvantagesusingNeo4jGraphDB

TimetomarketFlexibilityOpera<onalEfficiency

Queryspeed/Performanceforconnecteddata

SchemafreeTraversingvsJoins

Cypherquerylanguage

WhatisaNaBveGraphDatabase?

Applica<onMATCH(:Person{name:“Dan”})

-[:KNOWS]->(:Person{name:“Ann”})

ProcessingEngine

Agenda

•  Driversforchangeinenterprises•  Requirementscreatedforpolyglotarchitecture•  Tradi<onalapproaches•  Benefitsofusinggraphtechnology•  Neo4jasna<veGraphDB•  ArchitecturesandEcosystems•  Casestudies•  Conclusions

Admin

PaTernsinNeo4jarchitectures

•  PrimaryStore•  Reads+Write

•  SecondaryStore•  Mostlyread,scheduledbatchupdates

•  ODS/MDS•  Mostlyread,scheduledbatchupdates+integra<onlogic

PrimaryStorecase:Telenor

•  Iden<tyandaccessmanagementproblem.Needtocomputeresourceauthoriza<oninreal<me

Wastakingupto20minforlargecustomers->precalcula<on&cache->staledata

Originalarchitecture

500 requests / sec

Middlewareservices

Backend Backend Backend

Channel Channel Channel42 channels

35 systems

Newarchitecture

500 requests / sec

Middlewareservices

Backend Backend Backend

Channel Channel Channel42 channels

35 systems

Diditwork?

•  Response<mesreducedtosecondsandmilliseconds

•  Codemaintainabilityimproved.AccessrulesinCypherasgraphpaferns.

"TheNeo4jgraphdatabasegivesusdras@callyimprovedperformanceandasimplelanguagetoqueryourconnecteddata"–Sebas<anVerheughe,Architect&Developer

SecondaryStoreUseCase:Shutl/eBay

Wantedtocomeupwithquotesfordeliveryslots.Ittook2secondstocomeupwithaquoteforasingledeliveryslotandtheywantedtoextendtheservicetoshow20poten<aldeliveryslots

Originalarchitecturewasamonolithbasedonrails

Originalarchitecture

Requests quote

Railsmonolith Relational

SoluBon

Brokeapplica<onupintoservicese.g.quo<ng,booking,feedback.

Keygoalwastoimprovethespeedofquo<ngandneo4jbackedthisservice

Newarchitecture

Requests quote

Quo<ngservice Relational

Bookingservice

Feedbackservice

Nightly job to populate

ETL

Diditwork?

•  Quo<ngfor20<mewindowsdownfrom82,000msto80ms

•  Codecomplexitymuchreduced

“WefoundNeo4jtobeliterallythousandsof0mesfasterthanourpriorMySQLsolu<on,withqueriesthatrequire10-100<meslesscode.

Today,Neo4jprovideseBaywithfunc<onalitythatwaspreviouslyimpossible.”VolkerPacher,SeniorDeveloper,eBay

WhoelseusesNeo4jasadatasourcetofeedmissioncriBcalapplicaBons?

•  World’slargestparcelservice•  Theyno<cedthattheywouldnothavebeenabletodeliverpackagesbyChristmas2013

•  Hierarchicalrou<ngsystem(RDBMSbacked)•  ReplaceditwithNeo4jusinglateralrou<ng

•  Value:•  Minimized<me•  Maximizedusageofroadnetwork

•  Recommenda<onengines•  Frauddetec<on/preven<onarchitectures

Neo4j

WebApp Data

Integra<on

RDBMS(Oracle,MySQL,DB2,HANA…)

ManagementConsole(E.gBIToolssuchasTableau,Qlik,BO,MicroStrategyetc)

FraudAnalyst

Machine2Machinegeneratedac<ons

Alert

IncomingEvents

CRMSystem

Opera<onalSystem

ExternalData

SampleArchitectureforFraudDetecBon

SampleArchitectureforInternetRetailRecommendaBons

Neo4j

InternetShop

DataIntegra<on

InvoicingShipping

etc

User

Opera<onalSystem

ExternalData

CurrentProfiles

ODS/MDScase:Gov.Agency

•  Thegraphintegratesdatafrommul<plesources•  ODS:Addi<onalopera<onsontheintegrateddata•  MDS:Providereferencedatatootheropera<onalsystems

•  Europeangovernmentagencies

TradiBonalODS/MDSarchitecture

Relational ODS

Data Source 1

Data Source 2

Semi/unstructureddatasource

ETL

Opera<onalBI

Relational MDS

EDW

Opera<onalSystems

GraphbasedODS/MDSarchitecture

ETL

Real<meconnecteddataanalysisandexplora<on

EDW

Opera<onalSystems

Data Source 1

Data Source 2

Semi/unstructureddatasource

CommonInsuranceScenario

Internalusers

DataLayer

BusinessLogic

-DataConsumersareemployeesorSalesagents-BuiltonMainframes-RichBusinessLogic-DuringOfficeHourssuppor<ngOLTPqueries-Nightlybatchruns-Noreal<mesupport

24x7availability

BusinessLogic

Suppor<ngRT

DataLayer

“Yesterday”

“Today”

IoTTouchpoints

DeviceManagement

SupportDevicesProvisionDevices

Security/AccessMgmt

GrantAccess

RequestAccess

MDMLayer

SupplierMaster

CustomerMaster

ProductMaster

SupportMaster

…

RevenueSharing

Conclusions:

•  Polyglotarchitecturesarerequiredtosa<sfytheneedsexis<ngandcoming

•  Neo4jna<vegraphdatabaseprovidesacompete<veedge:

•  Agility

•  Flexibility

•  Opera<onalEfficiency

•  TimetoMarket

•  Moreinforma<on:

•  www.neo4j.com

•  Stefan.Kolmar@neo4j.com

NeoTechnologyTheGraphDatabaseLeader

September2016StefanKolmarDirectorFieldEngineering