Graph OrientedDatabases

Ciaociao

Vai a fare

ciao ciaoDr. Fabio Fumarola

Outline• Introduction• The Lack of relationship for RDBMS and NoSQL• Graph Databases: Features• Relations• Query Language• Data Modeling with Graphs• Conclusions

2

Introduction• We live in a connected world• Everything is connected: Social Network, Biology,

Bioinformatics• The NoSQL databases analyzed store data using

aggregate• Here we compare graph databases with relational

databases and aggregate NOSQL in storing graph data

3

Three Facts1. Relational Databases Lack Relationships

1. NoSQL Databases also Lack Relationships

2. Graph Databases Embrace Relationships

4

Relational Databases Lack Relationships

• For decades we tried to accommodate connected, semi-structured datasets inside relational databases.

• But: – relational databases are designed to codify tabular

structures– They struggle when modeling ad hoc exceptional

relationships that are in real world.

5

Relational Databases Lack Relationships• Relationships in relational database only mean joining tables• But we want to model the semantic of relationships that

connect real world• As outlier data multiplies:

1. The structure of the dataset becomes more complex and less uniform

2. The relational data becomes more complex and less uniform (large join tables, sparsely populated rows, a lot o null values)

6

Example of customer-centric orders• Complex joins• Foreign key constraints• Sparse table with null

values• Reciprocal queries are

costly “What products did a customer buy?”

7

NoSQL Databases Also Lack Relationships

• key-value, document, or column-oriented store sets of disconnected documents/values/columns

• One well-known strategy for adding relationships is to embed an aggregate’s identifier inside the field belonging to another aggregate

• But this require joins at the application level• Some NoSQL have some concept of navigability but it is

expensive for complex joins

8

Example of aggregate oriented orders

• Some properties are references to foreign aggregates

• This relationship are not first-class citizens

• Are not intended as real realtionships

9

Example of a Small Social Network• it’s easy to find a user’s

immediate friends• friendship isn’t always

reflexive• We can have brute-force

scan across the whole dataset looking for friends entries

10

Graph Databases Embrace Relationships

• The previous examples have dealt with implicitly connected data

• We infer semantic dependencies between entities• We model the data based on this connections• Our application have to navigate on this flat and disconnected

data, and deal with slow queries.• In contrast, in the graph world, connected data is stored as

connected data

11

Example Social Network

12

• The node user:Bob is a Vertex with a property Bob

• We also see relations which are Edges:• Boss_of• Friend_of• Married_to

GRaph DB Features

13

CAP Theorem

14

Consistency• Since Graph DBs operate on connected nodes, they

could not scale well distributing nodes across servers.• There are solutions supporting distribution:

– Neo4j uses one master and several slaves– OrientDB uses MVCC for distributed eventual data

structures– TitanDB partition data by using HBase or Cassandra

15

Transactions• Most of the Graph DB are ACID-compliant• Before doing an operation we have to start a

transaction.• Without wrapping operations in a transaction we will

get an Exception.

16

Availability• Neo4j from version 1.8 achieves availability by

providing for replicated slaves.• Infinity Graph, FlockDB and TitanDB provides for

distribute storage of the nodes.• Neo4J uses Zookeeper to keep track of the last

transaction Ids persisted on each slave node and the current master node

17

Going Into Relations

18

Relations• Relations in a graph naturally forms paths.• Querying or traversing the graph involves following a

path.• A query on the graph is also known as traversing the

graph• As advantage we can change the traversing

requirements without changing nodes and edges

19

Relations• In graph databases traversal operation are highly

efficient.• In the book Neo4j in Action, Partner and Vukotic

perform an experiment comparing relational store and Neo4j

20

Relations

• In a depth two (friend-of-friend), both relational db and graph db perform well enough

• But when we do the depth three it clear that relational db can no longer deal

21

Relations

• Both aggregate store and relational databases perform poorly because of the index lookups.

• Graphs, on the other hand, use index-free adjacency list to ensure that traversing connected data is extremely fast.

22

Relations another case study• Let us consider the

purchase history of a user as connected data.

• If we notice that users who buy strawberry ice cream also buy espresso beans, we can start to recommend those beans to users who normally only buy the ice cream.

23

Relations and Recommendations• The previous was a one dimensional

recommendation• We can join our graph with graph from other

domains.• For example, we can ask to fine

– “all the flavors of ice cream liked by people who live near a user, and enjoy espresso, but dislike Brussels sprouts.”

24

Relations and Patterns• We can use relations to query graph-patterns• Such pattern-matching queries are:

– extremely difficult to write in SQL– And are laborious to write against aggregate stores

• In both cases they tend to perform very poorly• In the other hand, graph databases are optimized for

such kind of queries

25

Query Language

26

Query Language• Graph DBs support query

languages such as Gremlin, Cypher and SPARQL

• Gremlin is a DSL for traversing graphs;

• It can traverse all the graph databases implementing the Blueprints

27

1. Indexing: Nodes and Edges • Indexes are necessary to find the starting node to

being traversal.• How Indexes works:

– Can index properties of nodes and edges.– Adds are done in transactions

• Nodes retrieved can be used to raise queries

28

2. Querying In- Out- Relationships

• Having a node we can query both for Incoming and Outgoing relationships.

• We can apply directional filters on the queries when querying for relations

29

3. Querying Breadth- Depth-• Graph databases are really powerful to query for

incoming and outgoing relationships.• Moreover, we can make the traverser go top-down or

sideways on the graph by using:– BREADTH_FIRST or– DEPTH_FIRST

30

4. Querying Paths• An other good feature of graph databases is

– finding paths between two nodes.– Determining if there are multiple paths– finding the shortest path

• Many Graph DBs use algorithms such as the Dijkstra’s algorithm for finding shortest paths.

31

5. Querying Paths• Finally, with Graph DBs it is possible to use Match

operator• The MATCH is used for matching patterns in

relationships• The WHERE filters the properties on a node or

relationship• The RETURN specifies what to get in the result set.

32

Data Modeling with GraphsIntro

33

Data Modeling with Graphs

how do we model the world in graph terms?

34

how do we model the world in graph terms?

• Formalization of the base model• Enrich the model• Testing the model

35

Formalization of the base model• Modeling is an abstracting activity motivated by a particular

need or goal• We model in order to define structures that can manipulated. • There are no natural representations of the world the way it

“really is,” • There are just many purposeful selections, abstractions, and

simplifications that useful for satisfying a particular goal

36

Formalization of the base model• Graph data modeling is different from many other

techniques.• There is a close affinity between logical and physical

models.• In relational databases we start from a logical model

to arrive to the physical model.• With graph databases, this gap shrinks considerably.

37

The Graph Model• A property graph is made up of nodes, relationships,

and properties.

38

Publishing Messages• We organize messages in order

39

NodesNodes contain properties•Think of nodes as documents that store properties in the form of arbitrary key-value pairs. •The keys are strings and the values are arbitrary data types.

40

RelationshipsRelationships connect and structure nodes. •A relationship always has a direction, a label, and a start node and an end node—there are no dangling relationships. •Together, a relationship’s direction and label add semantic clarity to the structuring of nodes.

41

Relationships: AttributesLike nodes, relationships can also have properties. •The ability to add properties to relationships is particularly useful for:

– Providing additional metadata for graph algorithms– Adding additional semantics to relationships (including

quality and weight), – and for constraining queries at runtime.

42

Modeling Steps: Outline• The initial stage of modeling is similar to the first

stage of many other data modeling techniques, that is:– to understand and agree on the entities in the domain– how they interrelate– and the rules that govern their state transitions

43

Describe the Model in Terms of the Application’s Needs• Agile user stories provide a concise means for

expressing an outside-in, user-centered view of the application needs.

• Here’s an example of a user story for a book review web application: – AS A reader who likes a book,– I WANT to know which books other readers who like the

same book have liked, – SO THAT I can find other books to read.

44

Describe the Model in Terms of the Application’s Needs• This story expresses a user need, which motivates

the shape and content of our data model. • From a data modeling point of view:

– the AS A clause establishes a context comprising two entities—a reader and a book—plus the LIKES relationship that connects them.

– The I WANT clause exposes more LIKES relationships, and more entities: other readers and other books.

45

Describe the Model in Terms of the Application’s Needs• The entities and relationships in analyzing the user

story quickly translate into a simple data model

46

Modeling Rationale• Use nodes to represent entities• Use relationships both:

– to express the connections between entities and – to establish semantic context for each entity

• Use relationship direction to further clarify relationship semantics

47

Describe the Model: Guidelines • Use node properties

– to represent entity attributes, plus any necessary entity metadata, such as timestamps, version numbers, etc

• Use relationship properties – to express the strength, weight, or quality of a relationship,

plus any necessary relationship metadata, such as timestamps, version numbers, etc.

48

Modeling Temporal Relations as Nodes• When two or more domain entities interact for a

period of time, a fact emerges • We represent these facts as separate nodes • In the following examples we show how we might

model facts and actions using intermediate nodes.

49

Example: EmploymentIan was employed as an engineer at Neo Technology

50

Example: PerformanceWilliam Hartnell played the Doctor in the story The Sensorites

51

Example: EmailingIan emailed Jim, and copied in Alistair

52

Example: Timeline Tree

53

A timeline tree showing the broadcast

dates for four episodes of a TV

program

Example: Linked-ListA doubly linked list representing a time-ordered series of events

54

Iterative and Incremental

• We develop the data model feature by feature, user story by user story

• This will ensure we identify the relationships our application will use to query the graph

• With the iterative and incremental delivery of application features we will be a corrected model that provides the right abstraction

55

Data Modeling: Enrich• The next steps diverges from the relational data

methodology• Instead of transforming a domain model’s graph-like

representation into tables, we enrich it.• That is, for each entity in our domain, “we ensure

that we’ve captured both the properties and the connections to neighboring entities necessary to support our application goals”.

56

Data Modeling: Enrich• Remember, the domain model is not totally aligned

to reality.• it is a purposeful abstraction of those aspects of our

domain relevant to our application goals. • By enriching our domain graph with additional

properties and relationships, we effectively produce a graph model aligned to our application’s data needs

57

Data Modeling: EnrichIn graph terms, we are ensuring that:•each node has the appropriate properties •every node is in the correct semantic context.

we do this by creating named and directed (and often attributed) relationships between the nodes to capture the structural aspects of the domain.

58

Data Modeling: Test• The next step is to test how suitable it is for

answering realistic queries• Also if Graph DB are great in supporting evolving

structures there are some design decisions to consider

• By reviewing the domain model and the resulting graph model at this early stage, we can avoid these pitfalls.

59

Data Modeling: Test• In practice there are two techniques that we can

apply here • The first, and simplest, is just to check that the graph

reads well. • We pick a start node, and then follow relationships to

other nodes, reading each node’s role and each relationship’s name as we go

• Doing so should create sensible sentences

60

Data Modeling: Test• The second one is to consider queries we’ll run on

the graph.• To validate that the graph supports the kinds of

queries we expect to run on it, we must describe those queries.

• Given a described query if we can easily write the query in Cypher or Gremlin we can be more certain that the graph meets the needs of our domain.

61

Conclusions

62

Avoid Anti-Patterns• In the general case, don’t encode entities into relationships. • It’s also important to realize that graphs are a naturally

additive structure • It’s quite natural to add facts in terms of domain entities and

how they interrelate adding nodes and relationships• If we model in accordance with the questions we want to ask

of our data, an accurate representation of the domain will emerge.

63

When to Use• Connected Data• Routing, Dispatch, and Location-based Services• Recommendation Engines

64

When Not to Use• When you need to update all or a subset of entities,

for example in analytics• In situation when you need to apply operations that

work on the global graph• When you don’t know the starting point of your

query

65