Introduction to Gremlin

Chicago Graph Database Meet-UpMax De Marzi

About Me

• My Blog: http://maxdemarzi.com• Find me on Twitter: @maxdemarzi• Email me: maxdemarzi@gmail.com• GitHub: http://github.com/maxdemarzi

Built the Neography Gem (Ruby Wrapper to the Neo4j REST API)Playing with Neo4j since 10/2009

Agenda

• What is Gremlin?• Gremlin in Neo4j• Gremlin Steps• Gremlin Recommends

What is Gremlin?

Not a Car

Not a little Monster

Gremlin is

• A Graph Traversal Language• A domain specific language for traversing

property graphs• Implemented by most Graph Database

Vendors• Primarily seen with the Groovy Language• With JVM connectivity in Java, Scala, and

other languages

Marko Rodriguezhttp://markorodriguez.com

Created by:

A Graph DSL

A Dynamic Language for the JVM

A Data Flow Framework

“JDBC” for Graph DBs

Gremlin in Neo4j

g = (neo4jgraph[EmbeddedGraphDatabase [/neo4j/data/graph.db]]

Gremlin in Neo4j

g.v(1)

1

g.v(1).first_name

1

first_name=Max

g.v(1).last_name

1

last_name=De Marzi

g.v(1).map()

1

first_name=Maxlast_name=De Marzi

g.v(1).outE

1

create

d

knows

knows

g.v(1).outE.since

1

create

d

knows

knows

null

since=2009

since=2010

g.v(1).outE.inV

1

create

d

knows

knows

2

3

4

g.v(1).outE.inV.name

1

create

d

knows

knows

2

3

4

name=neography

name=Neo4j

name=Gremlin

g.v(1).outE.filter{it.label==‘knows’}

1

create

d

knows

knows

g.v(1).outE.filter{it.label==‘knows’}.count()

2

g.v(1). outE.filter{it.label==‘knows’}.inV.name

1

create

d

knows

knows

3

4

name=Neo4j

name=Gremlin

g.v(1). out(‘knows’).name

1

create

d

knows

knows

3

4

name=Neo4j

name=Gremlin

g.v(1). out(‘created’)

1

create

d

2

g.v(1). out(‘created’).in(‘contributed’)

1

create

d

5

2

contributed

g.v(1). out(‘created’).in(‘contributed’).name

1

create

d

5 name=Peter

2

contributed

g.v(1). out(‘created’).in(‘contributed’).name.back(1)

1

create

d

5 name=Peter

2

contributed

g.v(1). out(‘created’).in(‘contributed’).name.back(1).sideEffect{g.addEdge(g.v(1), it, ‘collaborator’)}

1

create

d

collaborator5 name=Peter

2

contributed

Gremlin Steps

Gremlin Transform Steps

_ V E idlabeloutoutEoutV

ininEinVbothbothEbothVkeymap

memoizegatherscatterpathcapselecttransform

Gremlin Filter Steps

[i] [i..j]hashasNotbackandorrandom

dedupsimplePathexceptretainfilter

Gremlin Side-Effect Steps

groupBy groupCountaggregatetabletreeasoptionalstore

sideEffect

Gremlin Branch Steps

loopifThenElsecopySplitfairMergeexhaustMerge

Gremlin Recommends

Our Graph (from MovieLens)

Recommendation Algorithm

m = [:];x = [] as Set; v = g.v(node_id);

v. out('hasGenre').aggregate(x).back(2).inE('rated').filter{it. stars > 3}.

(continued)outV. outE('rated'). filter{it.stars > 3}.inV.filter{it != v}.filter{it.out('hasGenre').toSet().equals(x)}.groupCount(m){\"${it.id}:${it.title}\"}.iterate();m.sort{a,b -> b.value <=> a.value}[0..24]

Explanation

m = [:];x = [] as Set; v = g.v(node_id);

In Groovy [:] is a map, we will return thisThe set “x” will hold the collection of genres we want our recommendedmovies to have.

v is our starting point.

Explanation

v. out('hasGenre'). (we are now at a genre node)aggregate(x).

We fill the empty set “x” with the genres of our movie.These are the properties we want to make sure our recommendations have.

Explanation

back(2). (we are back to our starting point)inE('rated').filter{it. stars > 3}. (we are now at the link between our movie and users)

We go back two steps to our starting movie, go to the relationship ‘rated’ and filter it so we only keep those with more than 3 stars.

Explanation

outV. (we are now at a user node)outE('rated'). filter{it.stars > 3}. (we are now at the link between user and movie)

We follow our relationships to the users who made them, and then go to the “rated” relationships of movies which also received morethan 3 stars.

Explanation

inV. (we are now at a movie node) filter{it != v}.

We follow our relationships to the movies who received the, but filter out “v” which is our starting movie. We do not want the system to recommend the same movie we just watched.

Explanation

filter{it.out('hasGenre').toSet().equals(x)}.

We also want to keep only the movies that have the same genres as ourstarting movie. People gave Toy Story and Terminator both 4 stars,but you wouldn’t want to recommend one to the other.

Explanation

groupCount(m){\"${it.id}:${it.title}\"}.iterate();

groupCount does what it sounds like and stores the values in the map “m”we created earlier, but we to retain the id and title of the movies.

iterate() is needed from the Neo4j REST API, the gremlin shell does it automatically for you. You will forget this one day and kill30 minutes of your life trying to figure out why you get nothing.

Explanation

m.sort{a,b -> b.value <=> a.value}[0..24]

Finally, we sort our map by value in descending order and grab the top25 items… and you’re done.See http://maxdemarzi.com/2012/01/16/neo4j-on-heroku-part-two/for the full walk-through including data loading.

How to treat Gremlin in Neo4j

As the equivalent of Stored Procedures in SQL.

Allow only parameters from end-users, do not generate gremlin dynamically or you’ll have the mother of all SQL injection vulnerabilities…

Gremlin => Groovy => JVM => Full Power

Questions?

?

Thank you!http://maxdemarzi.com