Download - Indexes, what Indexes?

Alvin Richards -‐ [email protected]

Indexing

1Wednesday, May 25, 2011

mailto:[email protected]


What’s Easy About MongoDB Indexing?

It’s almost the same as in your RDBMS


What’s Hard About MongoDB Indexing?

It’s almost the same as in your RDBMS


Indexes Maintain Order

Index on a: ascending

{a: 0, b: 9}

{a: 2, b: 0}

{a: 7, b: 1}

{a: 3, b: 2}

{a: 3, b: 5}

{a: 3, b: 7}

{a: 9, b: 1}


Indexes Maintain Order

Index on a: ascending, b: descending

{a: 0, b: 9}

{a: 2, b: 0}

{a: 7, b: 1}

{a: 3, b: 7}

{a: 3, b: 2}

{a: 3, b: 5}

{a: 9, b: 1}


Query for {a: 7}

{...} {...} {...} {...} {...} {...} {...} {...} {...} {...} {...}

[-‐∞, 5)[5, 10)

[10, ∞)

[5, 7) [7, 9) [9, 10)[10, ∞) buckets[-‐∞, 5) buckets

With Index

Without index -‐ Scan

Index on a: ascending


db.blogs.ensureIndex({author: 1})

1 = ascending-‐1 = descending

An index on _id is automatic.

For more use ensureIndex:

Creating Indexes


Compound Indexes

db.blogs.save({ author: "James", ts: new Date()});

db.blogs.ensureIndex({author: 1, ts: -‐1})


db.blogs.save({ ... title: "My first blog"});

db.blogs.ensureIndex({title: 1}, {unique: true})

Unique Indexes


db.blogs.save({ ... stats : { views: 0, followers: 0 }});

db.blogs.ensureIndex({"stats.followers": -‐1})

db.blogs.find({"stats.followers": {$gt: 500}})

Indexing Embedded Documents


db.blogs.save({ ... comments: [ {author: "James", ts : new Date()} ]});

db.blogs.ensureIndex({"comments.author": 1})

db.blogs.find({"comments.author": "James"})

Indexing Embedded Arrays


db.blogs.save({ ... tags: ["mongodb", "NoSQL"]});

db.blogs.ensureIndex({tags: 1})

db.blogs.find({tags: "NoSQL"})

Multikeys


• New in 1.8• Query resolved in index only• Need to exclude _id from items projected

db.blogs.save({ ... author: "James",});

db.blogs.ensureIndex({author: 1})

db.blogs.find({author: "James"}, {author: 1, _id:0}))

Covered Indexes


• New in 1.8• Key value included if and only if the value is present• Reduces size of index• Limited to a single field

db.blogs.ensureIndex({loc: 1}, {sparse: true})

// loc key stored for Ben & Sarah onlydb.blogs.save({author: "Jim"})db.blogs.save({author: "Ben", loc: null})db.blogs.save({author: "Sarah", loc: "CA"})

Sparse Indexes


• Null and not-present are differentdb.blogs.ensureIndex({url:1}, {sparse:true,unique:true});

db.blogs.save({url:"www.10gen.com"})

// Can only have a single null valuedb.blogs.save({url:null}) db.blogs.save({url:null})E11000 duplicate key error index: test.blogs.$url_1 dup key: { : null }

// Can have multiple missing valuesdb.blogs.save({author:"James"})db.blogs.save({author:"Bob"})

Unique Sparse Indexes


http://www.10gen.com

http://www.10gen.com

db.blogs.save({ loc: { long: 40.739037, lat: 40.739037 }});

db.blogs.save({ loc: [40.739037, 40.739037]});

db.blogs.ensureIndex({"loc": "2d"})

Geospatial• Geo hash stored in B-Tree• First two values indexed


db.blogs.getIndexes()

db.blogs.totalIndexSize()

db.blogs.dropIndex({"author": 1})

Other Index Operations


When is an Index Used?Index on {a: 1}

db.c.find({a: 0})db.c.find({a: {$in: [0, 2]}})db.c.find({a: {$gt: 5}})db.c.find().sort({a: -‐1})db.c.find({a: 0}, {a:1, _id:0})db.c.count({a: 0})db.c.find({loc: {$near: [50,50] }})

db.c.find({b: 0}).sort({a: -‐1})

Partially:


When isn’t an Index Used?

Index on {a: 1, b: -‐1}

db.c.find({b: 0})db.c.find({a: {$ne: 1} })db.c.find({a: {$nin: [1, 3, 5, 7]} })


db.c.ensureIndex({x:1})db.c.ensureIndex({y:-‐1})

db.c.find({x: 10, y: “foo”})

scan

index on x

index on y remember

terminated

Picking an index


•Frequently used queries•Low response time•Avoid full collection scans

When are Indexes Needed?


Indexes Slow Down Writes

•Indexes take up space•Index maintenance will cause writes


Does my query use an Index?

db.blogs.find({title:"My blog"}).explain();


db.blogs.find({title:"My blog"}).explain();

{ "cursor" : "BasicCursor", "indexBounds" : {} "nscanned" : 57594, "nscannedObjects" : 57594, "n" : 3, "millis" : 108}

Explain - Scan all documents


{ "cursor" : "BtreeCursor x_1", "indexBounds" : {...}, "nscanned" : 123, "nscannedObjects" : 123, "n" : 10, "millis" : 4}

db.blogs.ensureIndex({title: 1});db.blogs.find({title: "My blog"}).explain();

Explain - Index used


{ "cursor" : "BtreeCursor x_1", "indexBounds" : {...}, "nscanned" : 123, "nscannedObjects" : 123, "n" : 10, "millis" : 4, "indexOnly" : true}

db.blogs.ensureIndex({title: 1});db.blogs.find({title: "My blog"}, {title: 1, _id: 0}).explain();

Explain - Covered Index used


96

{_id:4,x:6}

1 2 3 4 5 6 6

{_id:5,x:6}

{_id:1,x:6}

db.c.find({x: 6})

Index Internals - Equality match


db.c.find({x:6}).explain(){ "cursor" : "BtreeCursor x_1", "nscanned" : 3, "nscannedObjects" : 3, "n" : 3, "millis" : 1, "nYields" : 0, "nChunkSkips" : 0, "isMultiKey" : false, "indexOnly" : false, "indexBounds" : { "x" : [ [ 6, 6 ] ] }}

Equality match


"indexBounds" : {

"x" : [

[

6,

6

]

]

}

Equality match

Exact match will have an inclusive

range


"nscanned" : 3,

"nscannedObjects" : 3,

"n" : 3,

Equality match

Number of matching documents is 3


1

2

3 4

5

6

6

6 9

Equality matchdb.c.find({x: 6})


91 2 3 4 5 6 6 6

{y:4,x:6} {y:5,x:6}

{y:1,x:6}

Full document matchdb.c.ensureIndex({x: 1})

db.c.find({x: 6, y: 1})


db.c.find({x: 6, y: 1}).explain(){ "cursor" : "BtreeCursor x_1", "nscanned" : 3, "nscannedObjects" : 3, "n" : 1, "millis" : 1, "nYields" : 0, "nChunkSkips" : 0, "isMultiKey" : false, "indexOnly" : false, "indexBounds" : { "x" : [ [ 6, 6 ] ] }}

Full document match


"indexBounds" : { "x" : [ [ 6, 6 ] ] }

Full document match

Exact match will have an inclusive

range


"nscanned" : 3,


"n" : 1,

Full document match

Documents for all matching keys

scanned, but only one document

matched on non-index keys


81 2 3 4 5 6 7 9

Range matchdb.c.ensureIndex({x: 1})

db.c.find({x:{$gte:4, $lte:7}})


db.c.find( {x:{$gte:4,$lte:7}} ).explain(){ "cursor" : "BtreeCursor x_1", "nscanned" : 4, "nscannedObjects" : 4, "n" : 4, "millis" : 1, "nYields" : 0, "nChunkSkips" : 0, "isMultiKey" : false, "indexOnly" : false, "indexBounds" : { "x" : [ [ 4, 7 ] ] }}

Range match


"indexBounds" : {

"x" : [

[

4,

7

]

]

Range match

Range match will have an inclusive

range


"nscanned" : 4,


"n" : 4,

Range match

Number of matching documents is 4


1

2

3 4

5

6

7

8 9

Range match


insert query update delete getmore command flushes mapped vsize res locked % idx miss % 5345 0 0 0 0 1 0 8.75g 11.1g 132m 1 2 230 0 0 0 0 1 0 8.75g 11.1g 132m 1 7 1740 20 0 0 0 2 0 8.75g 11.1g 132m 3 19 0 120 0 0 0 1 0 8.75g 11.1g 132m 0 17 0 117 0 0 0 1 0 8.75g 11.1g 132m 0 16 0 119 0 0 0 1 0 8.75g 11.1g 132m 0 17

Mongostat


@mongodb

conferences, appearances, and meetupshttp://www.10gen.com/events

http://bit.ly/mongoF Facebook | Twitter | LinkedIn

http://linkd.in/joinmongo

download at mongodb.org

We’re Hiring [email protected]


