30
Rapid Development with Schema-less Data models Keith Chilek SVP, Engineering
Rapid Development with Schema-less Data models Keith Chilek SVP, Engineering
The online professional network for students and early professionals
3/8/13 | 2
With our data & tools, our members can better
Manage their education
Tell their story
Get a job
3/8/13 | 3
Growth
Technology Stack What is the MyEdu Profile?
MongoDb vs MySQL Decision Version 1 Using MongoDB
Learning From Our Mistakes
3/8/13 | 4
3/8/13 | 5
6,000,000
5,000,000
4,000,000
3,000,000
2,000,000
1,000,000
0
7,000,000
8,000,000
9,000,000
Jul 2012
Aug 2012
Sep 2012
Oct 2012
Nov 2012
Dec 2012
Jan 2013
We’re Building for Growth
MyEdu’s Technology Stack
Javascript
Scripting
Middleware
Database
3/8/13 | 6
3/8/13 | 7
“The only constant is change, continuing change, inevitable change, that is the
dominant factor in society today.
No sensible decision can be made any longer without taking into account not only
the world as it is, but the world as it will be.”
Isaac Asimov
3/8/13 | 8
MongoDB vs MySQL
Move fast, iterate often Minimize the downtime required
to launch new features Make sure the new product will
really, really scale
3/8/13 | 9
“We can do all of that in MySQL.”
“Schema-less design is impossible
to maintain.”
MongoDB vs MySQL
3/8/13 | 10
“We can do all of that in
MySQL.”
MongoDB vs MySQL
3/8/13 | 11
“We can do all of that in
MySQL.”
MongoDB vs MySQL
3/8/13 | 12
SELECT * FROM User, School, Major!WHERE User.school_id = School.school_id!AND User.major_id = Major.major_Id!AND User.user_Id = 1021!
“We can do all of that in
MySQL.”
MongoDB vs MySQL
3/8/13 | 13
“We can do all of that in
MySQL.”
MongoDB vs MySQL
3/8/13 | 14
SELECT * FROM User, User_School, User_Major, ! ! School, Major!
WHERE User_School.school_id = School.school_id!AND User_Major.major_id = Major.major_Id!AND User.user_id = User_Major.user_id!AND User.user_id = User_School.user_id!AND User.user_Id = 1021!
“We can do all of that in
MySQL.”
MongoDB vs MySQL
3/8/13 | 15
“We can do all of that in
MySQL.”
MongoDB vs MySQL
3/8/13 | 16
SELECT * FROM User, User_School, User_Major, User_Work, User_Project, School, Major, Work, Project!WHERE User_School.school_id = School.school_id!AND User_Major.major_id = Major.major_Id!AND User_Project.project_id = Project.project_Id!AND User_Work.work_id = Work.work_Id!AND User.user_id = User_Major.user_id!AND User.user_id = User_School.user_id!AND User.user_id = User_Work.user_id!AND User.user_id = User_Project.user_id!AND User.user_Id = 1021!
Yikes!
MongoDB vs MySQL
3/8/13 | 17
MongoDB vs MySQL
Looks like a Mongo document to me!
3/8/13 | 18
MongoDB vs MySQL
3/8/13 | 19
db.user_profile.find({“user_id” : 1021});!
“Schema-less design
is impossible to maintain.”
MongoDB vs MySQL
Lucky for us, the web services team had already been employing a Service / Mapper /
Model pattern.
With this pattern we are able to control the structure of each mongo document with a
defined data model.
3/8/13 | 20
MongoDB FTW!
MongoDB allows us to…
Move fast, iterate often Minimize the downtime required
to launch new features Make sure the new product will
really, really scale
3/8/13 | 21
MyEdu Profile (iteration 1: October)
3/8/13 | 22
MyEdu Profile (iteration 2: November)
3/8/13 | 23
MyEdu Profile (iteration 3: January)
3/8/13 | 24
Make sure the new product will really, really scale
With proper indexing alone, we have over 600k mongo profile documents on just a 3
member replica set.
We lowered our overall site speed time by 0.3 seconds and lowered our profile product
speed 300% when we introduced MongoDB.
3/8/13 | 25
It’s not all peaches and crème…
Oops! Bad Index
3/8/13 | 26
Lessons Learned
A schema-less data model has many benefits.
We can rapidly change our data to meet the needs of new product features and
requirements.
The document object is a familiar structure for a web developer.
3/8/13 | 27
What’s next?
Event tracking with MongoDB
Map Reduce
Using Json Patch for doing updates
3/8/13 | 28
