2011-07-26 OSCON 2011 1

How I learned to use SQLand then learned not to use it

Henrik Ingo

Froscon2011-08-20

2011-07-26 OSCON 2011 2

Henrik Ingo

open source technology and strategy specialist

active in MySQL, Drupal communities

current: Senior Performance Architect at Nokia Ovi

worked in mobile and LAMP with business management, sales, R&D

author of "Open Life: The Philosophy of Open Source"

www.openlife.cchenrik.ingo@openlife.cc

2011-07-26 OSCON 2011 3

1994

2011-07-26 OSCON 2011 4

1994

USSR fell 4 years ago

Finland yet to win first gold medal in hockey

Windows 3.0 replacing MS-DOS

Windows 95 did not yet exist

I learn how to write .bat scripts from my dad

Word 6.0 replacing WordPerfect 5.1

Excel 5.0 replacing Lotus

I had not yet used Internet

2011-07-26 OSCON 2011 5

1994

C++ & Visual Basic

LANPrinter Database

Client - Server architecture

SQLStandard Interpreted

Flexible and expressive command line env

"SQL for secretaries"Good for English speakers

2000: Bad for IDE w IntelliSense

Stored procedures rule

DBA is king of business logic

2011-07-26 OSCON 2011 6

1997

2011-07-26 OSCON 2011 7

1997

2011-07-26 OSCON 2011 8

Learning SQL

NoSQL advocates say:

SQL is hard to learn

To really scale, you must de-normalize

Most people don't get normalization right

Impedance mismatch between oo and sql

I just need a simple key-value store

Henrik says:

MS Access easy to learn

Darn, I always got normalization

I know, teaching n:n relations was always fun

INSERT INTO t ... serialize($obj)

SELECT value FROM tWHERE key=?;

2011-07-26 OSCON 2011 9

1997

WorldWideWebFree Software

1998: Open Source

LAMP stackWhere P = Perl, runs +90% of web "apps"

Oracle doesn't support Linux

PostgreSQL doesn't support Windows

MiniSQL has poor performance

Wish I had a simple data store for my web apps that was really fast and scalable and Free Software...

2011-07-26 OSCON 2011 10

1997

2011-07-26 OSCON 2011 11

1997

ERROR: Could not connect to MySQL: ERROR 1040: Too many connections

"Slashdotted"Web jargon ca 1997."Your site is down becauseit isn't web scale."

2011-07-26 OSCON 2011 12

History of MySQL

1985... ISAM key-value store - Trivia: Access is also ISAM db

SQL support added (HTML)

Data Warehousing: bulk loads, reporting. - Flexible attitude to errors during insert

ODBC (Excel) - BI tool #1

1995.. Released under FOSS-like license

2011-07-26 OSCON 2011 13

1997

"Read-mostly" - No MVCC - Not transactional

Not crash safe

Scales well for 2-3 CPUs - This will work well for the next 10 years

Not ANSI SQL

2011-07-26 OSCON 2011 14

1997

This site is currently offline for nightly backup. Please come back after 15 minutes.

(No online backups)

2011-07-26 OSCON 2011 15

1997-2007

InnoDB and BDB - Transactions!

SAP partnership - 4 CPU servers - Support SQL-92 "enterprise" features - Oracle acquires InnoDB and BDB

Replication - Statement based - Asynchronous

Online backup tool

2011-07-26 OSCON 2011 16

2007

We want a database that - does not conform to SQL standard - has no transactions - is not crash safe - impossible to do consistent backups

- but scales to more than 4 cores!

Users are not happy!

2011-07-26 OSCON 2011 17

You see where I'm going, don't you...

2011-07-26 OSCON 2011 18

Sorry, that was a cheap shot...

...I just couldn't resist :-)

2011-07-26 OSCON 2011 19

It really started with...

2011-07-26 OSCON 2011 20

Actual questions from Memcache users, ca 2007:

How can I dump all records from a memcached instance?

How can I make memcached highly available?

Backups?

2011-07-26 OSCON 2011 21

Answers from Memcache developers

It's a cache. Please use ittogether with a real database!

Why would you ever want to do that?

No no no, you're doing it wrong!

2011-07-26 OSCON 2011 22

So clearly there was a market demand...

2011-07-26 OSCON 2011 23

Things NoSQL guys do really well

No SQL (parsing)

Schemaless = Win! for agile development

HA with quorum consistency: R + W > N

Transparent sharding

Graph databases

N:N relationships, what's the big deal?

Actually makes sense to give up on SQL!

Map Reduce

Bypass ETL, get clean data

Implemented in Java or Python (or Erlang)

2011-07-26 OSCON 2011 24

Best of both worlds

NoSQLNo SQL.

Simple key-value store.

...and secondary indexes

Quorum consistency.

Graph databases

Map Reduce against text files

Java and Python.

MySQLHandlerSocketMemcache API, NDB API

BLOBSELECT v FROM ... WHERE k=?

Functional indexesVirtual columns, etc...

Synchronous replicationGalera, NDB

Damn N:N relations!

Map Reduce against RDBMS

C++ can be done rightDrizzle

2011-07-26 OSCON 2011 25

Clayton Christensen on Disruptive innovation

2011-07-26 OSCON 2011 26

Clayton Christensen on Disruptive innovation

First they ignore you

Then they laugh at you

Then they fight you

Then you win

- Gandhi

2011-07-26 OSCON 2011 27

It used to be

2011-07-26 OSCON 2011 28

The future is

2011-07-26 OSCON 2011 29

The future is

All Open SourceAll Open Source