Intercontinental Database Development

Mike Furgal

September 18th, 2007

© 2006 Progress Software Corporation2

Progress Software

Founded in 1981 Application Infrastructure

• Database Technologies– Relational (4GL and SQL access)– Object Oriented

• Middleware– Messaging products– ODBC/JDBC Drivers

• Event Processing

© 2006 Progress Software Corporation3

Progress Software

Headquartered in Bedford MA• 1600 people world wide

Development offices• Bedford, MA• Nashua, NH• Montreal, Canada• Raleigh, North Carolina• Hyderabad, India• Noida, India• Cambridge, England• Houston, Texas• San Diego, California

Sales offices worldwide

© 2006 Progress Software Corporation4

Progress Software

Who am I?• Development Manager

• Manage development teams located in many of the offices worldwide

• Started with Progress Software in 1989

© 2006 Progress Software Corporation5

Agenda

Database Architecture• How Data Is Stored and Retrieved

Global Development• Why Go Global

• Challenges

• Rewards

© 2006 Progress Software Corporation6

Agenda

Database Architecture• How Data Is Stored and Retrieved

Global Development• Why Go Global

• Challenges

• Rewards

© 2006 Progress Software Corporation7

Database Architecture

What happens when you read a record from a database?

SELECT * FROM customer WHERE city = “Boston”;

© 2006 Progress Software Corporation8

Database Layout

Database divided into areas, extents, with fixed-size blocks or “pages”

Different kinds of pages store different kinds of data• Records• Index information• Sequence values• Unused space

Disk i/o in multiples of page size units Each page has a unique identifier

• Its Storage Area Id• Its “dbkey” or page number

© 2006 Progress Software Corporation9

Index B-Tree

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Top Reserved

Free Space

o o o

. . . Compressed Index Entries . . .

Bot Index No.

Num Entries

Bytes Used

. . . Compressed Index Entries . . .

Dummy Entry . . .

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Top Reserved

Free Space

o o o

. . . Compressed Index Entries . . .

Bot Index No.

Num Entries

Bytes Used

. . . Compressed Index Entries . . .

Dummy Entry . . .

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Top Reserved

Free Space

o o o

. . . Compressed Index Entries . . .

Bot Index No.

Num Entries

Bytes Used

. . . Compressed Index Entries . . .

Dummy Entry . . .

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Top Reserved

Free Space

o o o

. . . Compressed Index Entries . . .

Bot Index No.

Num Entries

Bytes Used

. . . Compressed Index Entries . . .

Dummy Entry . . .

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Top Reserved

Free Space

o o o

. . . Compressed Index Entries . . .

Bot Index No.

Num Entries

Bytes Used

. . . Compressed Index Entries . . .

Dummy Entry . . .

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Top Reserved

Free Space

o o o

. . . Compressed Index Entries . . .

Bot Index No.

Num Entries

Bytes Used

. . . Compressed Index Entries . . .

Dummy Entry . . .

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Top Reserved

Free Space

o o o

. . . Compressed Index Entries . . .

Bot Index No.

Num Entries

Bytes Used

. . . Compressed Index Entries . . .

Dummy Entry . . .

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Top Reserved

Free Space

o o o

. . . Compressed Index Entries . . .

Bot Index No.

Num Entries

Bytes Used

. . . Compressed Index Entries . . .

Dummy Entry . . .

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Top Reserved

Free Space

o o o

. . . Compressed Index Entries . . .

Bot Index No.

Num Entries

Bytes Used

. . . Compressed Index Entries . . .

Dummy Entry . . .

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Top Reserved

Free Space

o o o

. . . Compressed Index Entries . . .

Bot Index No.

Num Entries

Bytes Used

. . . Compressed Index Entries . . .

Dummy Entry . . .

root

internal

leaf

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Top Reserved

Free Space

o o o

. . . Compressed Index Entries . . .

Bot Index No.

Num Entries

Bytes Used

. . . Compressed Index Entries . . .

Dummy Entry . . .

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Top Reserved

Free Space

o o o

. . . Compressed Index Entries . . .

Bot Index No.

Num Entries

Bytes Used

. . . Compressed Index Entries . . .

Dummy Entry . . .

records

© 2006 Progress Software Corporation10

Index - an Ordered List Of Rowids

City Rowid BOLONIA 3331BOLTON 5554BOLTON 9001BOLTON 9022BONN 8001BOSTON 1111BOSTON 1118BOSTON 7001BOSTON 9002BOSTON 9003BOSTON 9006BOSTON 9007BOSTON 9008BOSTON 9009BOSTON 9999CARDIFF 3333

© 2006 Progress Software Corporation11

Index - an Ordered List Of Rowids

City Rowid BOLONIA 3331BOLTON 5554BOLTON 9001BOLTON 9022BONN 8001BOSTON 1111BOSTON 1118BOSTON 7001BOSTON 9002BOSTON 9003BOSTON 9006BOSTON 9007BOSTON 9008BOSTON 9009BOSTON 9999CARDIFF 3333

© 2006 Progress Software Corporation12

What Is A Rowid?

Unique 32-bit identifier for a record in a table• Unique within an area

Encodes the “physiological” storage address• Used to locate record fast

Constant for life of record• Until you delete it

DBKEY is a composite of• Block # in Green• Row inside the block in Yellow

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0

© 2006 Progress Software Corporation13

Record Block

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Version Number

Free SpaceFreeDirs. Rec 0 Offset Rec 1 Offset

Rec 2 Offset Rec n Offset

NumDirs.

Contiguous Free Space

Record 0

Record 2

Record 1

© 2006 Progress Software Corporation14

Database Buffer Pool

Hash Table

LRU Chain “oldest”“newest”

Write Queue

M MM

M M

© 2006 Progress Software Corporation15

Get The Row

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Free SpaceFreeDirs. Rec 0 Offset Rec 1 Offset

Rec 2 Offset Rec n Offset

NumDirs.

Free Space

Used Data Space

Record 0

Record 2

Record 1

row buffer

copy

Share lockedrecord blockin buffer pool

© 2006 Progress Software Corporation16

Row is available

SELECT * FROM customer WHERE city = “Boston”;

Questions??

© 2006 Progress Software Corporation17

Agenda

Database Architectures• How Data Is Stored and Retrieved

Global Development• Why go global

• Challenges

• Rewards

© 2006 Progress Software Corporation18

Why Go Global

Economic Reasons

Recommended Reading• ISBN 0-374-29288-4

Communication is cheap

© 2006 Progress Software Corporation19

Progress Software Global Development

Development offices• Hyderabad, India• Noida, India• Cambridge, England• Bedford, MA• Nashua, NH• Montreal, Canada• Raleigh, North Carolina• Houston, Texas• San Diego, California

© 2006 Progress Software Corporation20

Challenges

Time Zone Differences Physical Distance Collaboration Culture

© 2006 Progress Software Corporation21

Time Zones Challenges

Development offices• Hyderabad, India• Noida, India• Cambridge, England• Bedford, MA• Nashua, NH• Montreal, Canada• Raleigh, North Carolina• Houston, Texas• San Diego, California

UTC-05

UTC

UTC+0530

UTC-06

UTC-08

© 2006 Progress Software Corporation22

Time Zones Solutions

Email• Clear concise• Define Deliverables

– What is expected– When is it expected

Scheduled meetings• Early morning with India and England• Afternoons with Texas and California• Not possible with India, Nashua, and California

– 8 AM in Nashua 5 AM in California 5:30 PM in India

© 2006 Progress Software Corporation23

Physical Distance Challenges

Software products are large• CD Image sizes 600+ MB

Source Code Control in Bedford, MA• Network Connection Latency

Building and Debugging• Network Connection Latency

© 2006 Progress Software Corporation24

Physical Distance Solutions

VMWare http://www.vmware.com/• Allows multiple Window Operating System

Images to be running on a single machine• 8 CPU Linux Servers with 16 GB of memory

– Serves 30 Windows Virtual Machines• Remote Users “Remote Desktop” to their

personal Virtual Machine• Virtual Machine is co-located with

development infrastructure• Back to the days of centralized computing

© 2006 Progress Software Corporation25

Collaboration Challenges

No Hallway Conversations No Whiteboard Can’t walk into someone’s office to work

out issues

© 2006 Progress Software Corporation26

Collaborations Solutions

Documentation Documentation Documentation

Netmeeting• Available on every Windows PC• Allows sharing application or desktop

Instant Message is a key tool

© 2006 Progress Software Corporation27

Culture Differences

Work Day, Lunch, etc• Going out to lunch everyday in England

Holidays• Who celebrates the 4th of July or Thanksgiving?

Managers, team leaders, individual contributors• Who is the lead

Economy• Job Hopping

Need to understand the cultural aspects• Plan for the issue• Work with the culture, not against it

© 2006 Progress Software Corporation28

Rewards

Economic rewards due to currency differences• Can be drastic

Follow the sun development

© 2006 Progress Software Corporation29

A Model For Success

Keep local expertise where possible Cross train remote staff

• Helps with turn-over Document, Document, Document Have a lot of Coffee on hand as you will

be up early and up late for conference calls

© 2006 Progress Software Corporation30

Questions?

Thank You