Adding Flexibility and Scalability to Bouygues Telecom with MongoDB @ MongoDBWorld

Adding Flexibility and

Scalability to Bouygues

Telecom with MongoDB

Pierre-Alban DEWITTE

@__pad__

Bouygues Telecom

MongoDB World – 06/24/2014

Pierre-Alban DEWITTE @__pad__ #MongoDBWorld

Bouygues Telecom


Who is speaking ?

@__pad__


Agenda

Adding Flexibility and Scalability to

Bouygues Telecom with MongoDB

1. Existing problems

2. Design choices

3. Build

4. A few steps back


Agenda



Existing problems

Consumer Customers

Existing Sytem Compagny Customers

LDAP access

Micropayment system

Identity and Oauth management system

Network Connexion system

ETL


Existing problems

Two years delay

Schema customization

very hard to design and

implement

Performance issues


Existing problems


Agenda


2. Design choices A. MongoDB

B. Tomcat

C. Storm


Design choices - MongoDB

Schema less data integration

High availability

Sharding

Less expensive data storage

allowing duplication


POC between Tomcat and

NodeJS

We choose our battle,

MongoDB !

Design choices - Tomcat



Design choices - Storm

Distributed real-time computation system



MongoDB database

REST Micropayment

system

Identity and Oauth management system

Network Connexion system

STORM

Consumer Customers

Compagny Customers


Agenda


2. Design choices

3. Build

A. Storm topology

B. Schema design

and refactor

C. DevOps


Storm topology

Customer files are pushed every night

Read file line by line

1

… DetectFile

Spout ProcessLine

Bolt

2

line


Storm topology

Emitting tuple for each set of functional

collections

ProcessLine Bolt

4

Oauth UpdaterBolt

Identity UpdaterBolt Transform a line into a

document & check delta

3


Storm topology


Storm topology


Storm topology



Message failure handle

ack(Object msgId)

failed(Object msgId)



We scaled and easily !


Build - Schema design and refactor

Conception User

- Name - Compagny - Billcycle - Payment

Mode

Access Point

- Id - Type - Value

1

1..n

Conception



Develo

pm

ent



Qualification


Build - Schema design and refactor D

evelop

men

t



« That works »

Qualification

Conception


Build - Schema design and refactor St

ress

Tes

t



Conception Conception



Conception Conception



Develo

pm

ent



Develo

pm

ent

Qualification

Conception

Stre

ss T

est



PERFECT !

Develo

pm

ent

Conception

Stre

ss T

es


Build - DevOps


Build - DevOps

Tooling first


Build - DevOps

Moving the line between Dev and Ops


Build - DevOps

Internal training by development team

Redaction of a play-book dedicated to ops

Pre production tests played by both team

Early write of stress tests, played during

qualification tests


6 months later


Agenda


2. Design choices

3. Iterations during build

4. A few steps back


A few steps back

Apache Tomcat Apache STORM


A few steps back

1. Choose your battle

2. Use the right tool for the

right job

3. Be prepare to fail and to

rework

4. To move organization,

training is a key

Thank you for listening

Questions ?

Further talks @__pad__


Photo credits

100 m start : http://fr.wikipedia.org/wiki/100_m%C3%A8tres_%28athl%C3%A9tisme%29#mediaviewer/Fichier:Mens_100m_finals_British_Champs_and_Olympic_Trials.jpg by Paul Foot from Birmingham, UK Question mark block : https://www.flickr.com/photos/jarbo/9379813470 by Jared Cherup Pelleteuse orange : http://fr.123rf.com/photo_3669950_un-grand-pelleteuse-orange-stationne-a-un-chantier-de-construction.html by Stephen Mcsweeny Punaise : https://www.flickr.com/photos/24362608@N05/3501112978/sizes/l/in/photolist-f5R8g4-6kiVfR-6koahu-6ko7US-6ko8Qh-d9NUKU-nDi2eb-ffg13f-8h56wx-cgAcib-hchCtD-decZ4p-6kocrL-6kj2J6-5doxaw-gbTPmK-nCFBBq-672Snf-2sCzvw-e1zAjM-8voRbc-c9Rh1m-amFFGt-2iZNBq-cFSnFU-81Ckk4-2XHZAy-9Ggpfw-m8FfKz-8hW6r5-4xSuxC-awXsp4-5aVn7g-fbXVHZ-9GAPLX-9qr7uU-fcddgA-cP1EN7-np2q92-6V8oBT-cenSsb-cyzaEd-8h5a2F-6V2b3F-ch1RME-fJ52s3-9XthVN-kBs5mW-6qSPaF-5cA9Qb/ par dractrain94 Une longue vue qui louche : https://www.flickr.com/photos/la_bretagne_a_paris/3847733265/ par Yann Caradec NASA-Apollo8-Dec24-Earthrise : http://www.hq.nasa.gov/office/pao/History/alsj/a410/AS8-14-2383HR.jpg par NASA / Bill Anders

http://fr.wikipedia.org/wiki/100_m%C3%A8tres_(athl%C3%A9tisme)




http://www.flickr.com/people/13809318@N00

https://www.flickr.com/photos/jarbo/9379813470

https://www.flickr.com/photos/jarbo/9379813470

http://fr.123rf.com/photo_3669950_un-grand-pelleteuse-orange-stationne-a-un-chantier-de-construction.html




















https://www.flickr.com/photos/24362608@N05/3501112978/sizes/l/in/photolist-f5R8g4-6kiVfR-6koahu-6ko7US-6ko8Qh-d9NUKU-nDi2eb-ffg13f-8h56wx-cgAcib-hchCtD-decZ4p-6kocrL-6kj2J6-5doxaw-gbTPmK-nCFBBq-672Snf-2sCzvw-e1zAjM-8voRbc-c9Rh1m-amFFGt-2iZNBq-cFSnFU-81Ckk4-2XHZAy-9Ggpfw-m8FfKz-8hW6r5-4xSuxC-awXsp4-5aVn7g-fbXVHZ-9GAPLX-9qr7uU-fcddgA-cP1EN7-np2q92-6V8oBT-cenSsb-cyzaEd-8h5a2F-6V2b3F-ch1RME-fJ52s3-9XthVN-kBs5mW-6qSPaF-5cA9Qb/





































































































https://www.flickr.com/photos/la_bretagne_a_paris/3847733265/



http://www.hq.nasa.gov/office/pao/History/alsj/a410/AS8-14-2383HR.jpg







Date post:	15-Jan-2015
Category:	Presentations & Public Speaking
Upload:	pierre-alban-dewitte
View:	440 times
Download:	2 times

Adding Flexibility and Scalability to Bouygues Telecom with MongoDB @ MongoDBWorld

Presentations & Public Speaking