Trading up: Adding Flexibility and Scalability to Bouygues Telecom with MongoDB

Adding Flexibility and Scalability to

Bouygues Telecom with MongoDB

Pierre-Alban DEWITTE@__pad__Bouygues TelecomMongoDB World – 06/24/2014

Pierre-Alban DEWITTE @__pad__ #MongoDBWorld

Bouygues Telecom

13 M Customers


Who is speaking ?

@__pad__

Thanks !


Agenda

Adding Flexibility and Scalability to Bouygues Telecom with MongoDB

1. Existing problems2. Design choices3. Build4. A few steps back


Agenda

1. Existing problems


Existing problems

ConsumerCustomers

Existing Sytem

CompagnyCustomers

LDAPaccess

Micropayment system

Identity and Oauth management system

Network Connexion system

ETL


Existing problems

Two years delay

Schema customization very hard to design and implementPerformance issues


Existing problems


Agenda

1. Existing problems

2. Design choicesA. MongoDBB. TomcatC. Storm


Design choices - MongoDB

Schema less data integrationHigh availabilityShardingLess expensive data storage allowing duplication


Design choices - MongoDB

Team with development and architecture skillsSchema design experienceCapability to train new actors


POC between Tomcat and NodeJS

We choose our battle, MongoDB !

Design choices - Tomcat



Design choices - Storm

Distributed real-time computation system



ConsumerCustomer

MongoDB database

CompagnyCustomers

REST Micropayment system

Identity and Oauth management system

Network Connexion system

API

STORM


Agenda

1. Existing problems2. Design choices

3. BuildA. Storm topologyB. Schema design and refactorC. DevOps


Storm topology

Customer files are pushed every night

Read file line by line

1

…DetectFile

SpoutProcessLine

Bolt

2

line


Storm topology


Storm topology

Emitting tuple for each set of functional

collections

ProcessLineBolt

4

OauthUpdaterBolt

IdentityUpdaterBoltTransform a line into a

document & check delta

3


Storm topology


Storm topology



Message failure handle

ack(Object msgId) failed(Object msgId)

Take care of system stability



We scaled and easily !



Build - Schema design and refactor

Conception User- Name- Compagny- Billcycle- Payment

Mode

Acces Point- Name- Compagny- Billcycle- Payment

Mode

1

1..n

Conception



Developm

ent



Qualification



Developm

ent



« That works »

Qualification

Conception



Stre

ss T

est



ConceptionConception



ConceptionConception



Developm

ent



Developm

ent

Qualification

Conception

Stre

ss T

est



PERFECT !

Developm

entConception

Stre

ss T

es


Build - DevOps


Build - DevOps

Tooling first


Build - DevOps

Moving the line between Dev and Ops


Build - DevOps

Internal training by development teamRedaction of a play-book dedicated to opsPre production tests played by both teamEarly write of stress tests, played during qualification tests


6 months later


Agenda

1. Existing problems2. Design choices3. Iterations during build

4. A few steps back


A few steps back

Apache Tomcat Apache STORM

Flexibility and Scale with a 100% Open Source Stack !


A few steps back

1. Choose your battle2. Use the right tool for

the right job3. Be prepare to fail and

to rework4. To move organization,

training is a key

Thank you for listening

Questions ?

Further talks @__pad__


Photo credits100 m start : http://fr.wikipedia.org/wiki/100_m%C3%A8tres_%28athl%C3%A9tisme%29#mediaviewer/Fichier:Mens_100m_finals_British_Champs_and_Olympic_Trials.jpg by Paul Foot from Birmingham, UKQuestion mark block : https://www.flickr.com/photos/jarbo/9379813470 by Jared Cherup Pelleteuse orange : http://fr.123rf.com/photo_3669950_un-grand-pelleteuse-orange-stationne-a-un-chantier-de-construction.html by Stephen McsweenyPunaise : https://www.flickr.com/photos/24362608@N05/3501112978/sizes/l/in/photolist-f5R8g4-6kiVfR-6koahu-6ko7US-6ko8Qh-d9NUKU-nDi2eb-ffg13f-8h56wx-cgAcib-hchCtD-decZ4p-6kocrL-6kj2J6-5doxaw-gbTPmK-nCFBBq-672Snf-2sCzvw-e1zAjM-8voRbc-c9Rh1m-amFFGt-2iZNBq-cFSnFU-81Ckk4-2XHZAy-9Ggpfw-m8FfKz-8hW6r5-4xSuxC-awXsp4-5aVn7g-fbXVHZ-9GAPLX-9qr7uU-fcddgA-cP1EN7-np2q92-6V8oBT-cenSsb-cyzaEd-8h5a2F-6V2b3F-ch1RME-fJ52s3-9XthVN-kBs5mW-6qSPaF-5cA9Qb/ par dractrain94 Une longue vue qui louche : https://www.flickr.com/photos/la_bretagne_a_paris/3847733265/ par Yann Caradec NASA-Apollo8-Dec24-Earthrise : http://www.hq.nasa.gov/office/pao/History/alsj/a410/AS8-14-2383HR.jpg par NASA / Bill Anders

http://fr.wikipedia.org/wiki/100_m%C3%A8tres_(athl%C3%A9tisme)




http://www.flickr.com/people/13809318@N00

https://www.flickr.com/photos/jarbo/9379813470

https://www.flickr.com/photos/jarbo/9379813470

http://fr.123rf.com/photo_3669950_un-grand-pelleteuse-orange-stationne-a-un-chantier-de-construction.html



https://www.flickr.com/photos/24362608@N05/3501112978/sizes/l/in/photolist-f5R8g4-6kiVfR-6koahu-6ko7US-6ko8Qh-d9NUKU-nDi2eb-ffg13f-8h56wx-cgAcib-hchCtD-decZ4p-6kocrL-6kj2J6-5doxaw-gbTPmK-nCFBBq-672Snf-2sCzvw-e1zAjM-8voRbc-c9Rh1m-amFFGt-2iZNBq-cFSnFU-81Ckk4-2XHZAy-9Ggpfw-m8FfKz-8hW6r5-4xSuxC-awXsp4-5aVn7g-fbXVHZ-9GAPLX-9qr7uU-fcddgA-cP1EN7-np2q92-6V8oBT-cenSsb-cyzaEd-8h5a2F-6V2b3F-ch1RME-fJ52s3-9XthVN-kBs5mW-6qSPaF-5cA9Qb/






https://www.flickr.com/photos/la_bretagne_a_paris/3847733265/



http://www.hq.nasa.gov/office/pao/History/alsj/a410/AS8-14-2383HR.jpg



Date post:	22-Dec-2014
Category:	Technology
Upload:	mongodb
View:	312 times
Download:	0 times

Trading up: Adding Flexibility and Scalability to Bouygues Telecom with MongoDB

Technology