Post on 15-Jul-2015
transcript
Backend MutationWith Symfony
Joris CalabreseHead Of
Industrialization
Etienne BroutinArchitect
Meetic corp
• Started in 2001
• Leader in Europe, 17 countries
• Millions of Monthly Active Users
• 150 people in IT team
Past•
Present•
Future
Start up + Rapid Growth=
Spaghetti
Monolithic code
Webservices
Backoffices
Mobile Web
WAP
Desktop
Cronjobs
…
700k
NLOC
Billing Systems
DataBase centric architecture
MailingBI
Procedures + Triggers
Some Metrics
4 000TABLES
900configurations
400Servers
Need to change something ?
A. Ask 400 peopleB. Read all the code, understand everything then notify everyone
C. Break everythingD. Add a dirty “if” for your specific caseD. Add a dirty “if” for your specific case
Spaghetti + Quality Approach=
J’ai de la chance
What Was Our Development Env?
• Locked Files System• 1 Database for all Developers• No Automated Tests
QA
• 1 release every 2 weeks• 50 projects by release• 1 button !
What Was Our Quality Approach?
Who wants to play?
Illustration: mailchimp • mailchimp.com
When it works…
…and when it fails!
Some Stats
10TEAMS
2Deploy/WEEK
2012 2015
Plateform Growth
1
5
65BUGS/WEEK
1ROLLBACK/W
EEK
Spec v1
Bug Cost
Source: Applied Software Measurement, Capers Jones, 1996
And, If we want to add automated tests?
A. Chuck Norris develops without bug
B. Only test get_next_week()
C. Mock 200 global variables and 35 functions
D. I will do it when I have timeD. I will do it when I have time
New Talent + Innovation Need
=New Architecture
Target
All Clients are born equal in dignity and rights1
SOA architecture
Prevents code duplicationMutualize features Web Services
But this single, centralized webservice will still be huge and complex ?
Too long…
All Web Services have a single responsibility
2
Message Photos Profile Search
Strict REST API
profiles messages
GET /profiles/xxxxPUT /profiles/xxxx
GET /messages/xxxxPOST /messages
profiles messages
Do not share your data
Inside Business services
Symfony serialization for JSON input / output
Business Logic
Cache Layer
Doctrine DBAL
Controller
Service
Proxy Cache
DAO
Online Photos Profile Rights Search Boost
That means clients should make several calls to display a single page?
Too long…
Online Photos Profile Rights Search Boost
But from client point of view
Online Photos Profile Rights Search Boost
But from client point of view
API Exposure Layer
Separation of concerns
business logicdatabase access
orchestrationsecurity, sessionsconfiguration
UX
API Exposure Layer
Exposure layer
API Exposure Layer
Web Business Services
Back Office
Mailing
Payments
Inside Exposure Layer
Use Guzzle with batch calls
FOS\RESTBundle
Orchestrate API callsAssemble results from APIs
Controller
Manager
API Client
Android application
Example to display a list of members
GET /members
Get a list of user ids using Location search
Fetch user data
Exposure Layer
Configuration:
group B
uses location search
AB Tests Search Profile
user is from group B
1 23
Migration Strategy
Webservices
Web Business Services
New Quality Approach
Testing Strategy: Unit test
Threshold Spirit
Controller
Service
DAO
Complexity Code Coverage
Complexity Code Coverage
Complexity Code Coverage
Testing Strategy: Integration test
Continuous Integration
60JOBS
5Agents
5Platform
10 builds in
prOgress
Product Centric
• Kept a good Knowledge
• Share Product Vision
• Share a same vocable
Behavior Driven Development
Testing Strategy: Behavior Test
Android application
GET /members
Exposure Layer
AB Tests Search Profile
1 23
Given I am a user of the B groupAnd my geoloc position is ParisWhen I launch a search on the onlineThen I see a list of profileAnd the first profiles are near of my position
Docker Focus
• Allow the dev to automated testing all technical stack
• 50 instances to launch in parallel
• Different scope and context for the automated tests
Docker Focus: How to?
DockerCompose
Exposure Layer
BusinessService
LegacyService
Oracle
MySQL
1
2
3
4
5
23
45
QA Position
Design Build Test Prod
QA Position
Design Build Test Prod
Team Autopsy
Front Developer• AngularJS, Grunt, …• Code Review• TDD, BDD
Back Developer• Design• Symfony2, Composer, …• Code Review• TDD,BDD
Product Owner• Define feature• Scenario writing• Define Business Indicators
Developer Community
• Continuous Learning
• Ease the sharing
• Improve the Skill Switching
• Community Of Practice
• BBL
Source Management
180Repositories
12groups
14Technos
50+Users
Jenkins/Gitlab Focus
master
Working-branch
Unit tests
Integration tests
Static Analyse
Hey all!Who wants to review my MR?
I would love to. You can assign it to me.
Dev Infrastructure
API
API
Some Stats
2012 2015
1x
7x
Bug In By Week
2012 2015
2
10
Deployment by Day
0,5%Of rollback
3 756Written tests
91,3
%Code
coverage
Our New Future
• Impact management
• Reusability
From Architecture point of view
Continuous Deployment
<< Dev Metrics
Prod Metrics >>
Next Challenges
Harder, Better, Faster, Stronger!
by