Post on 11-May-2015
transcript
Django Infrastructure @ UGentKevin Van Wilder
Why We Set Up the Django Infrastructure
As an Alternative to Plomino● Rapid application development● yet Reliable, Maintainable, Scaleable● Better at Handling Large Loads● Seperate Applications from Portal Site
Source Code Infrastructure
Git Repositories
Each project has 2 repositories
● site-<projectname>.git○ Django Source Code○ Business Logic of the application
● pkg-<projectname>.git○ Debian Package Source Code○ Installation Logic of the application
Each repository has a specific code structure.
○ Why? Automation!
Git Flow
http://nvie.com/posts/a-successful-git-branching-model/
Server Infrastructure
Load Balancerdjprd
Web + Appdjprd1
Web + Appdjprd2
...
DB
Overview Production Infrastructure
Managed by Developers
Managed by Jenkins Pipeline & Puppet
Load Balancerdjprd
Web + Appdjprd1
Web + Appdjprd2
DBframboo
s
Load Balancerdjqas
Web + Appdjqas1
Web + Appdjqas2
Web + Appdjtst1
Web + Appdjdev1
Complete Infrastructure
“Web + App” Servers
PuppetizedContinuous Deployment via JenkinsEach server contains all the appsLoad Balanced
Database
framboos.ugent.beMaar migratiemogelijkheden naar:● PostgreSQL● MySQL● SQLite● Oracle
Support Tools
Version Overview
Supporting Tools
Supporting Tools
Other Tools
AnsibleMaintenance Task Orchestration● db syncs● process restarts● etc
Continuous Deployment Pipeline
Jenkins
Project Specific Buildsteps
Project Independent Buildsteps
Build Unit Test Acceptance Test Package Deploy TST
Deploy QAS
Deploy PRD
Onderwijstips
Kwalifacts
Telefoonboek
UGent Mobile
Abstract Jenkins Pipeline
Abstract Jenkins Pipeline
Build
Unit Test
Acceptance Test
Package
Deploy TST
Deploy QAS
Deploy PRD
Start Project Build The only part of the pipeline that is specific to a project
Starts execution of the pipeline
Calls the “Build” job with the project-specific parameters:
PROJECT_NAME
PACKAGE_NAME
To add a new project pipeline, simply...
Duplicate this job
Change the values of the parameters
Abstract Jenkins Pipeline
Build
Unit Test
Acceptance Test
Package
Deploy TST
Deploy QAS
Deploy PRD
Start Project Build
Build
Clones the Git code for the web application
Unit Test
Finds all the written unit tests and executes them
Acceptance Test
Currently not yet implemented.
Finds all the written acceptance tests and executes them
Package
Clones the Git code for the debian package
Copies over the tested source code
Builds a Debian Package
Abstract Jenkins Pipeline
Package
Deploy TST
Deploy QAS
Deploy PRD
Start Project Build
Deploy TST
Built packages are automatically deployed to Testing
Deploy QAS
Manual trigger for deployment to Quality Assurance
Can only be triggered if successful deploy to TST
Deploy PRD
Manual trigger for deployment to Production
Can only be triggered if successful deploy to QAS
Build
Unit Test
Acceptance Test
Future
Future
Asynchronous Messaging (RabbitMQ)Caching
Response Caching (Varnish)Low-Level Caching (Memcached)
Acceptance Testing (Selenium)Seperate database for each environmentMore Monitoring (Graphite)
Kevin Van Wilder@KevinVanWilderOpen Source Consultant
INUITSDuboisstraat 50 - 2060 Antwerphttp://www.inuits.eu
QUESTIONS ?