Date post: | 21-May-2015 |
Category: |
Technology |
Upload: | perforce |
View: | 5,852 times |
Download: | 0 times |
1
Control Everything
Gil Clark System Architect Intuit, Inc. Payments Division
2
Topics / Agenda
• DevOps Automation • Not much Build Automation (lots of other
presentations around this) • Not Test Automation (whole other world)
3
But first…
• What are we supposed to control? • Source code? – Sure, we all know that • But there is so much more • Because the first DevOps rule is…
4
Automate Everything!
Even has its own T-‐Shirt
5
What is “Automate”?
1) Start with some content that represents “Everything”
OS PVM Code
Data
OS PVM Code
OS PVM Code
Network
2) Get it to the Compute Pla1orm
3) Configure and Run the Compute Pla1orm
6
What is “Everything” OS
(sudoers, IPtables, services,
crontab, etc., etc.)
PVM (heap space, treads, class paths, etc.,
etc.)
Source Code (preOy obvious)
Code Config (properQes files, class paths,
libraries, etc., etc.)
Data (schema, seed data, config data, etc.,
etc.)
But this is not complete, we have all the automaQon “code” as well – what is that?
7
“Get it to the platform” – Phase I Deployment Packaging Automation
1. Sync from P4 • App Code • App Config
2. Build libraries è jar, war, etc.
3. Build packages è zip, rpm, etc.
• Including App Config 4. Publish packages è file
server, yum, apt
Content Packaging
File Server HTTP FTP Yum Apt
Nexus Jenkins Archive
Perforce
Release “Library”
Puppet Chef Salt
Rundeck Capistrano Fabric
Rsync/Rdist
Content SynchronizaQon
Cloud Management
8
“Run the compute platform” – Phase II Environment Orchestration Automation
1. Sync from P4 • AutomaQon Code • Host Env Config
2. Build packages è zip, rpm, etc.
• Env specific App config • Host Env config
3. Publish packages è file server, yum, apt
File Server HTTP FTP Yum Apt
Nexus Jenkins Archive
Perforce
Release “Library”
Puppet Chef Salt
Rundeck Capistrano Fabric
Rsync/Rdist
Content SynchronizaQon and Process Management
Cloud Management
Content Packaging
9
Another View of the System
Application Code and Configuration,
Schema, Scripts
Middleware and Configuration
(Jboss, Tomcat, Apache, Mule, Oracle)
Operating System Configuration
(Linux)
Configuration
• Sync file content
• Install packages
• (VM Image Management)
Orchestration
• Push file content
• Start, stop services CI Server - Host metadata - Automation Code - Configuration Content - Application Content
Host machines are classified by role which determines the content chosen by the automaQon
ApplicaQon Stack
10
So, now we have Everything OS
(sudoers, IPtables, services,
crontab, etc., etc.)
PVM (heap space, treads, class paths, etc.,
etc.)
Source Code (preOy obvious)
Code Config (properQes files, class paths,
libraries, etc., etc.)
Cloud Config (VM templates, ACL config, VPC
config, CloudFormaQon scripts, etc., etc.)
Data (schema, seed data, config data, etc.,
etc.)
AutomaQon ConfguraQon
(job definiQons, recipies)
11
Control Everything!
Intuit uses Perforce to control all this code
ConfiguraQon as Code
Infrastructure as Code
12
Example Branching Model – Overall System
DevOps Code (ConfiguraQon and OrchestraQon)
ApplicaQon A
ApplicaQon B
Code Build and Package Config
Code Build and Package Config
Datacenter A Environment A1 Environment A2 Datacenter B
Environment B1 Environment B2
13
DevOps Specific Branching
Datacenter A Environment A1
Overall System OrchestraQon Jobs and Scripts ConfiguraQon Recipes System properQes config
OrchestraQon Jobs and Scripts ConfiguraQon Recipes System properQes config Environment A2
Datacenter B
There are different techniques for merging this content • Treat as modular and compose the content (templates, placeholders, etc.) • Treat as monolithic and use branch / merge SCM tools
14
Gluing Everything Together
Datacenter Config
Environment Config
ApplicaQon Config Puppet
Rundeck
Jenkins
App 1 App 1
App 1
App 1 App 1
App 2
App 1 App 1
App 3
Jenkins Source Code
Release Server
(Zip, RPM)
ArQfact Server (Nexus)
15
Example Process Steps // Start with overall systems state; for each datacenter { select datacenter specific content, save; } for each environment { select environment specific content, save; } for each application to be deployed { select app specific content; merge content from datacenter, environment, app } move merged content to master control server for the target environment in the target datacenter; populate configuration system content (Puppet manifests and modules); populate orchestration system content (Rundeck jobs); kick configuration system (Puppet kick ala Rundeck); run orchestration jobs (Rundeck); done;