Date post: | 08-May-2015 |
Category: |
Technology |
Upload: | akshaya-mahapatra |
View: | 1,971 times |
Download: | 2 times |
Software Development Life Cycle
A DevOps Approach
Akshaya Mahapatra
Akshaya Mahapatra Software Engineer @ VCE (Virtual
Computing Environment) Experienced in
◦ Java/J2EE◦ VMware products
vSphere, vCloud◦ Puppet
Developing solutions for application deployment on cloud
About me
3
Why DevOps? Why now? Stages in SDLC
◦Check-in◦Build and Test◦Infrastructure setup◦Release◦Monitoring and scaling
A typical use case Q & A
Agenda
Dev and Ops have different goals.◦Features vs SLA
Skills are different.◦Dev team lacks infrastructure knowledge◦Ops team unaware of complexity of
development. Slow feedback loop
◦Business -> Ops ->Dev
Dev Ops - Two Teams
An idea, practice or culture that brings Dev, Ops and Business together.
How?◦Repeatable processes◦End-to-end automation.◦Continuous monitoring
What is DevOps?
Virtualization and Cloud Computing◦Programmable Infrastructure◦Just-in-time Provisioning Network Compute Storage
Agile software development◦Speed is critical.
DevOps – Why now?
Application Needs◦Nodes can fail◦No “manual” configuration◦Heterogeneous Execution environments
Advancement of technology◦Puppet eco-system
DevOps – Why now?
Software Development Life Cycle
Continuous Integration
Continuous Deploymen
t
Check-in
Continuous Monitoring
Scale Out
Stages in SDLC - Packaging
Check-in
Version Control Continuous
Integration System
Build Test
Packaged Application
warearAppliancevApp
Stages in SDLC – Prepare Infra
Packaged Application
Infra Setup
LB APP
DB
vSphere
VM Templates
IP Assignment
APP
Stages in SDLC - Deployment
Configuration Management
MonitoringTools
Scale Out
MonitoringLB APP
DB
vSphere
Config
APP
Apache License Initial Release – October 2000Central Repository.
◦Better access controlFast branching and tagging.
Version Control - Subversion
Version Control - GIT
De-centralized. Inherent redundancy.
Local repository. Efficient use of space.Offline Use
SVN vs GIT
SVN GIT
svn checkout.svn addsvn commit.svn commitsvn diffsvn updateonline
git clonegit addgit commitgit pushgit diffgit pullonline/offline
Version Control – Best Practice
All “stable” code goes to trunk.Major development in branches.
◦Merge to trunk when stable.Snapshots for release are tagged.
Simple, Java basedXML based configurationDependency Management
◦Project ->TargetEase of logging and debugging. Integration with major IDEs.
Build – Apache Ant
Beyond “typical” build◦Reporting◦Collaboration.◦Project life cycle.
Automated dependency management
Sensible default configurationBetter suited for complex projects
Build - Maven
settings.xml◦ ${user.home}/.m2/settings.xml
<proxies> <proxy> <host>172.30.100.25</host> <port>8080</port> </proxy> </proxies><repositories> <repository> <id>central</id> </repository></repositories>
Maven
Dependency in pom.xml<dependencies> <dependency>
<groupId>log4j</groupId><artifactId>log4j</artifactId>
<version>${log4j.version}</version> </dependency></dependencies>
Maven
Distribution Management in pom.xml<distributionManagement><repository> <id>devops-snapshot</id> <name>DevOps Repository</name> <url>http://codecamp.com/repo/devops</url></repository> </distributionManagement>
Maven
Open source Continuous Integration server
http://jenkins-ci.org/◦Written in Java◦Performs jobs◦Numerous plugins◦Wide support◦Post build actions
Jenkins
Continuous Integration -Jenkins
Code Repository(GIT/SVN)
MavenAnt
JunitTestNG
Selenium
Check-in Workspace
Target Server
Jenkins(Hudson) Build
Update
Test3
1
2
4 Packaging
PackagedApplication
Infrastructure Setup
Clone VM
ConfigureVM
Blue PrintsTemplates
Infra Setup
3
1
2
Power On
BuildSuccessful
4
VM
VM
VM
VM
vSphere
Cloning Specification:◦Networking Info◦DNS◦Domain
Host/Cluster Information OS Customization
◦Sysprep in windows◦IP configuration in Linux
Infrastructure Setup
Supported by VMware and up-to-date Clone VM:
◦ Create Clone Spec: Set host name, network info, data store, host resources, folder etc.
◦ Task task = vm.cloneVM_Task(vcFolder, vmName, cloneSpec);
◦ if (task.waitForTask() == Task.SUCCESS) return true; else return false;
VMware API - VIJava
Supported by VMware and up-to-date Clone VM:
◦ Add-PSSnapin VMware.VimAutomation.Core◦ Connect-VIServer -Server $VC◦ New-VM -Name $cloneName -VM $sourceVM -
ResourcePool $respool -Datastore $datastore
VMware API - PowerCLI
PowershellNew-NetIPAddress –InterfaceAlias “Local Area Connection” –IPv4Address “192.168.1.100” –PrefixLength 24 -DefaultGateway 192.168.1.1
Set-DnsClientServerAddress -InterfaceAlias “Local Area Connection” -ServerAddresses 192.168.1.254, 192.168.1.253
IP Address- Windows
/etc/sysconfig/networkNETWORKING=yesHOSTNAME=codecamp.devops.comGATEWAY=192.168.1.1
/etc/sysconfig/network-scripts/ifcfg-eth0DEVICE=eth0BOOTPROTO=staticBROADCAST=192.168.1.255IPADDR=192.168.1.10
/etc/resolv.confsearch devops.com nameserver 192.168.1.2
IP Address- Redhat
Install Applications on Templates◦ Red hat with Apache◦ Windows with SQL Server◦ Cent OS with Tomcat◦ Red hat with ActiveMQ
Create Virtual Machines from templates.◦ Applications already installed.
Configure virtual machines.
VM Templates – Option 1
Templates with Base OS◦ Red hat◦ Windows◦ Cent OS
Create Virtual Machines from templates. Install required components on virtual
machines on demand using Puppet.
Better alternative..
Open source Configuration Management
http://puppetlabs.com/◦Written in Ruby◦Infrastructure as code.◦Client Server system.◦Gaining lot of traction◦Simple domain specific language (DSL)
Puppet
Puppet
Catalog
PuppetMaster
Node(Puppet Agent)
Node(Puppet Agent)
Facts
Facts
CatalogConfigRepository
Client Server Architecture
Puppet – DSL - Package
package { "sudo": ensure => "installed"
}
package { 'mysql': ensure => installed, source => ‘C:/mysql-winx64.msi',
}
Puppet – DSL - User
User { “codecamp":ensure => "present",uid => "1001",gid => "1001",comment => “Code Camp Developer",home => "/home/codecamp ",shell => "/bin/bash"
}
Puppet – DSL - File
file { "C:/apache": ensure => directory, recurse => true, source =>puppet:///modules/apache",}
Puppet – DSL - Exec
exec { "Install Apache" : command => 'install_apache.bat', require => File["C:/apache"'], path => "C:/apache/bin", }
Puppet – DSL - Service
service { "Apache2.4": ensure => running, require => Exec['Install Apache'], enable => true, }
Puppet – DSL - Class
Class apache { File { "C:/apache": }
exec { "Install Apache" : }
service { "Apache2.4": }}
Puppet – DSL - nodes.pp
node apache1 { include apache }
node tomcat1{ include tomcat}
Get information on all the nodes: Host Name, domain, IP Address facter -p
Puppet manifests can access them as global variable
Ex: “$::hostname” Customizable.
Facter – A complement
Puppet – DSL
class ntp { case $::operatingsystem {
centos, redhat: { $service_name = 'ntpd'}
debian, ubuntu: { $service_name = 'ntp' }
}…………}
Most of application deployment need Orchestration support◦Orchestration is “cumbersome” with
base puppet. Changes need to wait until next “agent” run..
Puppet – Orchestration
Open source Asynchronous Orchestration
◦Can manage puppet runs. Scalable
◦Parallel execution
Marionete Collective(mCollective)
Puppet with mCollective
PuppetMaster
Node(Puppet
and mCollective
Agent)
Catalog
mCollective Client
Node(Puppet and mCollective
Agent)
ActiveMQCommands Commands
Catalog
Find out reachable nodes◦mco ping
Find service status◦mco rpc service start service=httpd
Run puppet agent once on a node◦mco puppet runonce –W
hostname=apache1
mCollective Commands
Environments◦Development◦Test◦Production
Defined as a config parameter.Puppet Master can handle multiple environments.
Puppet Environment
Puppet Environment
PuppetMaster
Test
Development
Production
svn/dev
svn/test
/svn/prod
Managing what nodes get what classes..◦Nodes.pp grows out of control when you have thousands of nodes.
Solution: ◦External Node Classifier
Puppet – Large Deployments
Key/Value pair lookup toolProvides environment hierarchy
◦Default◦Node Name◦Environment◦Domain
Hiera (External Node Classifier)
Puppet and Hiera
PuppetMaster
Node(Puppet Agent)
Hiera
External Node Classifier
FactsCatalog(Classes)
Facts
Catalog(Classes)
Continuous Monitoring
VM
VM
VM
Hyperic
Infra Setup
VM
VMVM
PuppetMaster
Check Threshold
VMware Products◦Hyperic – OS/Infrastructure◦App Insight – Inside Application
Java Tools and APIs◦Sigar◦Jconsole
Monitoring
Use Case
A simple Web App
Web Server(Apache)
App Server
(Tomcat)
App Server
(Tomcat)
App Server
(Tomcat)
<application> <name>Puppet Demo</name>
<nodes><name>Tomcat Server</name><template>CentOS56 64bit</template><key>war_file</key><instances>3</instances>
</nodes><nodes>
<name>Apache Load Balancer</name><template>CentOS56 64bit</template>
</nodes></application>
App Blue Print
Clone templates to create virtual machines.◦One VM for apache Load Balancer◦Three VMs for Tomcat and App .war file.
Create Puppet Manifest or Configure ENC.◦Associate Puppet/Classes with node
names. Run Puppet Agent on Virtual Machines
◦In correct sequence.
Steps
Install Apache◦mco puppet -W hostname=apache1
runonce Install Tomcat
◦ mco puppet -W role=apserver runonce Configure Apache
◦mco puppet -W hostname=apache1 runonce
Steps
Puppet and Puppet Forge◦ puppetlabs.com◦ forge.puppetlabs.com/
VMware VIJava API◦ vijava.sourceforge.net/
Provisioning◦ github.com/puppetlabs/razor
More Info: