The guide to selecting the right DevOps tools for Business Agility & Optimized ROI
www.hexaware.com
WHITEPAPER
Table of Contents
Executive summary 3
DevOps at a glance 4
DevOps mantra 4
Gartner’s 3 categories of DevOps tools 5
Selection of the right DevOps tool made easy 6
DevOps metrics 7
Tap into Hexaware’s expertise to formulate DevOps automation strategy 8
Step 1: Determine the DevOps maturity 8
1. Organization Awareness 9
2. People and Process Maturity 9
3. Application Focus 9
4. Transition & Adoption Impact 9
Step 2: Determine technology stack & map tools 10
Step 3: Record tool usage pattern 10
Step 4: Analyze effectiveness of the tools with respect to fulfilling DevOps objectives 10
Sample outcome of the assessment 11
Take stock of your DevOps tools 11
About the author 11
References 11
Executive summary 3
DevOps at a glance 4
DevOps mantra 4
Gartner’s 3 categories of DevOps tools 5
Selection of the right DevOps tool made easy 6
DevOps metrics 7
Tap into Hexaware’s expertise to formulate DevOps automation strategy 8
Step 1: Determine the DevOps maturity 8
1. Organization Awareness 9
2. People and Process Maturity 9
3. Application Focus 9
4. Transition & Adoption Impact 9
Step 2: Determine technology stack & map tools 10
Step 3: Record tool usage pattern 10
Step 4: Analyze effectiveness of the tools with respect to fulfilling DevOps objectives 10
Sample outcome of the assessment 11
Take stock of your DevOps tools 11
About the author 11
References 11
Executive Summary
DevOps tools are not only a means to create products but also play a crucial role in automation which directly ties to consistency, repeatability
and faster time to market of the products.
Thus, it is important to understand the role of these tools. There is no “one size fits all” tool to address the challenges that may pop up during
your DevOps journey. Hence, you need to plan a structured approach on selecting the right tools and using them in the right manner so as to
optimize ROI.
This white paper talks about what DevOps is and why it is crucial to select the right DevOps tools. While doing this, we also highlight Gartner’s 3
categories of DevOps tools and ways of selecting the right tool from the plethora of tools available. To help you make an informed decision, we
have also listed a few parameters which can be considered while selecting a tool.
In the later pages of this white paper, we take you through our indigenously built models & tools, namely HDMI (Hexaware DevOps Maturity
Index) and HDMAT (Hexaware DevOps Maturity Assessment Tool). HDMI and HDMAT are being extensively used by our consultants to check
an organization’s readiness to adopt DevOps and to prepare a roadmap for adopting this methodology in the best possible manner.
DevOps at a Glance DevOps Mantra
DevOps at a Glance
Technology disruption is a threat to every industry. The implication is
that businesses have to think far ahead and push their IT teams,
which in turn need to be creative to build products with a shorter time
to market. The businesses also need to sustain this momentum and
IT has to follow suit every time. The processes lay out the framework
and people are motivated constantly to build in increments so that a
minimum viable product can be showcased in a shorter time frame.
Feedback mechanism is in place to constantly improve the product
quality over time.
DevOps is one such methodology that encompasses people,
processes and tools to achieve business agility, thereby shipping
software products faster to market. It is a technique that automates
the process of software delivery and changes in infrastructure while
bringing a sound coordination between the tasks carried out by
software developers and the rest of the IT team. Selecting the right
tools for DevOps is very crucial as it directly influences not just the
product development process but also the time to market of the
product. The tools also aid in team collaboration and in creating
process templates that can cover the Application Lifecycle
Management (ALM) aspects of the product.
DevOps Mantra
Many organizations have already adopted DevOps for their business
environment while several others are keen to understand how they can
transform their business by using this technique. A few others are waiting
to see the lessons learnt by early adopters before making up their mind to
use DevOps.
Below are a few areas where DevOps scores over traditional IT Ops:
• DevOps-oriented teams spend 21% less time rolling out
changes on a weekly basis
• DevOps-oriented teams spend 33% more time improving
infrastructure to avoid failures
• DevOps-oriented teams require nearly 60% less time per week
to handle support cases
There is no readymade guide to selecting the right tools for implementing
DevOps technique. An organization needs to do a reality check of its
business before adopting this technique and making it a success story. It
first needs to thoroughly review the tools it already uses and how these
tools are being used. To do this, finding answers to the following
questions can help:
• What takes more time – Build, Provision or Deployment?
• Where is the quality hit – Security, Performance or Scalability?
• Is the choice of tools making sense for Continuous Integration/
Continuous Delivery (CI/ CD)?
• Is there a sound collaboration between development and other IT
operations?
• How are the operations monitored and the feedback gathered?
Gartner’s 3 categories of DevOps tools
According to Gartner, DevOps tools can be divided into 3 major categories: DevOps ready, DevOps enabled, and DevOps capable. A brief
explanation of each of them is given in Fig 1 below (for more details, please refer to the link mentioned in the References section at the end of
this document):
1. DevOps-ready tools are out-of-the-box solutions designed to support at least one of the workflow steps in DevOps. These tools offer
application release automation and continuous configuration automation.
Examples: BMC, VMWare, Chef, Puppet, Ansible
2. DevOps-enabled tools are designed to work in a pipeline environment and enable activities for development, testing, quality assurance and
production for the application and infrastructure. The tools focus mainly on integrity and fidelity of the application and infrastructure. These
may not be new technologies, but they have direct extensibility for DevOps projects. Tools included in this category perform functions like
Continuous Integration, continuous quality, code review and static analysis (static application security testing [SAST]), testing automation, and
environment (pipeline) management.
Examples: Jenkins, Atlassian, Microsoft, Travis CI, Delphix, HP, IBM, CheckStyle, Coverity, SonarQube, Veracode, Cucumber, FitNesse, CloudBees, Electric Cloud, ElasticBox
3. DevOps-capable tools have (typically) been around for years, but they usually fall into another ITOM (IT Operations Management) category.
However, these stand-alone tools can work in a DevOps pipeline when configured correctly. Monitoring, security testing (dynamic application
security testing [DAST]) and lab management tools fall into this category. Some newer monitoring tools include characteristics and traits of
DevOps tools and are therefore more easily adopted specifically for DevOps projects
Examples: AppDynamics, New Relic, Datadog, Elasticsearch/Logstash/Kibana, Ganglia, Splunk, Veracode, PortSwigger, N-Stalker, OpenVAS, Microsoft, IBM, CollabNet, Amazon, Dell
DevOpsCapable
•Stand-alone tools that can work in a DevOps pipeline when configured correctly
•Tools that have been around for years and would most usually fall into another bucket
•Tools that areas close to a DevOps “out of the box” solution as possible
•Tools that are purpose-built for DevOps use casesDevOpsReady
DevOpsEnabled
•Tools designed to work in a pipeline environment that enable activities for
dev/test/QA/prod for the application and infrastructure, focusing on the integrity and
fidelity of the application and infrastructure
•Tools that may not be new technologies but have direct extensiblity for DevOps projects
Fig 2: Commonly used DevOps tools
Selection of the right DevOps tool made easy
There is no thumb rule for selecting a DevOps tool. The selection depends mainly on the nature and scope of product as well as your prime
requirements. The tools can be from either or all of the Gartner’s categories mentioned above. The Fig 2 below mentions some of the commonly
used DevOps tools.
The following tips can be helpful to select the right tool:
• If there is sufficient exposure to Cloud, then Cloud based DevOps tools like Azure VSTS (Visual Studio Team Services formerly known as
Visual Studio Online - VSO), AWS (Amazon Web Services – Code Deploy, CodePipeline, CodeCommit, OpsWorks, AMI, CloudFormation,
CloudWatch), Google App Engine can be used. Some organizations have begun to adapt open Source PaaS vendors in this category like
Cloudify, OpenStack etc.
• In case on-premise hardware and servers are available for development and QA (Quality Assurance), then deployment can be automated by
maintaining configuration in tools like Chef, Puppet or Ansible. Jenkins can act as a CI tool.
• ALM tools like Team Foundation Server (TFS), Atlassian JIRA and Rally are required to address collaboration between development, business
and operations team. Open Source vendors that participate in this space are Tuleap and OrangeScrum.
• Jenkins is the prominent CI tool that has a collection of plug-ins which integrates with a wide variety of tools from Java, Microsoft and open
source platforms. The other CI picks are Travis CI and Go Strider.Selection of the right DevOps tool made easy
There is no thumb rule for selecting a DevOps tool. The selection depends mainly on the nature and scope of product as well as your prime
requirements. The tools can be from either or all of the Gartner’s categories mentioned above. The Fig 2 below mentions some of the
commonly used DevOps tools.
SCORE
• To perform security and performance tests, Open-source tools like Metasploit Framework, Brakeman, Cuckoo Sanndbox, JMeter,
TheGrinder, Gatling, and Tsung can be explored.
• If there is a need to maintain configurations specific to various environments (Development, QA, UAT, and Production) and if the machines
need to be engaged and decommissioned as per requirements, there are open source tools available like Chef, Puppet, and Ansible. If using
Microsoft Azure, then Release Management and Lab Management aspects of VSTS would fit.
• Feedback mechanism can be configured by using Microsoft’s Application Insights from VSTS or open-source tools like Nagios and New Relic.
• Google Cloud Platform products like App Engine, Compute Engine and Container Engine can be an option if the Cloud provider is Google.
• Reporting and dashboard in DevOps is a complex undertaking and there are ALM (Application Lifecycle Management) vendors namely, VSTS,
JIRA, Rally, HP ALM, Clarity and ALMComplete. They have built their own tools to derive meaningful project lifecycle management like
overall status, Team Velocity, Sprint burndown, enhanced release burndown, sprint interference, remedial focus. The other aspect of the
monitoring is the Application health which is a crucial element of overall business success. There are various tools like Nagios, New Relic and
DataDog which can drilldown to hardware and server parameters like Internal thread count, Disk I/O, Inx Read/Write Operations etc. that
will serve as a dashboard for the administrators. Each of these tools produces metrics in its own way. In order to understand from ALM
perspective, the customer needs to switch between the data views and manually needs to figure out both project and application health
indices. Hence it is imperative to have a unified view of reporting dashboard across the ALM and monitoring tools. Hexaware has identified
ELK (Elastic, LogStash and Kibana) as a unified dashboard monitoring tool that aggregates across multiple data sources produced by different
tools. Logstash is a tool for managing events and logs and can be used to collect logs, parse them, and store them for later use (eg: for
searching). Elasticsearch is a search server based on Lucene. It provides a distributed, multitenant-capable full-text search engine with a
RESTful web interface and schema-free JSON documents. Kibana is a nifty tool to visualize logs and timestamped data.
The world is moving towards Microservices and hence it is important to get the thinking right in terms of choosing containers like Docker and
envision how other configuration management tools like Chef, Puppet, Salt Stack and Ansible fit into the DevOps chain. With the given set of
tools integrated and running in the deployment pipeline, it is important to have a disaster recovery environment setup planned for the same.
DevOps Metrics
There cannot be success without a measure. It is important to gather metrics like Deployment Frequency, Lead Time, % of Failed Deployments,
Mean time to Recovery (MTTR), Ticket Volume and % Change in User base from the tools and measure them periodically. The deployment
frequency is easy to measure as this depends on the release plan and business criticality. Lead time is the time taken for the new code to move
from development to production. Optimizing the lead time will result in better time to market. It is also important to keep a check on the
percentage of failed deployments. MTTR is more about the product’s robustness and resilience. High percentage of ticket volume indicates that
there are more issues incurred. Change in user base is addition of % of new users to the product. Most of these metrics can be obtained from the
DevOps tool(s), some of which are listed below:
• Deployment Frequency
• Lead Time (from Development to Deployment)
• % of Failed Deployments
• Mean Time to Failure (MTTR)
• % of Ticket Volume
• % Change in User base
Tap into Hexaware’s expertise to formulate DevOps automation strategy
Hexaware will assess the readiness to adopt DevOps at an organization, portfolio, program or at project level, with a roadmap and next steps as
the outcome. Post this assessment, Hexaware can then help in implementing the roadmap and its sustenance. It would typically cover tool suite
selection, Coaching, Continuous Integration, Testing Automation, Continuous Delivery, On Demand Provisioning and Release Management.
The assessment process is elaborated in 4 steps below:
Step 1: Determine the DevOps maturity
The objective here is to determine the team culture and the level of adaption the environment in question may have with respect to Release
Management automation and reporting.
Hexaware defined DevOps Maturity scale, HDMI
(Hexaware DevOps Maturity Index), is used to
determine DevOps maturity and its acceptance. With
HDMI as the scale, Hexaware has developed an
assessment tool, HDMAT (Hexaware DevOps Maturity
Assessment Tool) that is used by our consultants during
the assessment phase of the engagement to arrive at a
strategy and determine the roadmap and to plan next
steps.
DevOps
Expert
(Optimizing)
Defined
(Organization Framework)
Adaptive
(Process driven)
Organization Awareness
People & Process Maturity
Application Focus
Transition & Adoption Impact
Ad-Hoc
(People driven)
•Process Automation
•Continuous Improvement
•Self Organizing Team
•Knowledge Sharing
•Infrastructure Readiness
•Continuous Integration
& Delivery
•Artifact Repository
•App Performance
•Multi-Level Adaptive
Planning
•Project Progress Tracking
•Test Driven Development
•Adoption Impact
Fig 3: HDMAT Assessment Parameters
A high level overview about the assessment parameters (as mentioned in Fig 3 above) is given below:1. Organization Awareness:
It covers the organization’s cultural characteristics and its ability to adopt process automation successfully.
2. People and Process Maturity:
It covers exposure and acceptance levels of involved stakeholders (managers and Implementation team members)
for adopting DevOps automation tools. It also assesses maturity of "Process automation tool" to meet technical expectations (Non
Functional Requirements).
3. Application Focus:
It evaluates how process automation can help in the development of software.
4. Transition & Adoption Impact:
It tries to gather the data points around transition required for automation and plausible areas of impact.
Step 2: Determine technology stack & map tools
A prioritized list of projects from the portfolio inventory is used to determine the technology stack to be used as the basis to formulate a
strategy for DevOps automation. Based on this, the tools for SDLC/ PLC are identified in line with the organization’s IT strategy and customized
as per requirements.
The below stated are some DevOps lifecycle areas considered to align tools and identify respective challenges for each product or program:
DevOps
ALM Collaboration
Provisioning
Functional Testing Performance Testing
Monitoring & Feedback
MobilityCloud
Deployment
COTS
Security Testing
Unit Testing
BuildCI
Code Review Container Management
Step 3: Record tool usage patternThis and the next step are executed when a minimal state of automation exists or during steady state for audit/ health checks.
Apart from preparing tools metrics, it is important to understand how these tools are being used. For example, if there is a source control tool
used and if it is used on a monthly basis, then it is being under-utilized. Or if there are no automated builds, it means that developers have to
spend more time in building the product. The metrics used note the tool usage frequency along with effective score. Depending on the
frequency and the score, score card index is calculated using an appropriate method.
The metrics of tools data will result in many possibilities. The tools used in one product may not be a reflection of tools used in other products. In
this case, Hexaware team will strive to optimize tool suites used for CI (Continuous Integration), feedback and collaboration along with flavors
like Cloud and mobility
Step 4: Analyze effectiveness of the tools with respect to fulfilling DevOps objectivesThe objective here is to check if the tools & processes being used are actually meeting the desired maturity level. It would assess and track the
metrics that were identified and agreed upon at Step 1 (Baseline phase) and the goals set against each respectively. It will also give an
opportunity to identify current bottlenecks in enabling DevOps as well as areas of improvement. This would also allow to set a new target
maturity benchmark when the outcome determines that a particular targeted maturity level has been sustained for an agreed timeframe.
DevOps
DevOps
Sample outcome of the assessment
About the authorLakshminarayanan RS is a Solution Architect with Hexaware Technologies. He has extensive experience in consulting and building products in
.NET, Open Source technologies, BPM & SOA and adds DevOps flavor to every assignment.
References
http://www.gartner.com/technology/reprints.do?id=1-2EYV3IF&ct=150511&st=sb#t-d2e169
http://zeroturnaround.com/rebellabs/why-your-organization-hates-devops-and-wont-implement-it-this-year-again/
Take stock of your DevOps tools
Like any other technology, DevOps is shaping up fast and has reached a measurable stage where tangible and intangible benefits can be figured out well. The key lies in understanding the premise in which a business is being operated and how optimizations can be made in the deployment pipeline by using the right tools.It is always beneficial to align new tool choices with the existing toolset so as to sustain your future scale and growth without making unreasonably huge investments. Hexaware’s HDMI and HDMAT have been helping enterprises worldwide to take an informed decision on whether they are ready to adopt DevOps. Furthermore, our consultants not only provide a roadmap to adopt this methodology but also select the right tools for you and monitor if these tools are performing as per the expectations.In the end, the most important objective is to achieve desired time to market with efficiency and precision.
Safe Harbor StatementCertain statements in this press release concerning our future growth prospects are forward-looking statements, which involve a number of risks, and uncertainties that could cause actual results to differ materially from those in such forward-looking statements. The risks and uncertainties relating to these statements include, but are not limited to, risks and uncertainties regarding fluctuations in earnings, our ability to manage growth, intense competition in IT services including those factors which may affect our cost advantage, wage increases in India, our ability to attract and retain highly skilled professionals, time and cost overruns on fixed-price, fixed-time frame contracts, client concentration, restrictions on immigration, our ability to manage our international operations, reduced demand for technology in our key focus areas, disruptions in telecommunication networks, our ability to successfully complete and integrate potential acquisitions, liability for damages on our service contracts, the success of the companies in which Hexaware has made strategic investments, withdrawal of governmental fiscal incentives, political instability, legal restrictions on raising capital or acquiring companies outside India, and unauthorized use of our intellectual property and general economic conditions affecting our industry.
www.hexaware.com | [email protected]
About HexawareHexaware is one of the leading, global providers of IT, Application, Infrastructure, BPO and Digital services. Our business philosophy of Shrink IT, Grow Digital allows customers to significantly shrink commodity IT spend while partnering with them to embrace digitalization. The Company focuses on key domains such as Banking, Financial Services, Capital Market, Healthcare, Insurance, Manufacturing, Retail, Education, Telecom, Travel, Transportation and Logistics. Hexaware focuses on delivering business results and leveraging technology solutions by specializing in services like; Application support, development and maintenance, Enterprise Solutions, Human Capital Management, Business Intelligence & Analytics, Digital Assurance (Testing), Infrastructure Management Services, Digital and Business Process Services. Founded in 1990, Hexaware has a well-established global delivery model armed with proprietary tools and methodologies, skilled human capital and SEI CMMI-Level 5 certification. For additional information logon to: www.hexaware.com
India Headquarters152, Sector – 3Millennium Business Park‘A’ Block, TTC Industrial AreaMahape, Navi Mumbai – 400 710Tel : +91-22-67919595Fax : +91-22-67919500
EU HeadquartersLevel 19, 40 Bank Street,Canary Wharf,London - E14 5NRTel: +44-020-77154100Fax: +44-020-77154101
APAC Headquarters180 Cecil Street,#11-02, Bangkok Bank Building,Singapore 069546Tel : +65-63253020Fax : +65-6222728
NA HeadquartersMetro 101, Suite 600,101 WoodAvenue South, Iselin,New Jersey - 08830Tel: +001-609-409-6950Fax: +001-609-409-6910