The guide to selecting the right DevOps tools for Business Agility & Optimized ROI

www.hexaware.com

WHITEPAPER

Table of Contents

Executive summary 3

DevOps at a glance 4

DevOps mantra 4

Gartner’s 3 categories of DevOps tools 5

Selection of the right DevOps tool made easy 6

DevOps metrics 7

Tap into Hexaware’s expertise to formulate DevOps automation strategy 8

Step 1: Determine the DevOps maturity 8

1. Organization Awareness 9

2. People and Process Maturity 9

3. Application Focus 9

4. Transition & Adoption Impact 9

Step 2: Determine technology stack & map tools 10

Step 3: Record tool usage pattern 10

Step 4: Analyze effectiveness of the tools with respect to fulfilling DevOps objectives 10

Sample outcome of the assessment 11

Take stock of your DevOps tools 11

About the author 11

References 11

Executive summary 3

DevOps at a glance 4

DevOps mantra 4

Gartner’s 3 categories of DevOps tools 5

Selection of the right DevOps tool made easy 6

DevOps metrics 7

Tap into Hexaware’s expertise to formulate DevOps automation strategy 8

Step 1: Determine the DevOps maturity 8

1. Organization Awareness 9

2. People and Process Maturity 9

3. Application Focus 9

4. Transition & Adoption Impact 9

Step 2: Determine technology stack & map tools 10

Step 3: Record tool usage pattern 10

Step 4: Analyze effectiveness of the tools with respect to fulfilling DevOps objectives 10

Sample outcome of the assessment 11

Take stock of your DevOps tools 11

About the author 11

References 11

Executive Summary

DevOps tools are not only a means to create products but also play a crucial role in automation which directly ties to consistency, repeatability

and faster time to market of the products.

Thus, it is important to understand the role of these tools. There is no “one size fits all” tool to address the challenges that may pop up during

your DevOps journey. Hence, you need to plan a structured approach on selecting the right tools and using them in the right manner so as to

optimize ROI.

This white paper talks about what DevOps is and why it is crucial to select the right DevOps tools. While doing this, we also highlight Gartner’s 3

categories of DevOps tools and ways of selecting the right tool from the plethora of tools available. To help you make an informed decision, we

have also listed a few parameters which can be considered while selecting a tool.

In the later pages of this white paper, we take you through our indigenously built models & tools, namely HDMI (Hexaware DevOps Maturity

Index) and HDMAT (Hexaware DevOps Maturity Assessment Tool). HDMI and HDMAT are being extensively used by our consultants to check

an organization’s readiness to adopt DevOps and to prepare a roadmap for adopting this methodology in the best possible manner.

DevOps at a Glance DevOps Mantra

DevOps at a Glance

Technology disruption is a threat to every industry. The implication is

that businesses have to think far ahead and push their IT teams,

which in turn need to be creative to build products with a shorter time

to market. The businesses also need to sustain this momentum and

IT has to follow suit every time. The processes lay out the framework

and people are motivated constantly to build in increments so that a

minimum viable product can be showcased in a shorter time frame.

Feedback mechanism is in place to constantly improve the product

quality over time.

DevOps is one such methodology that encompasses people,

processes and tools to achieve business agility, thereby shipping

software products faster to market. It is a technique that automates

the process of software delivery and changes in infrastructure while

bringing a sound coordination between the tasks carried out by

software developers and the rest of the IT team. Selecting the right

tools for DevOps is very crucial as it directly influences not just the

product development process but also the time to market of the

product. The tools also aid in team collaboration and in creating

process templates that can cover the Application Lifecycle

Management (ALM) aspects of the product.

DevOps Mantra

Many organizations have already adopted DevOps for their business

environment while several others are keen to understand how they can

transform their business by using this technique. A few others are waiting

to see the lessons learnt by early adopters before making up their mind to

use DevOps.

Below are a few areas where DevOps scores over traditional IT Ops:

• DevOps-oriented teams spend 21% less time rolling out

changes on a weekly basis

• DevOps-oriented teams spend 33% more time improving

infrastructure to avoid failures

• DevOps-oriented teams require nearly 60% less time per week

to handle support cases

There is no readymade guide to selecting the right tools for implementing

DevOps technique. An organization needs to do a reality check of its

business before adopting this technique and making it a success story. It

first needs to thoroughly review the tools it already uses and how these

tools are being used. To do this, finding answers to the following

questions can help:

• What takes more time – Build, Provision or Deployment?

• Where is the quality hit – Security, Performance or Scalability?

• Is the choice of tools making sense for Continuous Integration/

Continuous Delivery (CI/ CD)?

• Is there a sound collaboration between development and other IT

operations?

• How are the operations monitored and the feedback gathered?

Gartner’s 3 categories of DevOps tools

According to Gartner, DevOps tools can be divided into 3 major categories: DevOps ready, DevOps enabled, and DevOps capable. A brief

explanation of each of them is given in Fig 1 below (for more details, please refer to the link mentioned in the References section at the end of

this document):

1. DevOps-ready tools are out-of-the-box solutions designed to support at least one of the workflow steps in DevOps. These tools offer

application release automation and continuous configuration automation.

Examples: BMC, VMWare, Chef, Puppet, Ansible

2. DevOps-enabled tools are designed to work in a pipeline environment and enable activities for development, testing, quality assurance and

production for the application and infrastructure. The tools focus mainly on integrity and fidelity of the application and infrastructure. These

may not be new technologies, but they have direct extensibility for DevOps projects. Tools included in this category perform functions like

Continuous Integration, continuous quality, code review and static analysis (static application security testing [SAST]), testing automation, and

environment (pipeline) management.

Examples: Jenkins, Atlassian, Microsoft, Travis CI, Delphix, HP, IBM, CheckStyle, Coverity, SonarQube, Veracode, Cucumber, FitNesse, CloudBees, Electric Cloud, ElasticBox

3. DevOps-capable tools have (typically) been around for years, but they usually fall into another ITOM (IT Operations Management) category.

However, these stand-alone tools can work in a DevOps pipeline when configured correctly. Monitoring, security testing (dynamic application

security testing [DAST]) and lab management tools fall into this category. Some newer monitoring tools include characteristics and traits of

DevOps tools and are therefore more easily adopted specifically for DevOps projects

Examples: AppDynamics, New Relic, Datadog, Elasticsearch/Logstash/Kibana, Ganglia, Splunk, Veracode, PortSwigger, N-Stalker, OpenVAS, Microsoft, IBM, CollabNet, Amazon, Dell

DevOpsCapable

•Stand-alone tools that can work in a DevOps pipeline when configured correctly

•Tools that have been around for years and would most usually fall into another bucket

•Tools that areas close to a DevOps “out of the box” solution as possible

•Tools that are purpose-built for DevOps use casesDevOpsReady

DevOpsEnabled

•Tools designed to work in a pipeline environment that enable activities for

dev/test/QA/prod for the application and infrastructure, focusing on the integrity and

fidelity of the application and infrastructure

•Tools that may not be new technologies but have direct extensiblity for DevOps projects

Fig 2: Commonly used DevOps tools

Selection of the right DevOps tool made easy

There is no thumb rule for selecting a DevOps tool. The selection depends mainly on the nature and scope of product as well as your prime

requirements. The tools can be from either or all of the Gartner’s categories mentioned above. The Fig 2 below mentions some of the commonly

used DevOps tools.

The following tips can be helpful to select the right tool:

• If there is sufficient exposure to Cloud, then Cloud based DevOps tools like Azure VSTS (Visual Studio Team Services formerly known as

Visual Studio Online - VSO), AWS (Amazon Web Services – Code Deploy, CodePipeline, CodeCommit, OpsWorks, AMI, CloudFormation,

CloudWatch), Google App Engine can be used. Some organizations have begun to adapt open Source PaaS vendors in this category like

Cloudify, OpenStack etc.

• In case on-premise hardware and servers are available for development and QA (Quality Assurance), then deployment can be automated by

maintaining configuration in tools like Chef, Puppet or Ansible. Jenkins can act as a CI tool.

• ALM tools like Team Foundation Server (TFS), Atlassian JIRA and Rally are required to address collaboration between development, business

and operations team. Open Source vendors that participate in this space are Tuleap and OrangeScrum.

• Jenkins is the prominent CI tool that has a collection of plug-ins which integrates with a wide variety of tools from Java, Microsoft and open

source platforms. The other CI picks are Travis CI and Go Strider.Selection of the right DevOps tool made easy

There is no thumb rule for selecting a DevOps tool. The selection depends mainly on the nature and scope of product as well as your prime

requirements. The tools can be from either or all of the Gartner’s categories mentioned above. The Fig 2 below mentions some of the

commonly used DevOps tools.

SCORE

• To perform security and performance tests, Open-source tools like Metasploit Framework, Brakeman, Cuckoo Sanndbox, JMeter,

TheGrinder, Gatling, and Tsung can be explored.

• If there is a need to maintain configurations specific to various environments (Development, QA, UAT, and Production) and if the machines

need to be engaged and decommissioned as per requirements, there are open source tools available like Chef, Puppet, and Ansible. If using

Microsoft Azure, then Release Management and Lab Management aspects of VSTS would fit.

• Feedback mechanism can be configured by using Microsoft’s Application Insights from VSTS or open-source tools like Nagios and New Relic.

• Google Cloud Platform products like App Engine, Compute Engine and Container Engine can be an option if the Cloud provider is Google.

• Reporting and dashboard in DevOps is a complex undertaking and there are ALM (Application Lifecycle Management) vendors namely, VSTS,

JIRA, Rally, HP ALM, Clarity and ALMComplete. They have built their own tools to derive meaningful project lifecycle management like

overall status, Team Velocity, Sprint burndown, enhanced release burndown, sprint interference, remedial focus. The other aspect of the

monitoring is the Application health which is a crucial element of overall business success. There are various tools like Nagios, New Relic and

DataDog which can drilldown to hardware and server parameters like Internal thread count, Disk I/O, Inx Read/Write Operations etc. that

will serve as a dashboard for the administrators. Each of these tools produces metrics in its own way. In order to understand from ALM

perspective, the customer needs to switch between the data views and manually needs to figure out both project and application health

indices. Hence it is imperative to have a unified view of reporting dashboard across the ALM and monitoring tools. Hexaware has identified

ELK (Elastic, LogStash and Kibana) as a unified dashboard monitoring tool that aggregates across multiple data sources produced by different

tools. Logstash is a tool for managing events and logs and can be used to collect logs, parse them, and store them for later use (eg: for

searching). Elasticsearch is a search server based on Lucene. It provides a distributed, multitenant-capable full-text search engine with a

RESTful web interface and schema-free JSON documents. Kibana is a nifty tool to visualize logs and timestamped data.

The world is moving towards Microservices and hence it is important to get the thinking right in terms of choosing containers like Docker and

envision how other configuration management tools like Chef, Puppet, Salt Stack and Ansible fit into the DevOps chain. With the given set of

tools integrated and running in the deployment pipeline, it is important to have a disaster recovery environment setup planned for the same.

DevOps Metrics

There cannot be success without a measure. It is important to gather metrics like Deployment Frequency, Lead Time, % of Failed Deployments,

Mean time to Recovery (MTTR), Ticket Volume and % Change in User base from the tools and measure them periodically. The deployment

frequency is easy to measure as this depends on the release plan and business criticality. Lead time is the time taken for the new code to move

from development to production. Optimizing the lead time will result in better time to market. It is also important to keep a check on the

percentage of failed deployments. MTTR is more about the product’s robustness and resilience. High percentage of ticket volume indicates that

there are more issues incurred. Change in user base is addition of % of new users to the product. Most of these metrics can be obtained from the

DevOps tool(s), some of which are listed below:

• Deployment Frequency

• Lead Time (from Development to Deployment)

• % of Failed Deployments

• Mean Time to Failure (MTTR)

• % of Ticket Volume

• % Change in User base

Tap into Hexaware’s expertise to formulate DevOps automation strategy

Hexaware will assess the readiness to adopt DevOps at an organization, portfolio, program or at project level, with a roadmap and next steps as

the outcome. Post this assessment, Hexaware can then help in implementing the roadmap and its sustenance. It would typically cover tool suite

selection, Coaching, Continuous Integration, Testing Automation, Continuous Delivery, On Demand Provisioning and Release Management.

The assessment process is elaborated in 4 steps below:

Step 1: Determine the DevOps maturity

The objective here is to determine the team culture and the level of adaption the environment in question may have with respect to Release

Management automation and reporting.

Hexaware defined DevOps Maturity scale, HDMI

(Hexaware DevOps Maturity Index), is used to

determine DevOps maturity and its acceptance. With

HDMI as the scale, Hexaware has developed an

assessment tool, HDMAT (Hexaware DevOps Maturity

Assessment Tool) that is used by our consultants during

the assessment phase of the engagement to arrive at a

strategy and determine the roadmap and to plan next

steps.

DevOps

Expert

(Optimizing)

Defined

(Organization Framework)

Adaptive

(Process driven)

Organization Awareness

People & Process Maturity

Application Focus

Transition & Adoption Impact

Ad-Hoc

(People driven)

•Process Automation

•Continuous Improvement

•Self Organizing Team

•Knowledge Sharing

•Infrastructure Readiness

•Continuous Integration

& Delivery

•Artifact Repository

•App Performance

•Multi-Level Adaptive

Planning

•Project Progress Tracking

•Test Driven Development

•Adoption Impact

Fig 3: HDMAT Assessment Parameters

A high level overview about the assessment parameters (as mentioned in Fig 3 above) is given below:1. Organization Awareness:

It covers the organization’s cultural characteristics and its ability to adopt process automation successfully.

2. People and Process Maturity:

It covers exposure and acceptance levels of involved stakeholders (managers and Implementation team members)

for adopting DevOps automation tools. It also assesses maturity of "Process automation tool" to meet technical expectations (Non

Functional Requirements).

3. Application Focus:

It evaluates how process automation can help in the development of software.

4. Transition & Adoption Impact:

It tries to gather the data points around transition required for automation and plausible areas of impact.

Step 2: Determine technology stack & map tools

A prioritized list of projects from the portfolio inventory is used to determine the technology stack to be used as the basis to formulate a

strategy for DevOps automation. Based on this, the tools for SDLC/ PLC are identified in line with the organization’s IT strategy and customized

as per requirements.

The below stated are some DevOps lifecycle areas considered to align tools and identify respective challenges for each product or program:

DevOps

ALM Collaboration

Provisioning

Functional Testing Performance Testing

Monitoring & Feedback

MobilityCloud

Deployment

COTS

Security Testing

Unit Testing

BuildCI

Code Review Container Management

Step 3: Record tool usage patternThis and the next step are executed when a minimal state of automation exists or during steady state for audit/ health checks.

Apart from preparing tools metrics, it is important to understand how these tools are being used. For example, if there is a source control tool

used and if it is used on a monthly basis, then it is being under-utilized. Or if there are no automated builds, it means that developers have to

spend more time in building the product. The metrics used note the tool usage frequency along with effective score. Depending on the

frequency and the score, score card index is calculated using an appropriate method.

The metrics of tools data will result in many possibilities. The tools used in one product may not be a reflection of tools used in other products. In

this case, Hexaware team will strive to optimize tool suites used for CI (Continuous Integration), feedback and collaboration along with flavors

like Cloud and mobility

Step 4: Analyze effectiveness of the tools with respect to fulfilling DevOps objectivesThe objective here is to check if the tools & processes being used are actually meeting the desired maturity level. It would assess and track the

metrics that were identified and agreed upon at Step 1 (Baseline phase) and the goals set against each respectively. It will also give an

opportunity to identify current bottlenecks in enabling DevOps as well as areas of improvement. This would also allow to set a new target

maturity benchmark when the outcome determines that a particular targeted maturity level has been sustained for an agreed timeframe.

DevOps

DevOps

Sample outcome of the assessment

About the authorLakshminarayanan RS is a Solution Architect with Hexaware Technologies. He has extensive experience in consulting and building products in

.NET, Open Source technologies, BPM & SOA and adds DevOps flavor to every assignment.

References

http://www.gartner.com/technology/reprints.do?id=1-2EYV3IF&ct=150511&st=sb#t-d2e169

http://zeroturnaround.com/rebellabs/why-your-organization-hates-devops-and-wont-implement-it-this-year-again/

Take stock of your DevOps tools

Like any other technology, DevOps is shaping up fast and has reached a measurable stage where tangible and intangible benefits can be figured out well. The key lies in understanding the premise in which a business is being operated and how optimizations can be made in the deployment pipeline by using the right tools.It is always beneficial to align new tool choices with the existing toolset so as to sustain your future scale and growth without making unreasonably huge investments. Hexaware’s HDMI and HDMAT have been helping enterprises worldwide to take an informed decision on whether they are ready to adopt DevOps. Furthermore, our consultants not only provide a roadmap to adopt this methodology but also select the right tools for you and monitor if these tools are performing as per the expectations.In the end, the most important objective is to achieve desired time to market with efficiency and precision.

Safe Harbor StatementCertain statements in this press release concerning our future growth prospects are forward-looking statements, which involve a number of risks, and uncertainties that could cause actual results to differ materially from those in such forward-looking statements. The risks and uncertainties relating to these statements include, but are not limited to, risks and uncertainties regarding fluctuations in earnings, our ability to manage growth, intense competition in IT services including those factors which may affect our cost advantage, wage increases in India, our ability to attract and retain highly skilled professionals, time and cost overruns on fixed-price, fixed-time frame contracts, client concentration, restrictions on immigration, our ability to manage our international operations, reduced demand for technology in our key focus areas, disruptions in telecommunication networks, our ability to successfully complete and integrate potential acquisitions, liability for damages on our service contracts, the success of the companies in which Hexaware has made strategic investments, withdrawal of governmental fiscal incentives, political instability, legal restrictions on raising capital or acquiring companies outside India, and unauthorized use of our intellectual property and general economic conditions affecting our industry.

www.hexaware.com | marketing@hexaware.com

About HexawareHexaware is one of the leading, global providers of IT, Application, Infrastructure, BPO and Digital services. Our business philosophy of Shrink IT, Grow Digital allows customers to significantly shrink commodity IT spend while partnering with them to embrace digitalization. The Company focuses on key domains such as Banking, Financial Services, Capital Market, Healthcare, Insurance, Manufacturing, Retail, Education, Telecom, Travel, Transportation and Logistics. Hexaware focuses on delivering business results and leveraging technology solutions by specializing in services like; Application support, development and maintenance, Enterprise Solutions, Human Capital Management, Business Intelligence & Analytics, Digital Assurance (Testing), Infrastructure Management Services, Digital and Business Process Services. Founded in 1990, Hexaware has a well-established global delivery model armed with proprietary tools and methodologies, skilled human capital and SEI CMMI-Level 5 certification. For additional information logon to: www.hexaware.com

India Headquarters152, Sector – 3Millennium Business Park‘A’ Block, TTC Industrial AreaMahape, Navi Mumbai – 400 710Tel : +91-22-67919595Fax : +91-22-67919500

EU HeadquartersLevel 19, 40 Bank Street,Canary Wharf,London - E14 5NRTel: +44-020-77154100Fax: +44-020-77154101

APAC Headquarters180 Cecil Street,#11-02, Bangkok Bank Building,Singapore 069546Tel : +65-63253020Fax : +65-6222728

NA HeadquartersMetro 101, Suite 600,101 WoodAvenue South, Iselin,New Jersey - 08830Tel: +001-609-409-6950Fax: +001-609-409-6910