EVALUATING SECURITY ORCHESTRATION AND AUTOMATION SOLUTIONS Intelligent Security Orchestration and Automation A Comprehensive Guide to Evaluating Security Orchestration and Automation Solutions OVERVIEW The market for solutions that automate some part of the process from security alert to eventual remediation is heating up. While many vendors opportunistically deem themselves security orchestration and automation players, most simply automate the collection and handing off of data to a person who must perform time-consuming and manual work. The purpose of this guide is to clarify the definition of a true security orchestration and automation platform and provide evaluation criteria when considering a product.
Transcript
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
A Comprehensive Guide to Evaluating Security Orchestration and Automation Solutions
O V E R V I E W
The market for solutions that automate some part of the process from security alert to eventual remediation is heating up. While many vendors opportunistically deem themselves security orchestration and automation players, most simply automate the collection and handing off of data to a person who must perform time-consuming and manual work. The purpose of this guide is to clarify the definition of a true security orchestration and automation platform and provide evaluation criteria when considering a product.
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
TableofContents
A Comprehensive Guide to Evaluating Security Orchestration and Automation Solutions........................1Overview......................................................................................................................................................1
Defining Security Orchestration and Automation...................................................................................3Understanding the Challenges Organizations Face...........................................................................................5
The Capacity Challenge ................................................................................................................................................................................................. 5
Time Can Be the Enemy ............................................................................................................................................................................................... 6
Impact on the Bottom Line .......................................................................................................................................................................................... 7
Functional Requirements.....................................................................................................................8Integration with Security Products..................................................................................................................8Broad Operating System Coverage.................................................................................................................9Automated Investigation Capabilities..............................................................................................................9Integrated Threat Intelligence......................................................................................................................10Content Inspection......................................................................................................................................11Ability to Learn from Results........................................................................................................................11Automated and Semi-Automated Remediation..............................................................................................11Closed-Loop Ticketing Integration................................................................................................................11No Persistent Agent.....................................................................................................................................11
Return on Investment........................................................................................................................12Speed of Integration....................................................................................................................................12Speed of Deployment..................................................................................................................................12No Additional Resources..............................................................................................................................12Predictable Cost Structure...........................................................................................................................12
About Hexadite.................................................................................................................................15
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
Defining Security Orchestration and Automation There’s been a lot of talk about security automation, but it’s increasingly unclear what is what. For example, a Network World article on security automation last year focused mostly on threat detection, a Gartner report on Intelligent and Automated Security Controls focused on the threat intelligence component, and another recent piece referenced security automation simply as “the automation of cybersecurity controls.”
The fact is, security automation is starting to go beyond prevention and detection technologies, reaching into other important components of IT infrastructure to more reliably protect organizations. Here are four of the newest and most advanced elements you should consider when discussing security automation:
1. Policy execution. As networks have grown significantly more complex, manually managing associated security policies has become nearly impossible. Enter policy execution automation, which refers to the automation of any administrative work required of IT security. A variety of vendors offer tools for automating the management of network security policies, which can help you more easily meet internal or regulatory security requirements. Some also offer automated services for administrative tasks like user onboarding/offboarding and user lifecycle management. Automating the provisioning, deprovisioning and user access can help IT teams gain greater control over data, costs and time, and the companies offering the tools sometimes refer to themselves – or are generically referred to by others – as offering security automation.
2. Alert monitoring and prioritization. Some people view the job of automation through the lens of monitoring and prioritizing alerts. Traditionally, alert monitoring and prioritization was a manual task, and a very tedious one at that. A team of analysts in a security operations center would have to compile alerts and literally stare at monitors all day in order to determine which data points were important. Today, there are methods for automating alert monitoring and prioritization that vary in sophistication. For example, this might include setting rules and thresholds, relying on threat intelligence or implementing more advanced behavioral analytics or machine learning technology. Setting rules and thresholds is dwindling in its effectiveness, as it relies on manual input from a person to determine which alerts are important and which aren’t. And it also requires regular maintenance of those rules because cybersecurity threats are constantly changing and often hackers know exactly which alerts companies will be looking for. Relying on threat intelligence, on the other hand, is a little more reliable. This form of automation refers to the collection of threat intelligence from multiple sources, and it can help companies know which alerts to look for and which are important. For instance, if a company is able to access and consume multiple intel sources, it would know when a certain type of attack is occurring across the globe. Automated threat intelligence can then help the company prepare to protect itself against that potential, incoming attack before it’s too late.
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
Behavioral analytics and machine learning are among the most advanced forms of automation for alert monitoring and prioritization because they don’t rely on rules and thresholds or “known threats.” Instead, this type of technology can learn what normal network behavior looks like, easily and immediately pinpoint any abnormal behavior, and then statistically score the priority of each potential threat that should be investigated.
3. Incident response planning. Incident response planning is also being referred to as security automation. One way to think about this technology is as a smart ticketing system that helps companies track the evolution of a security incident and coordinate the actions required to respond. Vendors in this space help companies develop playbooks for different types of threats so they can automate portions of their response when every second counts. They automate workflow so companies can make sure they’re communicating with the appropriate internal and external contacts, adhering to regulations for topics like privacy notifications, and establishing a clear audit trail.
4. Investigation, action and remediation. Automating the investigation, action and remediation of a cyber threat is about utilizing technology to perform tasks just as a qualified cyber analyst would. In a way, the other elements of security automation – from policies, to prioritization, to planning – are all working towards this end goal of quickly finding threats and shutting them down before they impact operations. There are different aspects of what a vendor might automate when it comes to investigation, action and remediation. For example, some might only address one of those three components, while others focus on a specific task, such as automating the containment of compromised devices. There are also companies that use automation and artificial intelligence to conduct the entire process from end-to-end, just as a cyber analyst would.
All of these security automation technologies free up overtaxed security resources, allowing security teams to be less focused on mundane – but essential – tasks, and more focused on strategic initiatives that will make their organization more secure.
According to data from the Breach Level Index, 1.9 million online records were compromised every day in 2015. That’s 80,766 records every hour, or 1,346 records every minute. The near constant occurrence of data breaches shows no signs of slowing down, so companies can’t afford to have any lingering questions about the concept and capabilities of security automation.
Prioritize the automation of your IT security infrastructure and recognize that multiple elements can be automated to help keep your business safe. Automating policy execution, alert monitoring and prioritization, and incident response planning can drastically increase company productivity and reduce costs. And by fully automating the investigation, action and remediation of threats, companies can simulate the experience and logic of experienced cyber analysts at scale, thereby guaranteeing stronger security and compliance overall.
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
U N D E R S T A N D I N G T H E C H A L L E N G E S O R G A N I Z A T I O N S F A C E
Today’s incident response teams are locked in an unfair fight. Resources are short, time is tight and prioritization—the only way to manage the ever-increasing volume of alerts—is a security risk. Automated cyber-attacks are driving this disparity. And yet, most cyber-analysts are still using manual tools and writing custom code to mitigate threats.
The Capacity Challenge Security teams just don’t have the capacity to match adversaries. Research from analyst firm EMA found that 92% of companies receive 500 cyber alerts or more each month. That’s equal to 15,000 per month, and that’s a lot. If one cyber analyst can investigate roughly 10 alerts per day – one at a time – that’s 300 per month.
The result: you’d need 150 cyber analysts working 8 hour shifts 7 days a week just to keep up with current alert volume. That’s not realistic.
Due to a well-documented worldwide cybersecurity skills shortage, there are 1 million cybersecurity jobs unfilled globally. Companies simply cannot hire the capacity problem away.
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
Time Can Be the Enemy Because of the mismatch between alerts and capacity, attacks often go undetected. 80% of data breach victims don’t realize they’ve been attacked for a week or longer, with attacks often going more than 200 days before being discovered. That’s a lifetime for attackers, and there are numerous costly high profile examples that show the damage that can be done.
When measuring security effectiveness, three key indicators must be considered:
1. Mean time to notification – Once a potentially malicious activity has been detected, how long does it take before an alert makes it to the person or system responsible for investigating?
2. Mean time to investigation – Once an alert has been sent, how much time passes before the investigation begins, and what is the duration of the investigation?
3. Mean time to remediation – From alert to investigation and incrimination, what is the total elapsed time to remediation?
Use these three measures to determine your organization’s current baseline, and return to these measures when evaluating the return on investment after automation. These metrics will serve as an objective judgement of improvement.
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
Impact on the Bottom Line Finally, the lack of capacity and the increased risk created by dwell time leads to breaches that can be measured in millions, if not billions in real dollars.
The recent data breach at Yahoo! could cost the company $1 billion dollars, as acquirer Verizon is seeking a discount in its offering due to privacy concerns and loss of goodwill related to the massive breach. In more concrete terms, the cost of data breaches average $154 per record, while the average cost per data breach has reached $3.79MM.
This translates to an estimated total cost of US $300 B to US $ 1Trillion a year.
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
Functional Requirements When considering a security orchestration and automation solution, there are several unassailable functional requirements. Consider the following.
I N T E G R A T I O N W I T H S E C U R I T Y P R O D U C T S
As the value of any security orchestration and automation tool is directly proportional to the quality of the data it acts upon, it is important to integrate with the detection systems that customers use. This can include SIEM, antivirus, EDR, IPS, IDS, and DLP solutions just to name a few.
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
B R O A D O P E R A T I N G S Y S T E M C O V E R A G E
While the majority of enterprises are Windows-based, using a solution that lacks support for Mac and Linux desktops, laptops, and servers won’t provide sufficient coverage. Any security orchestration and automation solution must be able to access, assess, and remediate any endpoint regardless of operating system.
A U T O M A T E D I N V E S T I G A T I O N C A P A B I L I T I E S
The true value of a security orchestration and automation solution lies in its ability to automatically investigate every cyber alert. By automating the investigation function companies are effectively adding a team of analysts working 24x7, and all perform faster than people. This allows organizations to retain their tier 1 and 2 analysts by giving them more interesting and important work….things that humans will always do better than machines.
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
I N T E G R A T E D T H R E A T I N T E L L I G E N C E
New threats are born every minute, and known IOCs can morph and disguise themselves quickly. In order to stay on top of threats, your security orchestration and automation solution must be able to compare signatures to multiple threat intelligence feeds to determine known good, known bad, and unknown threats.
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
C O N T E N T I N S P E C T I O N
While threat intelligence can incriminate threats that are known, your security and orchestration solution must be able to inspect those threats that have not yet been identified through signature alone. By detonating unknown threats in a sandbox, inspecting content and recording behavior, your security orchestration and automation solution can assess unknown threats and determine remediation actions.
A B I L I T Y T O L E A R N F R O M R E S U L T S
When an unknown threat becomes known through content inspection, your security orchestration and automation solution should launch new investigations to search for and remediate its newly learned threat. The ability to launch new investigations based on new findings is a main benefit of automation.
Additionally, automation allows for parallel investigations to happen simultaneously to find and remediate threats on multiple hosts without relying on separate alerts.
A U T O M A T E D A N D S E M I - A U T O M A T E D R E M E D I A T I O N
Although full automation will always perform faster than requiring human intervention, your security orchestration and automation solution should give you the option to approve remediation actions. Whether by directory, group, or machine, you’ll want the option to decide your level of automation.
C L O S E D - L O O P T I C K E T I N G I N T E G R A T I O N
Your automation solution should offer a closed loop process from alert to remediation and issue closure. As most companies rely on a ticketing system as a historic record and knowledge repository, being able to send the results of an investigation and closing the incident ticket are required.
N O P E R S I S T E N T A G E N T
Your security orchestration and automation solution should be agentless, requiring no persistent presence on all endpoints, allowing for effortless scaling and speed.
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
Return on Investment As with any security purchase, the capabilities added must justify the dollars spent and time invested. The following requirements should be considered when evaluating a solution.
S P E E D O F I N T E G R A T I O N
Once purchased, how quickly can you be up and running? Will the integration involve paid hours of professional services, and custom code? Determine whether the solution has been built to immediately integrate with your current security toolset to understand just how long the integration process will take.
S P E E D O F D E P L O Y M E N T
Once connected to alert sources, how quickly can you start automatically investigating alerts? While some approaches require you to write and maintain custom code to execute investigation actions, others favor an out-of-the-box approach that begins investigating immediately after receiving the first alert.
N O A D D I T I O N A L R E S O U R C E S
If the goal of automating incident response is to free up security resources, your security orchestration and automation solution should not require additional headcount to develop and maintain custom code in order for the system to run. Requiring more people defeats the purpose and the value.
P R E D I C T A B L E C O S T S T R U C T U R E
Without predictability, it’s impossible to determine value. As some solutions charge a cost per investigation, cost per remediation action, and cost for additional support hours, it can be futile to project yearly costs. This makes ROI a guess at best.
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
Evaluation Checklist Use the following checklist to evaluate security orchestration and automation solutions.
Feature Description Requirement Met?
Integration with Existing Security Toolset The orchestration and automation solution integrates with my
detection systems (SIEM, AV, EDR, etc.).
� Yes � No
Operating System Coverage The solution can work with my Windows, Mac, and Linux servers,
desktops, and laptops.
� Yes � No
Automatic Investigation Capabilities The solution does not require customization in the target
environment to perform investigations.
� Yes � No
Automated Remediation of Confirmed
Threats
Automatic remediation methods must include removing malicious
objects on the endpoint as well as dynamically reconfiguring
network security devices.
� Yes � No
Integrated Threat Intelligence Including provider's own threat data, for analysis of suspicious
objects, as well as provide dynamic analysis of files and URL objects
using multiple sandbox technologies
� Yes � No
Deploy as VM Ability to deploy as a virtual appliance into a standard VM
environment.
� Yes � No
Autonomous Remediation Ability to complete incident response flow from alert to
remediation without human interaction.
� Yes � No
Custom Approval Levels Ability to pause automation for human approval. � Yes � No
Rapid Investigation Trigger Ability to initiate an investigation within less than 30 seconds after
an alert is received.
� Yes � No
Immediate Results Ability to complete investigations and provide a finding within 15
minutes or less on average.
� Yes � No
Parallel Investigations Ability to conduct at least 50 parallel automated investigations with
default hardware specification.
� Yes � No
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Large Scale Coverage Ability to scale up with additional hardware resources. � Yes � No
Manual Investigation Capability Ability to launch a manual investigation on any supported object. � Yes � No
Coverage Customization Ability to dynamically limit investigations to pre-defined asset
groups based on Active Directory OUs.
� Yes � No
3rd Party Ticketing Integration Ability to push alert events back to a SIEM platform or log
repository using standard formats
� Yes � No
No Additional Resources The solution does not require additional headcount or professional
services to write and maintain custom code
� Yes � No
Predictable Cost Structure The solution is priced predictably, and I understand my yearly cost
and can calculate ROI.
� Yes � No
E V A L U A T I N G S E C U R I T Y O R C H E S T R A T I O N A N D A U T O M A T I O N S O L U T I O N S
Intelligent Security Orchestration and Automation
About Hexadite Hexadite is the first agentless intelligent security orchestration and automation platform for Global 2000 companies. By easily integrating with customers’ existing security technologies and harnessing artificial intelligence that automatically investigates every cyber alert and drives remediation actions, Hexadite enables security teams to amplify their ability to mitigate cyber threats in real-time. For more information, follow @Hexadite on Twitter or visit www.hexadite.com.
