CHOOSING THE BEST WEB APP SECURITY SCANNER

WHO AM I ?

Chirita Ionel

Application Security Analyst @

OWASP Chapter board member

WHAT DO WE WANT FROM A SCANNER? Wide Coverage

Fast scans

Low number of false positives

Low number of false negatives

Scalability

Easy to use

Permanent vulnerability database updates

To be Cheap !?

W.A.S. EVALUATION CRITERIA Hardware Requirements & support

Protocol support

Authentication

Session management

Crawling

Data Parsing

Testing

Command and control

Reporting

HARDWARE REQUIREMENTS & SUPPORT

Thick client vs cloud

PROTOCOL SUPPORTTransport support

HTTP1.0 & HTTP1.1

SSL/TLS

HTTP keep alive

HTTP compression

HTTP user agent configuration

Proxy support

HTTP1.0 & HTTP1.1 proxy

Socks 4 proxy

Socks 5 proxy

PAC file support

AUTHENTICATION Basic

Digest

HTTP negotiate – NTLM & Kerberos

Html form-based Automated Scripted Non-automated

Single sign on

Client SSL certificates

Other

SESSION MANAGEMENT Session management capabilities

Start a new session Detect if the session is expired Reacquire session token

Session management token type support HTTP cookies HTTP parameters HTTP URL path

Session token detection

Session token refresh policy

CRAWLING Define starting URL

Define additional hostname or exclusions for specific criteria

Support automated from submission

Detect error pages and custom 404 pages

Redirect support

DATA PARSING HTML

JavaScript

VBScript

XML

Plaintext

ActiveX Objects

Flash

TESTING

COMMAND AND CONTROL Schedule scans

Pause / resume

Real-time status of running scans

Run multiple scans simultaneously

GUI, CLI and web based interface

Extensibility & interoperability

REPORTING Executive summary

Technical detailed report

Delta reports

Compliance report

Customization

Report data file format

SO YOU SHOULD JUST USE THE BEST SCANNER, RIGHT? Why do you mean by “best” ?

Or the cheapest ?

By Larry Suto

WHAT ABOUT …

… running each vendor's scanner against each of the vendor's test sites and comparing the results

SUMMARY OF RESULTS

Acunetix

IBM Appscan

BurpSuite

Hailstorm

NTOSpider

Qualys

HP Webinspect

0 20 40 60 80 100 120

Falsely Reported and Missed Vulnerabilitites

False Negative False Positive

Acunetix

IBM Appscan

BurpSuite

Hailstorm

NTOSpider

Qualys

HP Webinspect

0 20 40 60 80 100 120 140 160

Vulnerability Findings

Trained Point & Shoot

SUMMARY OF RESULTS

Acune

tix

IBM A

ppsc

an

BurpS

uite

Hailst

orm

NTOSp

ider

Qualys

HP W

ebinsp

ect

0

20

40

60

80

100

120

140

160

Vuln's Found Vuln's Missed FP's Reported

CASE STUDY

By Chirita Ionel

FP's Rported

0 1 2 3 4 5 6 7 8 9

FP's reported

IBM Qualys WebInspectVeracode Acunetix

Vuln's Found

0 1 2 3 4 5 6 7 8 9 10

Vuln's Found

IBM Qualys WebInspectVeracode Acunetix

Scan Time

0 1 2 3 4 5 6 7 8 9 10

Scan Time

IBM Qualys WebInspectVeracode Acunetix

Stability

0 1 2 3 4 5 6 7 8 9 10

Stability

IBM Qualys WebInspectVeracode Acunetix

ON TOP OF ALL -> GARTNER MAGIC QUADRANT

SO ?