CHOOSING THE BEST WEB APP SECURITY SCANNER
WHO AM I ?Chirita Ionel
Application Security Analyst @
OWASP Chapter board member
WHAT DO WE WANT FROM A SCANNER? Wide Coverage Fast scans Low number of false positives Low number of false negatives Scalability Easy to use Permanent vulnerability database updates To be Cheap !?
W.A.S. EVALUATION CRITERIA Hardware Requirements & support Protocol support Authentication Session management Crawling Data Parsing Testing Command and control Reporting
HARDWARE REQUIREMENTS & SUPPORT
Thick client vs cloud
PROTOCOL SUPPORTTransport support
HTTP1.0 & HTTP1.1 SSL/TLS HTTP keep alive HTTP compression HTTP user agent configuration
Proxy support
HTTP1.0 & HTTP1.1 proxy Socks 4 proxy Socks 5 proxy PAC file support
AUTHENTICATION Basic Digest HTTP negotiate – NTLM & Kerberos Html form-based
Automated Scripted Non-automated
Single sign on Client SSL certificates Other
SESSION MANAGEMENT Session management capabilities
Start a new session Detect if the session is expired Reacquire session token
Session management token type support HTTP cookies HTTP parameters HTTP URL path
Session token detection Session token refresh policy
CRAWLING Define starting URL
Define additional hostname or exclusions for specific criteria
Support automated from submission
Detect error pages and custom 404 pages
Redirect support
DATA PARSING HTML JavaScript VBScript XML Plaintext ActiveX Objects Flash
TESTING
COMMAND AND CONTROL Schedule scans
Pause / resume
Real-time status of running scans
Run multiple scans simultaneously
GUI, CLI and web based interface
Extensibility & interoperability
REPORTING Executive summary
Technical detailed report
Delta reports
Compliance report
Customization
Report data file format
SO YOU SHOULD JUST USE THE BEST SCANNER, RIGHT? Why do you mean by “best” ?
Or the cheapest ?
By Larry Suto
WHAT ABOUT …
… running each vendor's scanner against each of the vendor's test sites and comparing the results
SUMMARY OF RESULTS
Acunetix
IBM Appscan
BurpSuite
Hailstorm
NTOSpider
Qualys
HP Webinspect
0 20 40 60 80 100 120
Falsely Reported and Missed Vulnerabilitites
False Negative False Positive
Acunetix
IBM Appscan
BurpSuite
Hailstorm
NTOSpider
Qualys
HP Webinspect
0 20 40 60 80 100 120 140 160
Vulnerability FindingsTrained Point & Shoot
SUMMARY OF RESULTS
Acuneti
x
IBM Appsca
n
BurpSu
ite
Hailstor
m
NTOSp
ider
Qualys
HP Web
inspe
ct0
20406080
100120140160
Vuln's Found Vuln's Missed FP's Reported
CASE STUDY
By Chirita Ionel
FP's Rported
0 1 2 3 4 5 6 7 8 9
FP's reportedIBM Qualys WebInspectVeracode Acunetix
Vuln's Found
0 1 2 3 4 5 6 7 8 9 10
Vuln's FoundIBM Qualys WebInspectVeracode Acunetix
Scan Time
0 1 2 3 4 5 6 7 8 9 10
Scan TimeIBM Qualys WebInspectVeracode Acunetix
Stability
0 1 2 3 4 5 6 7 8 9 10
Stability IBM Qualys WebInspectVeracode Acunetix
ON TOP OF ALL -> GARTNER MAGIC QUADRANT
SO ?