AppSec USA 2014 Denver, Colorado

Building Your Application Security Data Hub

The Imperative for Structured Vulnerability Information

This  presenta,on  contains  informa,on  about  DHS-­‐funded  research:  Topic  Number:  H-­‐SB013.1-­‐002  -­‐  Hybrid  Analysis  Mapping  (HAM)    Proposal  Number:  HSHQDC-­‐13-­‐R-­‐00009-­‐H-­‐SB013.1-­‐002-­‐0003-­‐I  

2

Dan  Cornell  with  a  respectable  hair  cut,  a  nice  shirt,  and  a  coat  

Dan  Cornell  •  Founder  and  CTO  of  Denim  Group  

•  SoQware  developer  by  background  (Java,  .NET,  etc)  

•  OWASP  San  Antonio  

Biography

3

So You Want To Run an AppSec Program?

4

•  ApplicaWon  Security  Challenges  – Spans  MulWple  Disciplines  – ComparaWvely  New  – Scale  of  the  Problem  

•  ApplicaWon  Security  Data  Hub  – Sources,  Sinks,  Flows  

•  Program  Metrics  and  Tracking  

Agenda

5

•  InformaWon  Security  – ApplicaWon  Security  

•  Audit  and  Compliance  •  Risk  Management  

•  (Oh  Almost  Forgot:  SoQware  Development)  •  (And  .  .  .  SoQware  Development  Is  Where  Most  of  the  Magic  Has  to  Happen)  

Spans Multiple Disciplines

6

•  Physical  Security:  Old  •  InformaWon  Security:  Kinda  New  •  ApplicaWon  Security:  Really  New  •  New  Discipline  Means  Immature  Metrics  

– Possibly  non-­‐existent,  certainly  not  generally-­‐accepted  

– Don’t  know  how  to  talk  about  the  problem  

•  New  Discipline  Means  New  Tools  – No  standards  for  interacWon  

Comparatively New Discipline

7

•  “Legacy”  Lines  of  Code  •  QuanWty  of  ApplicaWons  •  Dearth  of  Qualified  Professionals  

Scale of the Problem

8

 We  Have  a  Huge  Mul,disciplinary  Problem    

 In  An  Area  We  Can’t  Properly  Characterize    

   Where  We’re  Horribly  Outnumbered  

So . . .

9

•  Gather  Data  •  Communicate  to  Stakeholders  •  Automate  the  Heck  Out  of  Whatever  Possible  •  Repeat  

What to Do About It?

10

Applica,on  Security  Data  Hub  •  Sources,  Sinks  and  Flows  

•  Vulnerability  Data  •  DetecWon/PrevenWon  Sensors  •  Developer  Tools  •  Risk  Management  

So What Does This Look Like?

11

In  the  Absence  of  Automa,on  You’re  Doomed  •  Automate  everything  you  can  •  Free  up  people  cycles  for  people-­‐only  tasks  

Automation

12

ThreadFix  •  Create  a  consolidated  view  of  your  applicaWons  and  vulnerabiliWes  

•  PrioriWze  applicaWon  risk  decisions  based  on  data  

•  Translate  vulnerabiliWes  to  developers  in  the  tools  they  are  already  using  

•  GitHub  Site:  github.com/denimgroup/threadfix  

Open Source App Security Data Hub

13

List of Supported Tools / Technologies: Dynamic  Scanners  Acune&x  Arachni  Burp  Suite  HP  WebInspect  IBM  Security  AppScan  Standard  IBM  Security  AppScan  Enterprise  Mavituna  Security  Netsparker  NTO  Spider  OWASP  Zed  AAack  Proxy  Tenable  Nessus  Skipfish  w3aF    

Sta,c  Scanners  FindBugs  IBM  Security  AppScan  Source  HP  For&fy  SCA  MicrosoK  CAT.NET  Brakeman    

SaaS  Tes,ng  PlaHorms    WhiteHat  Veracode  QualysGuard  WAS    IDS/IPS  and  WAF  DenyAll  F5  Imperva  Mod_Security  Snort    Defect  Trackers  Atlassian  JIRA  MicrosoK  Team  Founda&on  Server  Mozilla  Bugzilla    Known  Vulnerable  Component  Scanner  Dependency  Check    

Supported Technologies

14

Supported Technologies

15

•  Vulnerability  DetecWon  

•  Vulnerability  MiWgaWon  

•  Vulnerability  RemediaWon  

Vulnerability Management

16

Vulnerability Detection

SAST   DAST   IAST  Known  

Vulnerable  Component  

Automated  

Threat  Modeling  

Code  Review  

PenetraWon  TesWng  

Manual  

Data  Hub  

17

What is a Unique Vulnerability?

•  (CWE,  RelaWve  URL)  –  Predictable  resource  locaWon  –  Directory  lisWng  misconfiguraWon  

•  (CWE,  RelaWve  URL,  InjecWon  Point)  –  SQL  injecWon  –  Cross-­‐site  ScripWng  (XSS)  

•  InjecWon  points  –  Parameters  –  GET/POST  –  Cookies  –  Other  headers  

18

Why Common Weakness Enumeration?

•  Every  tool  has  their  own  “spin”  on  naming  vulnerabiliWes  

•  OWASP  Top  10  /  WASC  24  are  helpful  but  not  comprehensive  

•  CWE  is  exhausWve  (though  a  bit  sprawling  at  Wmes)  •  Reasonably  well-­‐adopted  standard  •  Many  tools  have  mappings  to  CWE  for  their  results  •  Main  site:  hgp://cwe.mitre.org/  

19

Fill ThreadFix Up With Vulnerability Data

•  Manual  file  upload  •  REST  API  

–  hgps://github.com/denimgroup/threadfix/wiki/Threadfix-­‐REST-­‐Interface    

•  Command  Line  Interface  (CLI)  –  hgps://github.com/denimgroup/threadfix/wiki/Command-­‐Line-­‐Interface    

–  JAR  can  also  be  used  as  a  Java  REST  client  library  •  Jenkins  plugin  

–  Contributed  from  the  ThreadFix  community  (yeah!)  –  hgps://github.com/automaWondominaWon/threadfix-­‐plugin    

20

ThreadFix Jenkins Configuration

20  

21

What Does ThreadFix Do With Scan Results

•  Diff  against  previous  scans  with  same  technology  – What  vulnerabiliWes  are  new?  – What  vulnerabiliWes  went  away?  – What  vulnerabiliWes  resurfaced?  

•  Findings  marked  as  false  posiWve  are  remembered  across  scans  –  Hopefully  saving  analyst  Wme  

•  Normalize  and  merge  with  other  scanners’  findings  –  SAST  to  SAST  –  DAST  to  DAST  –  SAST  to  DAST  via  Hybrid  Analysis  Mapping  (HAM)  

22

Demo: Vulnerability Merge

23

Standard  Vulnerability  Data  Format  Couple  of  current  efforts:  •  SSVL  

– Based  on  lessons  learned  from  ThreadFix  – hgps://github.com/OWASP/SSVL  

•  OWASP  DEF  – OWASP  effort  –  hgps://www.owasp.org/index.php/OWASP_Data_Exchange_Format_Project  

•  Working  to  unify  these  

Know What Would Make My Life Easier?

24

Hybrid Analysis Mapping (HAM)

•  IniWal  research  funded  by  the  US  Department  of  Homeland  Security  (DHS)  Science  and  Technology  (S&T)  Directorate  via  a  Phase  1  and  (now)  Phase  2  Small  Business  InnovaWon  Research  (SBIR)  contract  – Acronyms!  

•  IniWal  goal:  SAST  to  DAST  merging  •  Results:  That,  plus  other  stuff  

25

Demo: Merging Static and Dynamic Scanner Results

26

Demo: Merging Static and Dynamic Scanner Results

27

Merging Static and Dynamic Results Is Cool

…But  I  want  more    •  Problem:  Many  DAST  scanners  handle  applicaWons  with  RESTful  

URLs  poorly  •  Problem:  Many  applicaWons  have  “hidden”  landing  pages  and  

parameters  that  will  not  be  found  by  standard  crawling  •  Problem:  DAST  scanner  results  can  be  hard  for  developers  to  

act  on  •  What  else  can  we  do  with  this  agack  surface  model  /  database?  

–  Clean  up  scanner  results  –  Enumerate  applicaWon  agack  surface  –  Map  dynamic  results  to  specific  lines  of  code  

28

Demo: De-Duplicate Dynamic RESTful Scanner Results

29

Demo: De-Duplicate Dynamic RESTful Scanner Results

30

Demo: Application Attack Surface (CLI)

31

Demo: Seed Scanner with Attack Surface

32

Vulnerability Mitigation

Data  Hub  

WAF/IDS/IPS  Sensor  

33

Demo: Generating Virtual Patches

34

Demo: Importing Sensor Logs

35

Security  Approaching  Development  Teams…  

•  PDFs  

•  Excel  spreadsheets  

•  “Log  into  this  new  system”  

Vulnerability Remediation

36

An  Alternate  Approach  

•  Help  ‘em  Out  

•  Take  Advantage  of  the  Tools  and  Processes  They  Are  Already  Using  

Vulnerability Remediation

37

Vulnerability Remediation

Data  Hub  

ApplicaWon  Lifecycle  

Management  

Integrated  Development  Environment  

This  is  also  called  “bug  tracking”  by  less-­‐fancy  people  

38

Mapping Vulnerabilities to Defects

•  1:1 mapping is (usually) a horrible idea –  500  XSS  turned  into  500  defects?  –  If  it  takes  longer  to  administer  the  bug  than  it  does  to  fix  the  code…  

•  Cluster like vulnerabilities –  Using  the  same  libraries  /  funcWons  –  Cut-­‐and-­‐paste  remediaWon  code  –  Be  careful  about  context-­‐specific  encoding  

•  Combine by severity –  Especially  if  they  are  cause  for  an  out-­‐of-­‐cycle  release  

•  Which developer “owns” the code?

39

Defect Tracker Integration

•  Bundle  mulWple  vulnerabiliWes  into  a  defect  – Using  standard  filtering  criteria  

•  ThreadFix  periodically  updates  defect  status  from  the  tracker  

40

Demo: Defect Tracker Integration

41

IDE Plug Ins

•  Import  vulnerability  data  to  integrated  development  environments  (IDEs)  

•  StaWc  (SAST)  scanners  – Easy  

•  Dynamic  (DAST)  scanners  – Possible  using  Hybrid  Analysis  Mapping  (HAM)  

42

Demo: Maping Vulnerabilities in IDE

43

•  Nobody  Likes  Uncertainty  

•  Measurement  Is  Key  

Risk Management

43  

44

Risk Management

Data  Hub  

GRC  

45

Vulnerability Filtering

•  Filter  vulnerability  data  – Scanner,  scanner  count  – Vulnerability  type  – Path,  parameter  – Severity  – Status  – Aging  

•  Save  filters  for  future  use  

46

Demo: Vulnerability Filtering

47

Reporting

•  Trending  •  Progress  by  Vulnerability  

– For  program  benchmarking  

•  Porpolio  Report  – For  resource  prioriWzaWon  

•  Comparison  – For  scanner/technology  benchmarking  

48

Metrics  That  Can  Help  •  Vulnerability  Prevalence  •  Vulnerability  ResoluWon  Rate  •  Mean  Time  To  Fix  (MTTF)  

What to Look For?

48  

49

Demo: Reporting

50

•  ApplicaWon  Security  Is  Hard  – Lots  of  people  and  systems  involved  

•  Data  Trumps  FUD  •  AutomaWon  Is  CriWcal  

So What Have We Covered?

50  

51

ThreadFix Links

•  Main  ThreadFix  website:  www.threadfix.org  –  General  informaWon,  downloads  

•  ThreadFix  GitHub  site:  github.com/denimgroup/threadfix    –  Code,  issue  tracking  

•  ThreadFix  GitHub  wiki:  hgps://github.com/denimgroup/threadfix/wiki    –  Project  documentaWon  

•  ThreadFix  Google  Group:  hgps://groups.google.com/forum/?fromgroups#!forum/threadfix    –  Community  support,  general  discussion  

52

Questions / Contact Information  

Dan  Cornell  dan@denimgroup.com  Twiger  @danielcornell  (210)  572-­‐4400                  

   

     

Contact