27
Security of Open Source Web Applications Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna
Security of Open Source Web Applications
Maureen Doyle, James WaldenNorthern Kentucky University
Students: Grant Welch, Michael WhelanAcknowledgements: Dhanuja Kasturiratna
1. Research Objective2. Evolution of Web App Security3. Security Resource Indicator4. Vulnerability Type Analysis5. Code Metrics6. Analysis Limitations7. Conclusion
10/12/2009University of Kentucky 2
Outline
Goal: Identify predictors for vulnerability density (VD) and change in VD for open source web applications.
Research questions:1. Can software security practices predict
evolution of VD over time?2. Can code size or complexity predict VD?3. Can code change metrics predict VD?
10/12/2009University of Kentucky 3
Research Objective
Static Analysis◦ Nagappan and Ball, ICSE 2005a◦ Coverity Open Source Report 2009◦ Fortify Open Source Security Study 2008
Complexity and Change Metrics◦ Nagappan and Ball, ICSE 2005b◦ Nagappan, Ball, and Zeller, ICSE 2006◦ Shin and Williams, QoP 2008
10/12/2009University of Kentucky 4
Related Work
Reported Vulnerabilities in NVD or OSVD◦ Coarse-grained time evolution.◦ Difficult to correlate with revision.◦ Undercounts actual vulnerabilities.
Dynamic Analysis◦ Expensive.◦ False positives and negatives.◦ Requires installation of application.
Static Analysis◦ Expensive.◦ False positives and negatives.◦ Static Analysis Vulnerability Density =
vulns/kloc.10/12/2009University of Kentucky 5
Measuring Vulnerabilities
Selection process◦ PHP web applications from freshmeat.net.◦ Subversion repository with 100 weeks of
revisions.Revisions
◦ One revision selected per week for analysis.Range of projects
◦ 14 projects met selection criteria.◦ 5,800 to 388,000 lines of code (2008).◦ Removing highest & lowest, SLOC range of 25-150 kloc.
10/12/2009University of Kentucky 6
Samples
Overall security improvement.◦ first week average: 8.88 vulns/kloc◦ final week average: 3.30 vulns/kloc
Average SAVD high vs. Coverity’s 0.30 SAVD.◦ Language differences: C/C++ vs. PHP.◦ Vulnerability differences buffer overflows vs XSS/SQL.
No correlation with NVD vulnerabilities.◦ NVD correlated with freshmeat popularity.
10/12/2009University of Kentucky 7
General Results
10/12/2009University of Kentucky 8
Evolution of SAVD: 2006-2008
10/12/2009University of Kentucky 9
Variation between Web Apps
10/12/2009University of Kentucky 10
Example: Addressing Security Issues
Public security resources on project site◦ URL for installation or configuration security◦ Dedicated address to report security bugs◦ Database of known security vulnerabilities◦ Documentation of secure development practices
Results◦ Correlation of r = 0.67 (p < 0.05) with D SAVD
10/12/2009University of Kentucky 11
Security Resource Indicator
10/12/2009University of Kentucky 12
Security Resource Indicator
Vulnerability Type Analysis
2006 2008
10/12/2009University of Kentucky 13
10/12/2009University of Kentucky 14
Type Changes: 2006-2008
University of Kentucky 10/12/2009 15
Changes by Vulnerability Type
10/12/2009University of Kentucky 16
Severity Class Changes: 2006-2008
Size measure◦ Source Lines of Code (SLOC)
Complexity measures◦ Cyclomatic Complexity◦ Nesting Complexity◦ Maximum, average, total
Change measures◦ Churn = lines added + changed◦ Lines deleted
10/12/2009University of Kentucky 17
Code Metrics
1
21. do loop2. stmt3. end loop
3
CC = E – N + 2 P = 3 – 3 + 2*1
10/12/2009University of Kentucky 18
Code Metrics (All releases)
10/12/2009University of Kentucky 19
Code Metrics (1st, Final release)
May not apply to apps that didn’t meet criteria◦ Non-PHP applications◦ No SVN repository with two years of history
False positives◦ 18.1% rate from two sample applications◦ Coverity found a rate under 14% for their study
SAVD will differ between static analysis tools
10/12/2009University of Kentucky 20
Analysis Limitations
OS PHP web app security improved:◦ 8.88 to 3.30 SAVD from 2008 to 2006.◦ But 8 of 14 apps increased SAVD over period.
SRI can indicate which apps will improve.
No single code metric is predictive for SAVD.◦ Complexity is an indicator for SAVD.◦ Churn is not an indicator for SAVD.
10/12/2009University of Kentucky 21
Conclusions
Why does app security vary so much?◦ Analyze security processes for each app.
How do we validate SAVD measurement?◦ NVD count correlates with popularity.
Java web applications◦ How does Java SAVD compare with PHP SAVD?◦ How do trends compare between Java and PHP?◦ More software metrics available for Java.
10/12/2009University of Kentucky 22
Future Work
10/12/2009University of Kentucky 23
Extra Slides
10/12/2009University of Kentucky 24
SAVD vs Time and Size
10/12/2009University of Kentucky 25
SAVD vs. Nesting
10/12/2009University of Kentucky 26
SAVD vs. Churn
University of Kentucky 10/12/2009 27
