Paraso& Proprietary and Confiden1al 1
3/25/14
What We’ve Learned from SATE Arthur “Code Curmudgeon” Hicken
Chief Evangelist
Paraso& Proprietary and Confiden1al 2 Paraso& Proprietary and Confiden1al 2
Improvements since SATE IV
More security rules
BeLer handling of large code
bases
BeLer mul1-‐core
performance
Smaller memory use Faster
Paraso& Proprietary and Confiden1al 3 Paraso& Proprietary and Confiden1al 3
Using Juliet as a standard
How do you compare tools?
Baseline for accuracy and performance
Repeatable results
No need to jus1fy
Paraso& Proprietary and Confiden1al 4 Paraso& Proprietary and Confiden1al 4
Juliet wish-‐list
Easier Suite
Analysis
BeLer dead-‐code
method
Improve code vs “other” issues
More code
.NET Android
Publish unfound
universally unfound
Unexpected unfound
Paraso& Proprietary and Confiden1al 5 Paraso& Proprietary and Confiden1al 5
CWE wish-‐list
CWE status • Too broad • Ambiguous
CWE to map tools • Ballpark – maybe • Precise – no • Two “rules” for the same CWE may look for en1rely different things
• Two engines look in different ways, find different instances
Paraso& Proprietary and Confiden1al 6 Paraso& Proprietary and Confiden1al 6
Paraso& Next Steps
Always: More rules
More accurate
Lighter/faster
Full CWE map
Possible new CWE items
Analy1cs
Paraso& Proprietary and Confiden1al 7 Paraso& Proprietary and Confiden1al 7
Current problems with Sta1c Analysis
§ False Posi1ves – Percep1on vs. reality § It’s not just seman1c
§ False nega1ves § Compare/combine tool results § Finding what’s most important § Coverage – what was really checked?
Paraso& Proprietary and Confiden1al 8 Paraso& Proprietary and Confiden1al 8
Pleasant discovery
CWSS
Scoring to understand what to work on
Enable a risk-‐based approach
Paraso& Proprietary and Confiden1al 9 Paraso& Proprietary and Confiden1al 9
PIE on Pi – a proposed solu1on
Paraso&’s
Paraso& Proprietary and Confiden1al 10 Paraso& Proprietary and Confiden1al 10
Paraso& Development Tes1ng Plagorm
Paraso& Proprietary and Confiden1al 11 Paraso& Proprietary and Confiden1al 11
Execu1on/ CI Build
DTP
Raw Observa1ons
Paraso& Development Tes1ng Plagorm
xTest 10.x (Server)
xTest 9.x (Desktop)
PHPMD
API
FindBugs
API
CheckStyle
API
Other 3rd Party …
API
Web UI
xTest (Desktop)
Desktop IDE
Web UI
External System Requirements
/ Defects
Source Control
Process Intelligence
Engine
Priori1zed Findings
Workflow (Task)
Intelligence (Dashboards/Reports)
Prac1ce/Domain Data (REST API)
Policy Check
Paraso& Proprietary and Confiden1al 12 Paraso& Proprietary and Confiden1al 12
Data from everywhere
Paraso& DTP
IDE
Sta1c Analysis
Unit test
Peer review
Func1onal Test
Penetra1on Test
Build automa1on
Reuirements
Bug tracking
Paraso& Proprietary and Confiden1al 13 Paraso& Proprietary and Confiden1al 13
What is P.I.E.
• Reports • Big data
• Events • Triggers • Timers • Tasks
• Connectors • Addi1onal Data
• Data from everything
• Open APIs
Open Marketplace
Analy1cs Ac1ons
Paraso& Proprietary and Confiden1al 14 Paraso& Proprietary and Confiden1al 14
Piedeas
§ Microso& apps without programming § Test alerts via iOS § CWSS via Android § NVD -‐ Protecode
Paraso& Proprietary and Confiden1al 15 Paraso& Proprietary and Confiden1al 15
Samples outside the server
Paraso& Proprietary and Confiden1al 16 Paraso& Proprietary and Confiden1al 16
Report Center
Paraso& Proprietary and Confiden1al 17 Paraso& Proprietary and Confiden1al 17
Marketplace
Paraso& Proprietary and Confiden1al 18 Paraso& Proprietary and Confiden1al 18
§ Web § hLp://www.paraso&.com/jsp/resources
§ Blog § hLp://alm.paraso&.com
§ Social § TwiLer: @Paraso& @CodeCurmudgeon
§ LinkedIn: hLp://www.linkedin.com/company/paraso&
§ Google+: +Paraso& +ArthurHickenCodeCurmudgeon § Google+ Community: Sta1c Analysis for Fun and Profit