Best Practices for a Mature Application Security Program

About the Presenters

• Dr. Larry Ponemon, Founder of the Ponemon Institute– Leads privacy and data protection research

practices

– Pioneer in privacy auditing and the Responsible Information Management (RIM) Framework

• Ed Adams, CEO of Security Innovation – Ponemon Institute Fellow

– Privacy by Design Ambassador

– CEO by trade; engineer by heart

Agenda

• Application Security Trends

• Emerging Challenges, Technologies & Threats

• Threat Modeling & Risk Rating Your Applications

• Optimization Your Secure Software Development Lifecycle (SDLC)

Where Were We a Few Years Ago?

• No Defined Software Development Process– Policies, Requirements, and Standards are often ad-hoc and not integrated into

SDLC

• Lack of formal AppSec training program

– Most organizations do not identify, measure, or understand application security risks

– Significant disconnect between executives and practitioners regarding application security maturity

• Half are not testing for application security– 2/3 of mobile apps weren’t being tested at all

*Findings from Application Security Maturity Research

Organizations Don’t Have a Defined SDLC

• SDLC Still Lacking– Tools aren’t integrated into the SDLC – Security automation often used after deployment (too late?)– Policies and standards are still rare

• Forrester– “Organizations implementing an SDLC showed better ROI than the overall

population”

• Aberdeen– Adopting a formal SDLC process increases security and reduces severity and

cost of vulnerability incidents while generating a 4x ROI than other application security approaches

Building Security In

• Department of Homeland Security (DHS)– “Regardless of which statistic is used, there is a substantial cost savings for fixing

security flaws during requirements gathering than deployment”*

• Gartner– “Finding bugs at operations time costs you up to 100 percent effort”

Source: IBM Systems Sciences Institute

Relative cost of fixing security flaws during the different development phases

Implementation6.5

Testing15

Post Release60

Design1

0

10

20

30

40

50

60

70

Time

Cost

Source: National Institute of Standards & Technology (NIST)

*DHS: Estimating Benefits from Investing in Secure Development

Most Organizations Don’t Understand AppSec Risk

• Hackers are going after the easy pickings: applications (various sources)– 90%+ of vulnerabilities are in application, not network layer– 92% of attacks are not difficult and exploit KNOWN vulnerabilities

• Applications are often not risk-ranked– How do you know which to test? How deep?

• Web application firewalls (WAFs), application whitelisting (AWLs), and run-time application self protection (RASP) not widely adopted and often misconfigured

• Organizations focus on vulnerabilities, not risk– Most vulnerabilities are treated equal and not traced back to business risk, which is contextual to

YOUR organization

– Many vulnerabilities can be mitigated with compensating controls… important to trace it back to business risk

• Mature organizations aware of effectiveness of standards– Enables audits and understanding of the threats against an organization– Improve security, architecture and coding practices

Significant Disconnect Between Execs and Practitioners• Cisco* report indicates that top product categories exploited were Applications

(32.6%) and Infrastructure (41.9%)– “We do a good job of building security into systems and applications”

– 90% of the SecOps people agree or strongly agree with that statement– 96% of the CISOs agree or strongly agree with that statement

• Ponemon Application Security Maturity Study

There are ample resources to ensure all IT security requirements are accomplished

IT security can hire and retain knowledgeable and experienced security practitioners

The IT security leader is a member of the executive team

IT security responds quickly to new challenges and issues

The IT security function is able to prevent serious cyber attacks such as advanced persistent threats

Appropriate steps are taken to comply with the leading IT security standards

IT security strategy is fully aligned with the business strategy

Security & data protection policies are well-defined and fully understood by employees

Security technologies are adequate in protecting our information assets and IT infrastructure

Application security is a top priority in my organization

0% 10% 20% 30% 40% 50% 60% 70%

36%

40%

41%

42%

46%

48%

50%

53%

54%

58%

34%

35%

35%

31%

33%

41%

39%

37%

44%

38%

Developers Security

*Cisco 2015 Annual Security Report

Disconnect between Management and Practitioners

Exec (28) Director (105)

Mgr (135) Sup (83) Tech (226) Staff (65)0%

10%20%30%40%50%60%70%80% 75%

56% 54%42%

23%35%

Existence of defined secure architecture standards

Exec (28) Director (105)

Mgr (135) Sup (83) Tech (226) Staff (65)0%

20%

40%

60%

80% 71% 66%56%

47%

19% 20%

Your organization keeps training programs up to date for development teams

>3X opinion difference between Tech and Exec staff

>3X opinion difference between Tech and Exec staff

Aligning Management and Staff – implications

• Developers don’t always understand InfoSec and security policies– “Ensure applications are coded so as not to be susceptible to OWASP Top

10” what does this mean to a an objectiveC iOS developer?

• Policy may be “in place”, but lack of enforcements renders mandate invisible

• Management, security, engineers speak different languages– “Confidential data must be protected”

• Protected from what?• How do I protect it?

– Architecture guidance?– Coding standards? – Remediation specifics once vulnerabilities are found?

» e.g., user input sanitation…. how do I do that in ASP.NET 3.5?

No Formal AppSec Training Program

• Ponemon research: 19% of developers believe their organizations keep training programs up to date for development teams

• Mature organizations have application security training programs in place for their developers that focus on:

– Specific role-based responsibilities– Offensive and defensive tactics– Application security policies– Areas of vulnerability– Best practices and standards to be followed– Various platforms and languages they are developing in/on

• Forrester– “Effective developer education program can reduce vulnerabilities by ~25%”

• Veracode– “30% fix rate improvement compared to those that don’t have an elearning program

Robert Half IT Security Survey

54% of CIOs plan enhance IT Security training in 2015

*promising if this includes AppSec

Agenda

• Application Security Trends

• Emerging Challenges, Technologies & Threats

• Threat Modeling & Risk Rating Your Applications

• Optimization Your Secure Software Development Lifecycle (SDLC)

Implications for Application Security

• IoT dependent on software – Software engineers not trained/confident regarding security

• IDC – By 2018, fully 75 percent of chief security officers (CSO) and chief information security

officers (CISOs) will report directly to the CEO, not the CIO

• Execs and practitioners still out of alignment – ~70% of executives (misinformed?) think their AppSec program is mature or very mature

compared to 20% of practitioners (the ones actually doing the work)

• Rise of nation state attacks – 74% of organizations rate their ability to recognize a nation state attack as poor or very

poor; yet, 81 percent are concerned or very concerned– 75% say their organization lacks expertise to detect or prevent a nation-state attack. This

points to a substantial lack of effective technology.– The “missiles and bombs” being used are software

• More regulatory and legal involvement

Threats Vary for Every Platform, Technology, Language

• Web, embedded, mobile attacked in different ways– You cannot implement effective design and coding

countermeasures without knowing these specific attacks– Adobe Flash vulnerabilities exploited numerous times– PHP is widely known as being insecure– Java frameworks littered with security flaws

• Applications written in web scripting languages have higher prevalence of SQL injection and Cross-Site Scripting than applications written in .NET or Java.

• Each Technology has different security features and vulnerabilities

• .NET and J2EE protect against buffer overflows but objectiveC does not

• Heartbleed, Shellshock, POODLE, GotoFail, Stuxnet– All software born exploits

Mobile Application Security

• Ponemon Institute– 2013 - 65% of developers do not test mobile applications– 2015 - 33% of companies still not testing their applications

• $34m spent developing mobile apps– 5% allocated to securing mobile apps– 50% of companies devote zero budget to mobile app security– 40% of companies surveyed are Fortune 500

• Gartner– Claimed in 2014 that more than 75 Percent of Mobile Applications will Fail Basic

Security Tests Through 2015– They likely got it right (see above)

• Veracode– Mobile applications have the highest rate of cryptographic issues, at 87 percent for

Android and 80 percent for iOS– Developers don’t understand how to properly implement crypto for the various

mobile platforms they are developing for

Why Are 90% of Attacks STILL at Application Layer?

Application Security 5%

More than 80% of IT security spending continues to be at the network layer, primarily focused on perimeter security

Source Information Security – Wave 17

Applications Will Continue to be Targets

• Department of Homeland Security (DHS)– “90 percent of security incidents result from exploits against defects in software”

• Tim Clark, Head of Brand Journalism at SAP, in a recent Forbes blog– “Many organizations have significant network security in place but it’s not enough as 84 percent

of all cyber-attacks are happening on the application layer”

• Cisco 2015 Annual Security Report– “Intruders are increasingly targeting the application stack for exploitation”– “Rise of cloud apps and the ubiquity of do-it-yourself (DIY) open-source content management

systems (CMS) has created vulnerable websites and SaaS offerings. – Underlying systems/networking layers may withstand malicious attacks, but application-level

components built by developers are often riddled with vulnerabilities.”

• CNET– “Programmers are copying security flaws in to your software– “Programmers don’t write all of their code. They routinely borrow code from others, and

they’re not checking the code for security flaws.”

• Wall Street– “Most developers have not been trained on secure coding practices”

Financial Services & Retail Under Attack

Sources: Imperva Web APPLICATION ATTACK REPORT #5 and CAST’s Biennial CRASH Report (2015)

• Poor code quality exposed more than two-thirds of financial services applications to cyber-attacks similar to the Heartbleed bug

• 70% of retail and 69% of financial services applications featured data input validation violations, a form of coding error central to the Heartbleed bug

– There are known, best practices for sanitizing input

• Financial services industry has highest number of input validation violations per application (224)

Industry Regulation - NY Dept of Financial Services

• Conducted risk assessments and found more needs to be done to secure financial systems

• Highlights of proposed mandates– Implement and maintain written cyber security policies– Designate a CISO and submit annual report assessing cybersecurity program– Application Security: maintain and implement written procedures,

guidelines, and standards reasonably designed to ensure the security of all applications utilized by the entity.

– Mandatory training to cyber security personnel and require key personnel to stay abreast of changing threats and countermeasures

– Conduct annual pen testing and quarterly vulnerability assessments

Bottom Line: Rollout AppSec program and you’ll meet most industry and compliance regulatory mandates

Agenda

• Application Security Trends

• Emerging Challenges, Technologies & Threats

• Threat Modeling & Risk Rating Your Applications

• Optimization Your Secure Software Development Lifecycle (SDLC)

Managing Risk – Tracing Vulnerabilities to Exposure

• Vulnerability - a weakness in some aspect/feature of system that makes an exploit possible

• Threat - undesired or negative occurrence that might damage or compromise an asset or objective. May/may not be malicious in nature

• Attack/Exploit - an action or set of actions taken against a system that utilizes one or more vulnerabilities to realize a threat

• Probability - likelihood of a specific attack being realized. Takes into account discoverability and difficulty of exploitation

• Exposure - amount of damage to an organization if a specific threat is realized

• Risk - exposure and probability weighted ranking of a given threat

• Countermeasures - address vulnerabilities to reduce the probability of attacks or the impacts of threats. Do not directly address threats

Taking a Risk-Based Approach to Assessments

• Conventional approaches to application security not risk-based– Typically no more than automated vulnerability scanning that look for some

pre-determined set of common vulnerabilities – Frequently fail to address each application’s unique code-, system- and

workflow-level vulnerabilities– Provides little practical guidance on prioritizing defect remediation – Does not enable the creation of a roadmap to guide enterprise application

security posture improvements

• The majority of application security programs focus on:*– Automated security testing during development (41%)– Secure coding standards that are adhered to (32%)– A secure SDLC process improvement plan (30%)

*”The State of Application Security Maturity” – Ponemon Institue & Security Innovation

• Provides more leverage than any other security activity• Aligns technical and management teams on threat areas• Makes EVERY engineering activity more impactful• We threat model every day in our personal lives

– Apply the same approach to software applications

Threat

Mitigation

Vulnerability

Attacker

Threat Modeling

“To know your enemy, you must become your enemy.”

- Sun Tzu

Threat Modeling – Value Across the Whole IT Team

• Architects– Shape your application design to meet security objectives– Understand and prioritize assets to protect– Build security architecture to protect critical assets

• Developers– Build countermeasures based on key threats– Use secure coding best practices to protect against attack

• Testers– Drive test plans to ensure attacks focus on high threat areas

• IT/DevOps– Conduct a deployment review against the threat model to ensure server

configurations are appropriates– Threat Model a portfolio of applicaitons

Application Risk Rating

• Helps Ensure:– Assessment and mitigation activities are done cost effectively– Prioritization is based on real business risk– The business doesn't get distracted by minor risks while

ignoring more serious risks that are less well understood

• Inappropriate security assessments are costly– Deep inspection on all applications is neither feasible nor necessary– Running an automated scan on critical application will lead to trouble

• Allows you to understand risk-based options– Remove, replace, take off-line, or implement compensating controls

Business Criticality is driving factor when determining which applications to secure and level of regular assessment needed

Framework for Application Security Risk Rating

• Risk = Likelihood * Impact

• Remember that threats can be inherited from system dependencies and connectivity

– Attackers can leverage non-critical apps to get to critical apps

• There is no standard formula – Risk tolerance and data classification are contextual to each organization

• Make sure risk-rating framework is:– Transparent so decisions and calculations can be easily explained– Adaptable so each group can apply unique drivers, goals, resources – Practical so you end up with something that works

Defining Tiers: 3 Tier Approach

* Customer-facing applications would include internet-facing applications as well as applications that reside on mobile or in-home devices

• Assign a score for each category, e.g., 0 to 3– This gives you a scale of 0 to 12 for each application using the 4 categories above

• Define risk thresholds that qualify an application to fall into a tier– Score of 0-3 is Tier 3– 4-8 is Tier 2– 9 or higher is Tier 1

Aligning Test Efforts with Application Risk

Security Testing Depth and Frequency

Threat Rating

Static Analysis (Source Code)

Dynamic Analysis (Live Application)

Manual (Penetration) Testing

Threat Modeling

Complete Frequency Complete Frequency Complete Frequency Complete Frequency

Tier 1 (critical)

Required Major code changes

Required Major code changes

Required Per-Milestone

Required Per-Release

Tier 2 (high)

Suggested Monthly Required Quarterly Required Per-Release

Suggested Per-Release

Tier 3 (low)

Optional Quarterly Required Annually Optional As-Needed Optional As Needed

• Calibrate the frequency and depth of testing according to risk rating– Also contextual to your organization– Driven by your risk tolerance and disaster recovery plans

Remediation Prioritization

• Determine what to fix – Are there compensating controls that can be implemented to mitigate?– Can your organization live with the risk?– Some loss is to be expected, justifiable based upon remediation cost

• The model should evolve over time– Choose factors that represent what's important for your organization

• military application might add impact factors related to loss of human life • window of opportunity for an attacker or encryption algorithm strength

– Weight factors to emphasize more significance for business operations– Fine-tune the model by matching it against risk ratings that the business agrees

are accurate

Agenda

• Application Security Trends

• Emerging Challenges, Technologies & Threats

• Threat Modeling & Risk Rating Your Applications

• Optimization Your Secure Software Development Lifecycle (SDLC)

Security Engineering Doesn’t Require Changing Your Process

Just augment it with a set of high-impact security activities

Different Activities Yield Different Values

• Threat Modeling – Ensures key threats are considered during coding

and testing, but is only a framework

• Code Reviews – Critical activity to prevent vulnerabilities from propagating but doesn’t consider

the as-deployed state or flaws introduced by environment

– Often more important than

• Manual penetration testing – Finds the most elusive and deeply rooted vulnerabilities, but is time

consuming and requires skill

• Scanner find common vulnerabilities faster than humans but– Can’t detect business logic and compound vulnerabilities– Are prone to time-wasting false positives

1. Review Org Structure and Team Roles

2. Analyze Policies & Standards Requirements

3. Analyze &Aggregate Data 4. Refine via Focused Interviews

(usually team leads)

5. Create Gap Analysis Reportwith Recommendations

SDLC Process Assessment – Graphical View

Best Practices

Five Maturity Levels from SI-Ponemon Research

• Level 1 (Initial)– Lack of discipline and maturity in the SDLC; no focus on security

• Level 2 (Repeatable)– Disciplined and repeatable SDLC but security is purely reactive– Security is addressed by patching issues based on penetration testing results

and public incidents

• Level 3 (Defined)– Security in the SDLC is standardized and defined– Corporate application security policies are defined– Formal security requirements are defined during development process– Secure coding standards are in place and the code is reviewed to these

standards– Security testing is part of the normal testing cycle

“The State of Application Security” https://www.securityinnovation.com/uploads/ponemon-state-of-application-security-maturity.pdf

Five Maturity Levels – Cont’d

• Level 4 (Managed)– Predictable process that is measurable and managed

– Threat modeling used to assess and prioritize risk in each phase

– Secure architecture standards in place and design is reviewed to these standards

– Development teams (and the code they produce) are measured against compliance security requirements as well as secure architecture and coding standards

– Application security risk is measured and well understood across the application portfolio

– Third party security audits are conducted for all high-risk applications

• Level 5 (Optimized)– Continuously improving process that is mature and optimizing

– Risk metrics are used to guide application security decision making

– Vulnerability discoveries are used to update requirements and standards

– Security activities are analyzed and improved based on assessments of vulnerabilities in the code

Rolling out a Secure SDLC

• A mature SDLC has formal requirements, designs, implementation and testing procedures in place

– Mature organizations also have security procedures defined at each phase– Organizations are not adequately emphasizing process, let alone security,

during development

• View security as yet another aspect of software quality

– Treat vulnerabilities as bugs• just a different kind of one

– Integrate security bugs with your defect triage and managementprocess

Application Security Policy – Key Components

• Describes contextual, technical guidelines on how security should be integrated within the SDLC

– By phase, role, technology, application type– Leverages internal secure development champion(s) for input

• Maps to compliance mandates

• Considers criticality of application and data– Requirements, activities and level of detail needed will differ

• Has clear exception policy– What if minimum standards can’t be met? What is considered acceptable? Who

approves?

• Includes internally built and third party applications

• Reflects current maturity and secure development skills– The more skilled, the less explicit you need to be with policies

Secure, Repeatable Development Works: the Microsoft SDL

• Major Challenges:• Needed to roll out the Microsoft Security Development

Lifecycle (SDL) to hundreds of development teams• Internal instructor-led training was effective, but not scalable and couldn’t be

re-purposed for new employees• Needed a way to train vendors on the Microsoft SDL to ensure software

consumed by Microsoft had security considered

• Security Innovation Solution: • Customized 14 eLearning courses specific to the Microsoft SDL

• Same content base as current courses in our eLearning library

In 24 months, Microsoft was able to go from having 30% of its product teams trained on the SDL to 70% (over 3,000 users)

Investing in Your SDLC Works!

Consistent application of sound security practices during all phases of development will facilitate compliance and result in fewer vulnerabilities

http://www.microsoft.com/security/sdl/about/benefits.aspx

Best Practices for Enterprise Assessments

• Use automated tools for heavy lifting– Find known vulnerabilities faster than humans– Adopt when you have the skills to use properly– Be sure tools are integrated into SDLC and used

at key checkpoints

• Complement with manual efforts– Necessary to find deeply rooted, business logic, and other vulnerabilities

that tools can not find– Be sure to leverage a threat model to focus on high-risk areas

• Support vulnerability remediation– Problem isn’t solved when found, only when corrected properly– Match test efforts with your organization’s ability to remediate

• Measure effectiveness

The Application Security Conundrum

Secure at the Source Protect in Play Assess/Test

InfoSec Standards Secure Coding Standards Secure SDLC Training

Web ApplicationFirewalls

Application Whitelisting RASP DLP

Vulnerability Scanning Penetration Testing Manual or Automated Code or in Production

These strategies should not be mutually exclusive – leveraging all three will have an exponential effect in risk reduction

Conclusion

• Application risk rating is a critical step in managing security efforts across a large number of applications

– Needs to be contextual to your organization's risk tolerance

• Be sure tools are integrated into SDLC and used at key checkpoints

• Build the skills needed to get more out of toolsand find vulnerabilities that tools can not

• Leverage a threat model throughout your SDLC

– Match test efforts with application criticality and your ability to remediate

• Measure effectiveness

Who is Security Innovation & How Can We Help?

• Pioneer in Application Security– 15+ years research on threats and vulnerabilities

– Security testing methodology adopted by SAP, Symantec, Microsoft, and McAfee

– Authors of 18 books, 10 with Microsoft

• Help clients navigate application security risk

ASSESSMENT - Show me the Gaps!

STANDARDS - Set goals and make it easy

EDUCATION - Enable me to make the right decisions

Assess Your Application Security Maturity

• Defined an effective software development process?

• Are automated tools integrated into your SDLC?

• Defined a set of corporate application security policies?

• Defined secure coding and architecture standards?

• Measuring testing applications against compliance requirements?

Find Out How You Stack Up!

http://securityinnovation.com/uploads/asm-questionnaire/start.html

Questions?

Thank You!

• All attendees will receive a recording of today’s session

• Email us at marketing@securityinnovation.com for a copy of the presentation

• All On-Demand Webinars can be found at www.securityinnovation.com/security-lab/webcasts

Connect With Us!