Test-Driven Security

Louis Nyffenegger @snyff

About Me

• Security consultant working for Securus Global in Melbourne

• 2 side projects:

– PentesterLab (.com): cool (web) training

– PNTSTR (.com): easy first round for interviews

• And today… I’m going to talk about Secure Development… in a way ;)

Too often when people talk about

secure development they explain…

Security?

How most people do it…

Security?

How you should do it…

Agile??

Agile software development is a group of software development methods based on iterative and incremental

development, where requirements and solutions evolve through collaboration between self-organizing, cross-

functional teams. It promotes adaptive planning, evolutionary development and delivery, a time-boxed iterative approach, and encourages rapid and flexible response to change. It is a conceptual framework that

promotes foreseen interactions throughout the development cycle.

Agile

• Projects evolved with clients’ needs, not based on project managers’ fantasy ;)

• No formal list of functionality

• New code is push to production “all the time”

– Etsy: 20 times a day

• No predefined milestones

TL; DR;

WHAT???

But how can people deploy all the time? four-leafed clover and rabbit's foot on each

production servers

Magic

Super awesome developers who don’t do any mistakes

Coverage of everything using tests and all tests are run before every push to production

But how can people deploy all the time? four-leafed clover and rabbit's foot on each

production servers

Magic

Super awesome developers who don’t do any mistakes

Coverage of everything using tests and all tests are run before every push to production

def test_can_see_exercises

get "/exercises"

assert last_response.status == 200

end

def test_can_access_login

get "/login“

assert last_response.body =~ /login/

assert last_response.body =~ /password/

assert last_response.body =~ /email/

end

Example of tests

• Some people even create test libraries that use plain English:

• And a developer writes the logic behind each line

Scenario: Regular numbers

* I have entered 3 into the calculator

* I have entered 2 into the calculator

* I press divide

* the result should be 1.5 on the screen/

Given /I have entered (\d+) into the calculator/ do |n|

@calc.push n.to_i

end

Example of tests… more

It can even be FUN

• Everyone can write test cases

• When a bug is found, a dedicated test is written…

-> A bug can only appears once

• New code can be deployed really quick

• All test cases written will be checked before each push to production

Summary

As a security person, I can only say one thing

10 points for Gryffindor

• Test-Driven Security

• Create security champion

• Get other people to write test cases

• Pair programming/Peer review

• Continuous integration

Back to security… agenda()

• A lot of security related functions are tested:

• A user can log in ?

• A user can change his password?

• A user can access his profile

• But I never, ever see things like:

• A user can’t log in with an invalid password

• A user can’t log in with an empty password

• A user can’t log in without password

• A user can’t access other users’ profile

Current test cases

def login(user,password)

creds = { :email => user,

:password => password }

post("/login", creds)

end

def assert_redirect_to_login

assert last_response.header["Location"] =~ /\/login$/

assert last_response.status == 302

end

Functions needed

def test_cannot_secret_without_login

get "/secret"

assert_redirect_to_login

end

def test_cannot_login_with_blank_password

login("louis@pentesterlab.com", "")

assert_redirect_to_login

end

Functions needed

def test_cannot_login_with_wrong_password

login("louis@pentesterlab.com", "wrong")

assert_redirect_to_login

End

def test_logout_on_access_other_users_stuff

login("louis@pentesterlab.com", “password")

get "/other_users_stuff"

assert_redirect_to_login

End

Functions needed

It’s pretty simple and straightforward, but not many

people are doing it :/

You can even go further… and create more security checks

• When I put a single quote in a field

• Do I get an error

• If it’s echoed back in the page, is it encoded?

• Same for ‘<‘

• Same for ‘>’

• Same for ‘”’

• If the application uses files, what happens if I put “../” in the file path

More test cases

But to do that you need developers with security training…

Not necessarily, Half of the test cases should be

based on business logic… Modern frameworks take care of

the other half. But it’s always good to have some

security champions.

• Steps:

• Take a developer

• Teach him everything about security: Top 10, Detection, Exploitation, …

• Put him back in the development team

• Pros:

• You have now a good security person

• Cons:

• Likely to go away to do pentesting

FIRST RECIPE

• Steps:

• Take a developer

• Teach him how to detect potential bugs

• Put him back in the development team

• Pro:

• You don’t have a wannabe hacker in your team

• You have someone who can find and fix bugs quickly

• Cons:

• The training was probably less interesting

SECOND RECIPE

• Forget everything you know about security

• Aside from business logic bugs… most security issues are based on: “Breaking the syntax”

• XSS: breaking JS or HTML syntax

• Code injection: breaking code syntax

• SQL injection: breaking SQL syntax

• …

• You just need to explain that correctly

Detecting potential bugs?

• Project managers:

• They are close to the business

• They can now write test cases in plain English

• Security people:

• Most of them should be able to write test cases

• They understand security

• Every time a bug is found they can write a test case to make sure it will never happen again

Get non-devs involved

• Perform sensibility training when the project starts:

• To avoid things like SQL built on the client side

• Introduction to test driven security

• Architecture review (SSL, Session mgmt…)

• If you perform penetration test, write issues as new test cases…

• Get a security person to review the “security test cases”

• Get a project manager to review the “business logic” security checks

As a process…

• Pair programming and security:

• junior/senior team

• dev/security team

• Peer review and security:

• Bug spotted earlier

• With modern versioning system (ie: git > 1.7.9), you can even sign commits:

Peer review

• You can automatically setup code review tools to scan your application

• You can automatically setup (free) web scanners to scan your application

• Cons:

• Lot of time spent setting that up

• Need to filter all possible false positive

• Pros:

• Sleep like a baby

Continuous integration

Good news

• Production vs Testing

• You can’t prevent things like:

• Weak crypto

• Weak PRNG

• Cookies related issues (“user=admin”)

• Or can you?

• More testing…

• This is when security people should start writing test cases.

Limitations

• No rocket science here…

… Just simple things to test

• If your developers don’t use tests… I guess you have other problems than security to take care of :/

• Reliable and simple way to increase your applications’ security

Conclusion

Questions?