Advanced PHP and MySQL

transcript

Some Adventures and Experiments

DIG 4104c – Spring 2013

J. M. Moshell

-2 - -2 -

Midterm Exam Results

Cumulative grades not available – not all presentations

(finish those today)

BUT – most projects & presentations 85% to 95%, "orbiting

around A/B", so the Midterm, Final & Project 3 are 75% of

overall score.

-3 - -3 -

Midterm Exam Results

Cumulative grades not available – not all presentations

(finish those today)

BUT – most projects & presentations 85% to 95%, "orbiting

around A/B", so the Midterm, Final & Project 3 are 75% of

overall score.

MTX: 225-250: Like an A. 200-224: Like a B. 150-199: C-ish

-4 - -4 -

The Rest of the Semester(by popular request:)

PHP and MySQL

SOAP and Web Services

Evaluating Web Services: Classroom Feedback Systems

Commercial Payment Systems & E-Commerce

Security Adventures and PCI

-5 - -5 -

Context:Registration Systems Lab

PHP and MySQL

SOAP and Web Services

Evaluating Web Services: Classroom Feedback Systems

Commercial Payment Systems & E-Commerce

Security Adventures and PCI

-6 - -6 -

PHP custom coded

registration system

MySQL database

(one per conference)

Uses several credit

card gateways

(client owned) as well as RSL's own authorize.net gateway

-7 - -7 -

25 to 30 conferences/year

We charge $9 to $14

per registrant

We had 26 conferences

in 2012

Employees:

Carole Mann, President

David Mann, IT Manager

Mandy Mann, Conference Manager

+2 ladies and one part time professor/designer

-8 - -8 -

Specialized feature:

multiple gateways

for one (complex)

conference.

Problem: Hackers are growing more sophisticated

PCI (Payment Card Industry) compliance – getting tougher

-9 - -9 -

SystemArchitecture

ICSE13core011

icse13MJAA13

mjaa13PLDI13

mjmembers

conf. code core code gateways

rslrslrslrsl

... etc

-10 - -10 -

System Manager: moma

moma ("Moshell's Manager) is built with Drupal

-11 - -11 -

System Manager: moma

(passwords obscured)

-12 - -12 -

Today's problem:Insider Attack

Assume a hacker wants to capture our clients' credit card info.

Assume they're already inside our system, can modify code.

(We consider how to keep 'em out ... later)

What can we do to stop these bandits?

Idea 1: Don't keep cc info in the database.

- This is a basic rule for PCI* compliance -

Payment Card Industry Association

-13 - -13 -

Today's problem:Insider Attack

Idea 2: Develop a system to detect any changes to your code.

A kind of 'burglar alarm'.

Design constraints:

* must run whenever the code runs, to prevent use when contaminated.

* must not impact the system's functionality

== speed

== frequent interruptions of service

-14 - -14 -

Developing a Burglar Alarm

Attacking the Burglar Alarm Idea

1) What if bandito replaces 100% of your code?

* Must have a periodic external 'audit' to detect this ploy.

* Unless this audit runs frequently, SOME data will be lost.

2) What if the bandito scans your code and deactivates the alarm?

* Don't make this easy for them.

-15 - -15 -

Developing a Burglar Alarm

Some axioms of computer security:

1) Nothing is going to work ALL the time. You need layers.

2) Humans are the weakest point in the system. Automate it!

3) Security by Obscurity is a weak basis for a design. But you

must start somewhere.

== >> ACT NOW << ==

-16 - -16 -

Digital Signatures

Why don't we just make a duplicate

copy of the software, and compare for modifications?

ICSE13core011

ICSE13acore011a

= =? = =?

-17 - -17 -

Easy solutions that don't work

1) Comparing a hundred files over & over ... inefficient

2) The bandit could simply modify BOTH copies

ICSE13core011

ICSE13acore011a

= =? = =?

-18 - -18 -

What about a signature?

?? Can we design a unique shadow of some kind

which is (a) fast to compute, (b) unique? (c) informative?

ICSE13core011

ICSE13a

signature

core011a

-19 - -19 -

What about a signature?

?? Can we design a unique shadow of some kind

which is (a) fast to compute, (b) unique? (c) informative?

Fast: Something built into PHP's system, not a PHP loop

across 100+ files, 150,000 lines of code.

Unique: every different code-set has a different shadow.

Informative: if shadow1 != shadow2,

what does that tell us?

-20 - -20 -

Comes the HASH CODE:

A hash code is produced by an algorithm.

Input: a body of data (e. g. a text file.)

Output: a big integer or string.

Properties:

1) Same input tomorrow yields same output.

2) Different inputs are very unlikely to yield same output.

3) Process is not reversible.

-21 - -21 -

Really dumb HASH CODE:

Take in all the characters, convert to numbers,

add 'em up. Throw away high order digits.

This is a text for which we want the hash code.

etc...

---------

453664 now the 4 digit hash is 3664.

Change any text letter ... hashcode (probably) changes.

-22 - -22 -

Really smart HASH CODE:

sha1 is a hash algorithm built into PHP

- widely used for cryptographic purposes

- used for creating unique keys in git

- input: any file of up to 2^64 bits (a LARGE number)

- it's quite fast, because its widely used & needed

- Produces something like this:

4b5437055d8adaeb9b47c7dfda18f400907cc146

-23 - -23 -

The architectural concept of the Alarm:

ICSE13

self-check

core011

self-check

ICSE13a

signaturecore011a

signature

First line of defense: self-checking against a stored signature.

(Hidden, somewhere in our file hierarchy)

ICSE13a

signaturecore011a

signatureHidden signature files

-24 - -24 -

The architectural concept of the Alarm:

ICSE13core011

ICSE13a

signaturecore011a

signature

Second line of defense: periodic audit checks

against signatures on a DIFFERENT computer

ICSE13a

signature

core011a

signature

Remote signature files

Remote

managerlocal

-25 - -25 -

Focus on the first line of defense:

How would you attack this system?

-26 - -26 -

How would you attack this system?

1) find the hidden

signature files

2) find the self-check

code in ICSE13

or in core011

-27 - -27 -

How would you attack this system? Why that's hard:

1) find the hidden * our system has 11 gb

signature files in 17,000 files

* filenames not known

2) find the self-check

code in ICSE13 * our system has

or in core011 lots of places to look

(and what are you

looking for?)

-28 - -28 -

Here's a partial list of the code modules:

and it's not going to be called "security scan" ... !

-29 - -29 -

A common tactic: Trigger an error message

and then search the code base for that error message.

* Defenses:

1) generate your error messages from a database

2) scramble the source code so it's unsearchable.

* But remember ... Security by Obscurity is a weak defense!

-30 - -30 -

Another common tactic: run image of code

Bandit copies our code, runs in his on WAMP environment.

Looks for file accesses, error messages if not found.

* Remedy: use file_exists to check for files,

only write to files already found.

* Turn off error messages, so no squawks if files not found.

* A VERY GOOD hacker will get you anyway, by hacking

PHP itself. But maybe we're too much trouble ....

-31 - -31 -

Designing our Burglar Alarm

Criterion 3:

Informative: if shadow1 != shadow2,

what does that tell us?

We want our signature to not only holler BURGLAR!

but to tell us which "room" he's in,

so that we can examine the attack.

-32 - -32 -

An idea: An XML signature

directory1

directory2

-33 - -33 -

An idea: An XML signature

directory1

directory2

<name>directory1</name>

<file>

</file><file>

</file>

</dir>

-34 - -34 -

Compare two signatures.

Where sha

don't match,

retrieve the filename

and report it.

<name>directory1</name>

<file>

</file><file>

</file>

</dir>

-35 - -35 -

So now I know what my tasks are.

1) read the directory structure

2) construct an XML representation, with sha for each file

3) construct a comparator that can report file with difference

4) build Level 1 (self-test) into a conference

(both for core and for conference-specific code)

5) build Level 2 (auditor test) into moma, across all conferences

-36 - -36 -

Step 1: Prototype directory reading

prototype code hmm1.php

Key PHP functions: YOUR job: understand, investigate or ASK!

You need to know WHAT it does, and WHY I used it.

$d = dir($path);

$entry = $d -> read();

file_exists($filepath);

$fs=filesize($filepath);

$fstuff=implode('',file($filepath))

$fsha=sha1($fstuff)

-37 - -37 -

Key programming techniques:

1) Show your results in detail (with <table>)

to make it easier to diagnose and debug

2) Recursion: dirget CALLS ITSELF!

3) Limiting recursion. Why do we exclude path '.' ?

-38 - -38 -

Key programming techniques:

1) Show your results in detail (with <table>)

to make it easier to diagnose and debug

2) Recursion: dirget CALLS ITSELF!

3) Limiting recursion. Why do we exclude path '.' ?

-39 - -39 -

PRACTICE PROBLEM #1

Note: There will be several Practice Problems through this

lecture. If you want an A on the final exam,

WORK MOST OR ALL OF THEM.

If you want to not get a demerit for next Monday's lecture,

WORK AT LEAST ONE OF THEM.

Your entire team can work the same one, as long as you

can demonstrate and explain your results.

-40 - -40 -

PRACTICE PROBLEM #1

Take the demo program hmm1.php and modify

it so that it simply prints out a nice looking, hierarchical listing

of the contents of the directory to which it is pointed.

example:

file1.php

file2.php

file3.php

file4.php

-41 - -41 -

Step 2: XML

Prototype hmmXML2.php

Goal: create an XML text file that stores the results of

the directory traverse from prototype 1.

Method: Find a working XML example, and "steal" elements

of it.

The example function 'xemit' is my "resource mine".

-42 - -42 -

Step 2: XML

Prototype hmmXML2.php

examine 'xemit'. Note how it wants an xml string as a 'seed'.

I discover that the example's XML string seed setup requires

a specific syntax (left over from VERY EARLY PHP.)

Analyze hmmXML2.php.

Identify the key new commands.

-43 - -43 -

PRACTICE PROBLEM #2

"Retrograde" Example hmmXML2.php

That is, make it write the file 'xout.html' from the

movie example, rather than from the directory system.

Note: at this point I'm using an old function 'textsaver'

that was designed to write out arrays of text. But I have

only one 'line' of text (i. e. one string variable) and so

I put it into an array cell, $text[0].

-44 - -44 -

Step 3: Read a stored file & compare

Skipping forward to prototype hmmXML6.php:

Read MAIN to see what's happening:

1. Load a file named xdata.xml (the previous scan.)

2. store this text in $xtext1.

3. Do the dirget magic to create new $xtext2.

4. Write this as the NEW xdata.xml file

5. Now we scan for a mismatch, using substr.

if no mismatch, print "no mismatch found"

else try to find the <file> tag and say WHERE!

-45 - -45 -

Step 4: Production Code

I have replaced critical information with xyz in the

fourth ("production") version, as it's embedded in live commercial code.

Demonstrate with localhost:icse13

control=xyz; then modify regsystem.php, try xyzcheck

then try control=regtest

examine the function 'unspooger'.

Discuss how vulnerable this code REALLY is ...

Dreamweaver can seek out the word 'correct' in <1 second.

Part 2: MySQL Extended Example

In 3134 we do 'toy' problems with small tables.

In RSL we have real-world

databases

(complex, but small)

Table structure:

In 3134 we do 'toy' problems with small tables.

Objectives of the system:

1) Flexibility: each conference has different data needs, but

we DO NOT want a unique database structure for each.

2) Historical record: We need to know all additions, deletions,

errors and corrections. This is accounting for big bucks.

So – we analyzed Drupal's table structure and stole (much of) it.

users:

attendee number, login ID, password (encrypted),

(We'll discuss 'salt' in the security lecture.)

transactions:

transaction ID, attendee number, date, time, worker

So a given user can have any number of transactions

Identified by 'tid' (transaction ID) an integer.

Part 2: MySQL Extended Examplea transaction tracks 4 kinds of information:

transtrings: Any data not financial, e. g. names, addresses.

tid, fieldname, fieldvalue (up to 50 characters)

trantexts: like transtring but can have text of ANY size

tid, fieldname, fieldvalue (any size)

Part 2: MySQL Extended Examplea transaction tracks 4 kinds of information:

transtrings: Any data not financial, e. g. names, addresses.

trantexts: like transtring but can have text of ANY size

tid, fieldname, fieldvalue (any size)

tranumbers: how many of something, the person buys

tid, fieldname, value, attendee type, paywhen, annotation

tranmoney: payments, refunds, balances due

tid, fieldname, amount, payclass, ..when, .. etc

Part 2: MySQL Extended Exampletranstrings: Any data not financial, e. g. names, addresses.

Example: Attendee 15001 – Joe Bloe

transaction: attnum=15001 tid = 1 Date = 3 Jan 13

transtrings:

tid fieldname fieldvalue

1 lastname Bloe

1 firstname Joe

1 address1 345 River Street

etc ... you can store INFINITE detail with simple structure.

Part 2: MySQL Extended Exampletranstrings: Later... change his address

Example: Attendee 15001 – Joe Bloe

transaction: attnum=15001 tid = 1 date=3 Jan 13

transaction: attnum=15001 tid = 192 date = 5 Mar 13

transtrings:

tid fieldname fieldvalue

1 lastname Bloe

1 firstname Joe

1 address1 345 River Street

192 address1 678 Elm Street

etc ... demonstrate history report.

Focus briefly on tranmoney:

tranmoney: payments, refunds, balances due

tid, fieldname, amount, payclass, paywhen, paytype,

cctransactionid, ccapprovalcode,ccexpdate, annotation

'fieldname': like moneybaldue, moneypayment

'payclass': which of the gateways was used

'paywhen': early, late, onsite

'paytype': visa, mc, american express, cash, check, etc.

'cctransactionid etc': codes to track back to the gateway

'annotation': worker can explain unusual situations here

Part 2: MySQL Extended ExampleThis structure lends itself to producing complex

reports ... e. g.

Part 2: MySQL Extended ExampleThis structure requires complex queries, e. g.

$q="SELECT fieldname, sum(fieldvalue),tranumbers.tid, transactions.attnum

FROM tranumbers,transactions,batchlinks

WHERE tranumbers.fieldname LIKE 't%'

AND tranumbers.tid=transactions.tid

AND transactions.attnum=batchlinks.attnum

AND batchlinks.batchname='$thisbatch'

GROUP BY fieldname";

Let's analyze this query:

$q="SELECT paytype, paywhen, date, SUM(amount), COUNT(amount)

FROM tranmoney ,transactions, batchlinks

WHERE fieldname='moneypayment$showpayclass'

AND tranmoney.tid=transactions.tid"

AND transactions.attnum=batchlinks.attnum

AND batchlinks.batchname='$thisbatch'

AND tranmoney.amount>0

GROUP by paytype, paywhen, date";

Typical questions: Why is a variable like $thisbatch in the middle

of the query $q?

Why is the term 'fieldname' not specified as 'tranmoney.fieldname'?

What is the effect of GROUP? Relation to COUNT?

How to gain experience with MySQL?

I don't want to make up a toy problem for you

or pull out a chunk of my working code

(like I have done with the 'security' activity)

So ... the best way to gain more experience with MySQL

is to attack a problem that you need to solve

take advantage of Adam and myself, while you

have us!

FOR NEXT WEEK:

1) The programming mini-projects set out above

2) Read

http://en.wikipedia.org/wiki/SOAPhttp://www.w3schools.com/webservices/ws_intro.asp

http://www.w3schools.com/soap/soap_intro.asp

3) Make up an imaginary SOAP service to provide some

simple information (such as your weight, today)

4) Compose an example message to your imaginary

service, to request the information it can provide.

Advanced PHP and MySQL

Documents