Lecture 1nmbnmb

ECON3209: LECTURE 1INTRODUCTION TO PROBABILITY

Professor Alan Woodland

School of EconomicsUNSW

Professor Alan Woodland (ECON3209 ) Probability Theory cSchool of Economics, UNSW 1 / 28

INTRODUCTION TO PROBABILITY

The theory of probability is often attributed to Pascal and Fermat ( 1650).

It is remarkable that a science which began with consideration of games ofchance should have become the most important object of human knowledge....Themost important questions of life are indeed, for the most part, really only problemsof probability.... The theory of probabilities is at bottom nothing but common sensereduced to calculus. Laplace, Theorie Analytique des Probabilites, 1820


FATE LAUGHS AT PROBABILITIES.. BULWER

It is 1941 and the Germans are bombing Moscow. Most people in Moscow flee tothe underground bomb shelters at night, except for a famous Russian statisticianwho tells a friend that he is going to sleep in his own bed, saying that There isonly one of me, among five million other people in Moscow. What are the chancesIll get hit?

He survives the first night, but the next evening he shows up at the shelter. Hisfriend asks why he has changed his mind. Well, says the statistician, there arefive million people in this city, and one elephant in the Moscow Zoo. Last night,THEY GOT THE ELEPHANT!


DO WE NEED PROBABILITY THEORY?

Probability theory and statistics are useful in dealing with the following types ofsituations:

Games of chance: throwing dice, shuffling cards, drawing balls out of urns.

Quality control in production: you take a sample from a shipment, count how manydefectives.

Actuarial Problems: the length of life anticipated for a person who has just appliedfor life insurance.

Scientific Eperiments: you count the number of mice which contract cancer whena group of mice is exposed to cigarette smoke.

Quantum mechanics: the proportion of photons absorbed by a polarization filter

Statistical mechanics: the velocity distribution of molecules in a gas at a givenpressure and temperature.

Economics: the probability of future wage rate outcomes (high, low, medium) overan individuals lifetime.

Econometrics: the probability distribution for disturbances in a regression model;the probability of correct forecasts of a recession.


PROBABILITY: CONCEPT AND DEFINITIONS

Probability Theory is a mathematical representation of random phenomena. Aprobability is a numerical measurement of uncertainty.

DEFINITION (CLASSICAL)If there are N equally likely possibilities, of which one must occur and n areregarded as favorable, or as success, then the probability of a success is givenby the ratio nN .

Example: A box contains 4 blue balls and 6 red balls. Randomly pick a ball fromthe box, what is the probability it is blue?

Caveat: ideal situation with very limited scope

DEFINITION (RELATIVE FREQUENCY)Probability of an event is the proportion of the time that events of the same kindwill occur in the long run.

Repeat a game a large number of times under the same conditions. Theprobability of winning is approximately equal to the number of wins in the repeatedtrials.

Consistent with Classical approach


PROBABILITY: CONCEPT AND DEFINITIONS

DEFINITION (SUBJECTIVE APPROACH)A degree of belief in a proposition.

Subjective evaluations.

Educated guess.

DEFINITION (AXIOMATIC APPROACH)Kolmogorov developed the first rigorous approach to probability ( 1930). He builtup probability theory from a few fundamental axioms.

Probabilities are mathematical objects that behave according to well-defined rules.Any of the previous concepts can then be used in applications, as long as theyobey the rules.


SAMPLE SPACE AND EVENTS

DEFINITIONWith every situation (random experiment) with uncertain outcome we associate asample space, noted S, which represents the set of all possible outcomes(described by the characteristics which we are interested in)

Each outcome is an element or sample point of the sample space.

Discrete sample space: contains a finite number of elements, or an infinite butcountable number of outcomes.

Continuous sample space: elements are not countable. Eg. measurements ofphysical properties (time,pressure,length..)

Events are associated with subsets of the sample space.

combinations of two or more events are also events

from given observable events we can derive new observable events byset theoretical operations.


SET THEORETICAL OPERATIONS

There are two ways to to denote a set: either by giving a rule, or by listing theelements. Here are the formal definitions of set theoretic operations. Letters A andB, etc denote events from the same sample space S, and I is an arbitrary index set. xstands for an element and x A means that x is an element of A:

1 A B {x : x A x B}: Containment.2 A B = {x : x A and x B}: Intersection.3 iIAi = {x : x Ai for all i I}.4 A B = {x : x A or x B}: Union.5 iIAi ={x : there exists an i such that x Ai}.6 AC = {x : x / A but x S} (Event Complement)

BASIC LAWSAssociative Law: A B = B A and A B = B ADistributive Law:(A B) C = A (B C) and (A B) C = A (B C)DeMorgans Law: (A B)C = AC BC and (A B)C = AC BC


PROOF OF DEMORGANS LAW


AXIOMS OF PROBABILITY

Consider an experiment whose sample space is S and let E be an event in the samplespace S.

Probabilities are values of a set function.

Probability measure: assigns real numbers to the subsets of sample space S.For each event E of the sample space S, we assume that a number P(E) is definedand satisfies the following three axioms:

AXIOMS OF PROBABILITY1 0 P(E) 1.2 P(S) = 1.3 For any sequence of n mutually exclusive events E1,E2, ...,En:

P (ni=1) =n

i=1 P(Ei )

PROPOSITIONS1 P

(EC)

= 1 P (E).2 If E F , then P(E) P(F ).3 P(E F ) = P(E) + P(F ) P(E F ).


COUNTING RULES FOR PROBABILITY

We will be working in a finite probability space in which all simple events have equalprobabilities. To compute the probability of a certain event, one must count theelements of the set which this event represents. In other words, we count the numberof different ways to achieve a certain outcome.

SUM RULEIf a first task can be done in m ways and a second task can be done in n ways, and ifthese tasks cannot be done at the same time, then there are n + m ways to do eithertask. Example: A library has 10 stat textbooks and 15 biology textbooks. How manytextbooks can a student choose from if he is interested in learning about either stat orbiology?

EXAMPLEHow many textbooks can a student choose from if he is interested in learning aboutstat and biology?

MULTIPLICATION PRINCIPLEIf two successive choices are to be made with n1 ways in the first stage, and n2 ways inthe second stage, then the total number of successive choices is n1 n1.


COUNTING RULES FOR PROBABILITY

EXAMPLEYou throw two dice.Let A = sum of numbers shown is five or lessLet B = both of the numbers shown are five or less

S:11 12 13 14 15 1621 22 23 24 25 2631 32 33 34 35 3641 42 43 44 45 4651 52 53 54 55 5661 62 63 64 65 66

6 6 = 36 possibleoutcomes.

A =11 12 13 1421 22 2331 3241

,

P(A) = 1036 .

B =11 12 13 14 1521 22 23 24 2531 32 33 34 3541 42 43 44 4551 52 53 54 55

,

P(B) = 2536 .

GENERALIZED PRODUCT PRINCIPLE

If r successive choices are to be made with exactly nj at each stage j , then the totalnumber of outcomes is n1 n1 ... nr = rj=1nj .

EXAMPLEYou throw two dice and you flip two coins. What is the sample space?Use Tree diagram.


COUNTING RULES FOR PROBABILITY - SOLUTION FOR EXAMPLE


PERMUTATIONS AND COMBINATIONS

DEFINITION (PERMUTATION)A permutation of a set is its arrangement in a certain order (an ordered arrangement).The number of permutations is the number of ways a set can be written withoutrepeating its its elements. From the multiplication principle, the number of permutationsof a set of n elements is n (n 1) (n 2)... 2 1 = n! (n factorial). By definition,0! = 1.

DEFINITION (ORDERED K-TUPLETS)If one does not arrange the whole set, but interested in the number of k-tuples made upof distinct elements of the set of n, then the number of possibilities isn (n 1) (n 2)... (n k + 2) (n k + 1) = n!

(nk)! .We use the notation Pn,k for the number of permutations of n elements taken k at atime.



EXAMPLE (THE BIRTHDAY PROBLEM, DG P.30)Determine the probability p, that at least two people in a group of k people(2 k 365) will have the same birthday, that is, will have been born on the same dayof the same month but not necessarily in the same year. (Can you make a guess fork = 30)?


PERMUTATIONS AND COMBINATIONS - BIRTHDAY PROBLEM SOLUTION


PERMUTATIONS AND COMBINATIONSSuppose that the order of the elements in the k-tuples is not important, how manyk -element (1 k n) subset does a set of n distinct element have?RULE OF COMBINATIONSThe number of k -element subsets of an n-element set is :Cn,k =

Pn,kk! = (

nk ) ==

n!(nk)!k! . It is pronounced as: n choose k .

THEOREM (BINOMIAL COEFFICIENTS)For all numbers x and y and each positive integer n,

(x + y)n =n

k=0

Cn,k xk ynk

PASCAL TRIANGLE

11 1

1 2 11 3 3 1

1 4 6 4 11 5 10 10 5 1

=

(00)(10) (

11)

(20) (21) (

22)

(30) (31) (

32) (

33)

(40) (41) (

42) (

43) (

44)

(50) (51) (

52) (

53) (

54) (

55)



DEFINITION (MULTINOMIAL COEFFICIENTS)The multinomial coefficient is:

Cn,n1,n2,...,nk =( n

n1, n2, ..., nk

)=

n!n1!n2!...nk !

.

. This can be interpreted as the number of combinations of n objects divided into kgroups comprising n1, n2, ..., nk objects respectively.

THEOREM (MULTINOMIAL THEOREM)For all numbers x1, x2,..,xk and each positive integer n,

(x1 + x2 + ... + xk )n =( n

n1, n2, ... nk

)xn11 x

n22 ...x

nkk =

n!n1!n2!...nk !

xn11 xn22 ...x

nkk

where the summation extends over all possible combinations of nonnegative integersn1, n2, ...nk such that n1 + n2 + ... + nk = n


CONDITIONAL PROBABILITY AND INDEPENDENCE

DEFINITIONIf we know that the event B has already occurred, then we know that the outcome of theexperiment is one of those included in B. To evaluate the probability of an event A, wemust consider the set of those outcomes in B which also result in the occurrence of A.

P(A|B) = P(A B)P(B)

Draw a Venn Diagram

MULTIPLICATION RULE AND INDEPENDENCEThe multiplication rule is a result of the conditional probability rule:

P(A B) = P(A|B) P(B) = P(B|A) P(A).Two events A and B are independent if and only if

P(A|B) = P(A) and P(B|A) = P(B).Independence of A and B implies that P(A B) = P(A) P(B).

CAUTION:What is the difference between Independent events and Exclusive events?


INDEPENDENCE

EXAMPLEYou have an urn with 5 white and 5 red balls. You take 2 balls without replacement. A isthe event that the first ball is white, and B that the second ball is white.

What is the probability that the first ball is white?

What is the probability that the second ball is white?

Are these two events independent?

Are these two events disjoint?

What is the probability that both balls have the same color?

EXAMPLE (EXAMPLE SOLUTION)


BAYES THEOREM

THEOREMIn its simplest form Bayes theorem reads:

P(A|B) = P(B A)P(B|A)P(A) + P(B|AC)P(AC)

THEOREM (GENERALIZATION OF BAYESSFORMULA)If B1,B2, ...,Bk constitute a partition of thesample space S and P(Bi ) 6= 0, fori = 1, 2, ..., k; then for any event A in S suchthat P(A 6= 0),

P(Bj |A) = P(Bj A)ki=1 P(A|Bi )P(Bi )


BAYES THEOREM

EXAMPLEAIDS diagnostic tests are usually over 99.9% accurate on those who do not have AIDS(i.e.,only 0.1% false positives) , and 100% accurate on those who have AIDS (i.e.,nofalse negatives at all). A test is called positive if it indicates that the subject has AIDS.Assuming that 0.5% of the population actually have AIDS, compute the probability thata particular individual has AIDS, given that he or she has tested positive.

EXAMPLE (ANSWER)Advice: Never trust your intuition. Always follow the ruleLet A be the event that the subject has AIDS, and let P be the event that the subjecttested positive. The information given to us is the following:

P(A) = 0.5% = 0.005 P(AC) = 1 0.005 = 0.995P(P|A) = 100% = 1P(P|AC) = 0.1% = 0.001We want P(A|P)P(A|P) = P(P|A)P(A)

P(P|A)P(A)+P(P|AC )P(AC )

P(A|P) = 10.00510.005+0.0010.995 = 50005995 = 0.8340.


FUN TIME

Three roommates slept through their midterm statistics exam on Monday morning.Since they had returned together by car from the same hometown late Sundayevening, they decided on a great little falsehood. The three met with the instructorMonday afternoon and told him that an ill-timed flat tyre had delayed their arrivaluntil noon.The instructor, while somewhat skeptical, agreed to give them amakeup exam on Tuesday.

When they arrived the instructor issued them the same makeup exam andushered each to a different classroom. The first student sat down and noticedimmediately the instructions indicated that the exam would be divided into Parts Iand II weighted 10% and 90% respectively. Thinking nothing of this disparity, heproceeded to answer the questions in Part I. These he found rather easy andmoved confidently to Part II on the next page. Suddenly his eyes grew large andhis face paled. Part II consisted of one short and pointed question.......

Which tyre was it?


FURTHER READING

A. DeGroot.Chapter 1, Chapter 2.

For special note:Ch.1 Sampling with Replacement.Ch.1 Sampling without Replacement.Ch.1 Blood Types, DG pp.34.Ch.1 Statistical Swindles, DG pp.51.Ch.2 Clinical Trial.Ch.2 The Game of Craps.

B. Miller and Miller.Chapter 1, Chapter 2.

For special note:Ch. 1 Combinatorial Methods.Ch. 1 Binomial Coefficients.


Probability ConceptsSet TheoryAxioms of ProbabilityCounting Rules and Combinatorics Conditional Probability and Independence Bayes Theorem Fun Time Fun Time

Date post:	28-Sep-2015
Category:	Documents
Upload:	nathan-huang
View:	4 times
Download:	0 times

Lecture 1nmbnmb

Documents