utdallas .edu /~metin Page 1 Outcomes, Counting, Measures and Probability Outline Experiments, Outcomes, Events Counting β Finite Outcomes Β» Number of distinct strings, sequences, subsets and multisets β Infinite but Countable Outcomes β Uncountable Outcomes Probability Prof. Metin ΓakanyΔ±ldΔ±rΔ±m used various resources to prepare this document for teaching/training. To use this in your own course/training, please obtain permission from Prof. ΓakanyΔ±ldΔ±rΔ±m. If you find any inaccuracies, please contact [email protected] for corrections. Updated in Fall 2019
Transcript
utda
llas.edu /~m
etinPage 1Outcomes, Counting, Measures and Probability
Outline Experiments, Outcomes, Events Counting
β Finite OutcomesΒ» Number of distinct strings, sequences, subsets and multisets
β Infinite but Countable Outcomesβ Uncountable Outcomes
Probability
Prof. Metin ΓakanyΔ±ldΔ±rΔ±m used various resources to prepare this document for teaching/training. To use this in your own course/training, please obtain permission from Prof. ΓakanyΔ±ldΔ±rΔ±m.
If you find any inaccuracies, please contact [email protected] for corrections.Updated in Fall 2019
utda
llas.edu /~m
etinPage 2Probability
Why Probability? β Limited information about experiments / events caused by β¦ β Information collection
Β» Costly: Surveying all customers, voters, Β» Inaccurate: Measuring devices for speed, humidity, Β» Impossible: Location of an electron around the nucleus, causes of interior tumors
β Some uncertainty remains and is incorporated into models Probability or Statistics? β Both and interactively.
Events: A={product is odd}, B={sum is odd}, C={product & sum are odd}, D={product & sum are even}
Product odd β Each odd β Sum even, hence product and sum are never odd togetherProduct either odd or even. Product odd β Sum even. Product even β Sum either odd or even
- Dice is plural of singular die. - But many use dice as singular.
If the speeds π£π£| & π£π£ππ can be adjusted by a well-calibrated thumb (or a machine under some ideal conditions), the outcome of the coin flip experiment is deterministic (and controllable).
Ang
ular
spe
ed
Time to the max height = Vertical speed/gravitational constant
Time to max heightis 1/4 second
Too long in the air, practically irrelevant
Usually time to max height is ΒΌ second, or π‘π‘ππππππ = 1/2 seconds
β π£π£πππ£π£|
ππβ 12ππ + 1
4,ππ + 3
4becomes
β π£π£ππ β 2ππ + 12
, 2ππ+ 32
yields π»π» β ππ
If a coin has ππ = 20 revolutionsβ π£π£ππ β 40.5,41.5 guarantees π»π» β ππβ But π£π£ππ β 39.5,40.5 βͺ (41.5,42.5)
yields π»π» β π»π»β Flip a coin with at most 2% variation
in angular speed?Β» Yes β the outcome is NOT random
utda
llas.edu /~m
etinPage 7Counting the Sample Space Counting is assigning natural numbers to objects
β Natural numbers 0, 1, 2, 3, 4, 5, β¦. are infinitely many. When we are assigning numbers to infinite objects, we are not going to run out of numbers as long as we can imagine a process (sequence, order) for assigning numbers.
A set is countable when its elements can be assigned to natural numbers.
Countable sets Uncountable setsFinite sets
Eg: Cars in β¦
Infinite butCountable setsEg: Natural numbers
Infinite andUncountable setsEg: ??
Ex: All sets with finite cardinality are countable. Eg, UTD students, leaves on a tree, hair strings on your head are all countable.
How many marbles are there in the jar? 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Multiplication principle: When the outcome of an experiment is defined by β K characteristics that are independent of each other, use the multiplication principle.β Start by enumerating the number ππππ of ways characteristic ππ β [1:πΎπΎ] can materialize. β Then the number of outcomes is ππ1ππ2 β¦πππΎπΎ .
Ex: An online dress retailer carries 3 styles of lady dresses: Night dress, Corporate dress and Sporty dress. Each style has 20 cuts, 8 sizes and 5 colors. A stock keeping unit (sku) for the online retailer is defined by the dress style, cut, size and color as these four characteristics fully describe the dress item and are used to buy dresses from the suppliers. The number of skus for this retailer is 3 Γ 20 Γ 8 Γ 5 = 2400.
If the characteristics are not all independent of each other, we can still use the multiplication principle with some adjustments.
Ex: After a market research study, the online dress retailer decides to customize its offerings. It offers 22 cuts of Night dresses, 18 cuts of Corporate dresses and 34 cuts of Sporty dresses. Night dresses need to fit more closely so they have 10 sizes, while Corporate and Sporty dresses have respectively 8 and 6 sizes. The number of skus become (22 Γ 10 + 18 Γ 8 + 34 Γ 6) Γ 5. The color is independent of other characteristics. Within each style, the cut and the size characteristics are independent.
Ex: How many of the four-digit integer numbers contain no repeated digits? β Digits are {0,1,2,3,β¦,8,9} but 0 cannot appear as the first digit in a four-digit integer. Because 0987 is not a
four-digit integer, it is a three-digit integer. 1. The first digit say ππ can take one of the nine values from {1,2,3,β¦,8,9}. 2. The second digit ππ can take one of the nine values from {0,1,2,3,β¦,8,9}\{ππ}. 3. The third digit ππ can take 8 values from from {0,1,2,3,β¦,8,9}\{ππ,ππ}. 4. The fourth digit ππ can take 7 values from from {0,1,2,3,β¦,8,9}\{ππ, ππ, ππ}. The answer is 9x9x8x7.
utda
llas.edu /~m
etinPage 9Finite Sets β 2. Permutations
Permutation of k objects out of n distinct objects is picking up k objects out of n objects and sequencing these k objects. Each sequence is a different permutation.
Objectn β¦β¦.Object
n-1Object
n-2Object
n-3Objectn-k+2
Objectn-k+1
k objects picked out of n
Ex: A professor picks 5 chapters out of 10 chapters of a textbook and sequences these 5 chapters arbitrarily to teach them in his course. How many different sequences of 5 chapters (syllabi) can be made from 10 chapters? ππ510.
The number of permutation of k objects out of n distinct objects is n(n-1)(n-2)β¦(n-k+1)=:ππππππ.
1: 2: 3: 4: 5: 6:
Ex: Two marbles are picked out of three marbles with colors green, blue and red. How many distinct sequences are possible with these two marbles: ππ23 =3(2).
Ex: A professor hands out 6 exams papers back to 6 students. What is the number of ways exam papers can be sequenced? ππ66.
β¦Objectn-k
Object1
Not picked
utda
llas.edu /~m
etinPage 10Finite Sets β 2. Permutations
Ex: A university network requires passwords that are strings made out as follows:β Lowercase letters {a,β¦,z} and uppercase letters {A,β¦,Z} and digits {0,1,2,β¦,9} can be used in a string. A total
of 52 distinct letters and 10 distinct digits.β The length of the string must be exactly 6. No letter or digit can be repeated.β At least 2 digits and at least 2 letters must be used in the string.
How many passwords are possible? [A]. If exactly 2 digits in the password and 4 letters. Number of digit strings is ππ210and number of letter strings ππ452.
6!/(4!2!)=15 total mergers of ππ210digit strings and ππ452 letter strings: 15ππ210 ππ452. [B]. If exactly 3 digits in the password and 3 letters. Number of digit strings is ππ310and number of letter strings ππ352.
6!/(3!3!)=20 total merges of ππ310digit strings and ππ352 letter strings: 20ππ310 ππ352. [C]. If exactly 4 digits in the password and 2 letters. Numbers of mergers is the same as in [A]. 15ππ410 ππ252. The total number of passwords: 15ππ210 ππ452+20ππ310 ππ352+15ππ410 ππ252.
utda
llas.edu /~m
etinPage 11Finite Sets β 3. Combinations
Combination of k objects out of n distinct objects is picking up k objects out of n objects without paying attention to sequencing of these k objects.
ππobjects β¦.ππ β 1
objectsππ β 2
objectsππ β 3
objectsππ β ππ + 2
objectsππ β ππ + 1
objects
k objects but their k! sequence does not matter
The number of combinations of k objects out of n distinct objects is ππππππ
ππ! =:πΆπΆππππ.β Sequence 1, 2, β¦, k! are all counted as one combination
Ex: A professor picks 5 chapters out of 10 chapters of a textbook to teach them in his course. How many different ways are there to pick 5 chapters from 10 textbook chapters? πΆπΆ510.
1: 1: 2: 2: 3: 3:
Ex: Two marbles are picked out of three marbles with colors green, blue and red. How many distinct sequences subsets are possible with these two marbles: πΆπΆ23 =3(2)/2=3.
The same subset The same subset The same subset
β¦Objectn-k
Object1
Not picked
The number of distinct subsets of size k that can be made from a set of size n is also πΆπΆππππ.
utda
llas.edu /~m
etinPage 12Finite Sets β 4. Combinations with Repeating πΆπΆππππ is the number of subsets. Each subset can have each element at most once, so elements
cannot be repeated.
If the elements are repeated in a collection, the collection is called a multiset. In a multiset,
β the total number of elements, including repetitions, is the cardinality of the multiset β the number of times an element appears is the multiplicity of that element.
Ex: Suppose that we are to pick colors B, R, G to create multisets of cardinality 3. β <B,R,G> is the unique multiset whose elements do not repeat, so <B,R,G > is also a set. β As repetition is allowed in a multiset, some elements can be used twice or thrice while the others are not used at
all. If G is not used, we can still construct multisets <B,R,R>, <B,B,R>, <B,B,B>, <R,R,R>. β If G must be used, we can construct some other multisets <B,G,G>, <R,G,G>, <G,R,R>, <G,B,B>, <G,G,G>. β The multiplicity of B in <B,R,R> is π₯π₯π΅π΅ = 1, while the multiplicity of B in <B,B,R> is π₯π₯π΅π΅ = 2.
From members of a cardinality-ππ underlying set, how many cardinality k multisets can we create? Each multiset is uniquely identified by the multiplicities {π₯π₯1,π₯π₯2, β¦ ,π₯π₯ππ} of members of its
underlying set. β Since we set the cardinality equal to k, we need to insist on π₯π₯1 + π₯π₯2 + β―+ π₯π₯ππ = ππ. β Also each multiplicity must be a natural number (non-negative integer). β The number of multisets with cardinality k = The number of natural number solutions to
etinPage 13Finite Sets β 4. Combinations with Repeating The number of natural number solutions to π₯π₯1 + π₯π₯2 + β―+ π₯π₯ππ = ππ. First consider a seemingly different problem with the same number of solutions. Suppose that we have ππ + ππ β 1 objects denoted by β+β:
Encircle n-1 of these + objects to obtain exactly n segments containing no/some + objects. β If ππ β 1st & ππth encircled +s are next to each other β the ππth segment has no + or π₯π₯ππ = 0. β In general, π₯π₯ππ is the number +s in the ππth segment.
The number of ways of encircling ππ β 1 objects out of ππ + ππ β 1 objects is πΆπΆππβ1ππ+ππβ1 = πΆπΆππππ+ππβ1.
Numbers 1 β 4 indicate order of discussion above
utda
llas.edu /~m
etinPage 15Infinite Sets β Countable Sets
Some experiments have an infinite sample space (set with cardinality of infinity).
Ex: Throwing a coin, rolling a dice and calling a call center are experiments that can be repeated. Each time they are performed, they generate outcomes: {Head, Tail} for throwing a coin, {1,2,3,4,5,6} for rolling a dice, {Busy, Available} for calling a call center.
β If we perform these experiments independently m times, the outcomes respectively become {H,T}m, {1,2,3,4,5,6}m, {B, A}m.
β Here superscript m denotes the Cartesian product applied m times, e.g., {H,T} 2:={H,T} Γ {H,T}. If ππ β β, above sets become infinite but are countable.
Ex: Throwing a coin until heads show up has the following elements (outcomes) in the sample space: {H, TH, TTH, TTTH, TTTTH, β¦}. This sample space has infinite elements but it is countable, i.e., the number of Ts in an element is the number assigned to that element: 0 for H, 1 for TH, β¦
In general, if we are waiting for a particular outcome from an experiment (e.g., winning lottery), the number of sample space elements (purchasing lottery tickets) is infinite but countable.
When an outcome takes countable values, it is (can be represented by) a discrete variable.
utda
llas.edu /~m
etinPage 16Uncountable - Infinite Sets Any interval of real numbers is uncountable.
β Ex: Demand forecasters tend to consider the interval ππ , ππ as the forecast. So the sample space for the (experiment of) demand (realization) is the interval and uncountable.
When an outcome takes uncountable values, it is a continuous variable. Where are continuous variables coming from?
β Quantum (discrete) physics, accounting and others say variables are countable.β Time is a continuous variable but it is an invention of humanity.β Although analysts observe discrete variables in real-life, they invent continuous analogs for the
purpose of ease of analysis/communication. Β» An accountant says the cost is in [10,11] rather than {10.0, 10.1, 10.2, β¦, 10.9, 11.0}
What is in an interval? It is a collection of Real numbers! Unlike integers or rational numbers, Real numbers are not countable;
β To convince yourself, see appendix in the notes. Limiting the analysis to integers or rational numbers gets rid of calculus (continuity,
integral, etc.) and makes the analysis tedious.
Can we deal with an uncountable set without counting it but by focusing on some collections of subsets of it?
utda
llas.edu /~m
etinPage 17Are All Sets or Only Some Measurable?Can we assign probabilities to All or Some? Borel ππ βalgebra on β is obtained by taking complements & countable unions of intervals. A measure assigns numbers from [0,β) to sets
β It needs to be countable additive (measure of a set is the sum of measures of its countable partition) and assign 0 to the empty set
β Answer: Yes. Construct a set to contradict countable additivity of Lebesgue measure: Theorem 1.4.9 of Cohn (2013). The set is uncountable union of meticulously picked reals.
β All subsets of (0,1) is not measurable β consider only a certain collection of subsets. Question 2: Is there a set in Borel ππ βalgebra that is not Lebesgue measurable?
β Answer: No. All sets in Borel ππ βalgebra are measurable: Theorem 1.3.8 of Cohn (2013).β Remark: Countable unions are in Borel ππ βalgebra whereas the counterexample in Q1 has uncountable unions.
To achieve measurability, consider only ππ βalgebras.
Ex: The Ο-algebra also includes intersection of events. This can be shown by taking the complement of the unions. Hence we can measure the probability of two events in the algebra happening together.
Countable sets, Discrete variables Uncountable setsContinuous variablesFinite sets Infinite but
Countable sets
Up to hereinsufficient
Up to hereincorrectUp to here
sufficient & correct
Increasingly strict additivity assumptions
utda
llas.edu /~m
etinPage 20Axiomatic Framework Countable additivity based framework is due to Kolmogorov's short book with a long-title:
Grundbegriffe der Wahrscheinlichkeitsrechnung (probabilistic computation), from the early 1930s. Before Kolmogorov's work, others were experimenting with different set of axioms.
β Borel introduced countable additivity to mathematics and carried it over to probability. β Borel & Hilbert motivated mathematicians in the first half of the 20th century to develop a framework. β Some researchers made more assumptions than necessary, referring to this, Kolmogorov wrote in his preface:
Β» βIn the pertinent mathematical circles it has been common for some time to construct probability theory in accordance with this general point of view. But a complete presentation of the whole system, free from superfluous complications, has been missing.β
β Simultaneously with Kolmogorov, Frechet was writing a book to provide a similar framework. Kolmogorovwrote faster & a shorter book, which was noted later as a significant achievement in Frechet's book:
Β» βIt is not enough to have all the ideas in mind, to recall them now and then; one must make sure that their totality is sufficient, bring them together explicitly, and take responsibility for saying that nothing further is needed β¦. This is what Mr. Kolmogorov did. This is his achievement.β
Frechet was debating the necessity of countable additivity with de Finetti. β de Finetti was against countable additivity & asking Frechet to explain its meaning. β Kolmogorov on the other hand was a pragmatic and said:
Β» βSince the new axiom [countable additivity] is essential only for infinite fields of probability, it is hardly possible to explain its empirical meaning β¦.β
β One of the examples used by de Finetti to attack countable additivity is the infinite lottery. Β» An integer is chosen at random among all integers (countable set). One would like to argue that the
probability of choosing each integer is the same, which is impossible under countable additivity: Β» i) Assign 0 to this probability for every integer β the sum of all probabilities = zero,
but the probability of choosing an integer (probability of countable union of choosing an integer) = 1. Β» ii) Assign ππ > 0, the sum of all probabilities ββ. de Finetti wanted to rule out countable additivity and to develop probability with only finite additivity.
Kolmogorov
Frechet
Borel
Hilbert
de Finetti
utda
llas.edu /~m
etinPage 21Summary
Experiments, Outcomes, Events Counting
β Finite OutcomesΒ» Number of distinct strings, sequences, subsets and multisets
β Infinite but Countable Outcomesβ Uncountable Outcomes
