Home > Documents > Creating User Interfaces Review midterm Sampling Homework: User observation reports due next week.

# Creating User Interfaces Review midterm Sampling Homework: User observation reports due next week.

Date post: 30-Dec-2015
Category:
Author: sherilyn-oconnor
View: 214 times
Embed Size (px)
of 30 /30
Creating User Interfaces Review midterm Sampling Homework: User observation reports due next week
Transcript
• Creating User InterfacesReview midterm SamplingHomework: User observation reports due next week

• SamplingBasic technique when it is impossible or too expensive to measure everything/everybodyPremise: possible to get random sample, meaning every member of whole population equally likely to be in sampleNOTE: not a substitute for monitoring directly activity on / with interface

• SourceThe Cartoon guide to Statistics by Larry Gonick and Woollcott Smith HarperResource

Procedures (formulas) presented without proof, though, hopefully, motivated

• TaskWant to know the percentage (proportion) of some large groupadults in USAtelevision viewersweb usersFor a particular thingthink the president is doing a good jobwatched specific programviewed specific commercialvisited specific website

• Strategy: SamplingAsk a small groupphonesolicitation at a mallFollow-up or prelude to access to webpageother?Monitor actions of a small group, group defined for this purposeMonitor actions of a panel chosen ahead of timeALL THESE: make assumption that those in group are similar to the whole population.

• Two approachesEstimating with confidence interval c in general population based on proportion phat in sample Hypothesis testing: H0 (null hypothesis) p = p0 versus Ha p > p0

• Estimation processConstruct a sample of size n and determine phatAsk who they are voting for (for now, let this be binomial choice)Use this as estimate for actual proportion p.

but the estimate has a margin of error. This means : The actual value is within a range centered at phat UNLESS the sample was really strange.The confidence value specifies what the chances are of the sample being that strange.

• StatementI'm 95% sure that the actual proportion is in the following range.

phat m

• Image from Cartoon bookYou are standing behind a target.An arrow is shot at the target, at a specific point in the target. The arrow comes through to your side.You draw a circle (more complex than +/- error) and say Chances are: the target point is in this circle unless shooter was 'way off' . Shooter would only be way off X percent of the time. (Typically X is 5% or 1%.)

• Mathematical basisSamples are themselves normally distributedif sample and p satisfy certain conditions.Most samples produce phat values that are close to the p value of the whole population.Only a small number of samples produce values that are way off.Think of outliers of normal distribution

• Actual (mathematical) processCan use these techniques when n*p>=5 and n*(1-p)>=5The phat values are distributed close to normal distribution with standard deviation sd(p) =

Can estimate this using phat in place of p in formula!Choose the level of confidence you want (again, typically 5% or 1%). For 5% (95% confident), look up (or learn by heart the value 1.96: this is the amount of standard deviations such that 95% of values fall in this area. So .95 is P(-1.96

• Notesp is less than 1 so (1-p) is positive.Margin of error decreases as p varies from .5 in either direction. (Check using excel).if sample produces a very high (close to 1) or very low value (close to 0), p * (1-p) gets smaller(.9)*(.1) = .09; (.8)*(.2) = .16, (.6)*(.4) =.24; (.5)*.5)=.25

• NotesNeed to quadruple the n to halve the margin of error.

• FormulaUse a value called the z transform95% confidence, the value is 1.96

• MechanicsProcess is Gather data (get phat and n)choose confidence level Using table, calculate margin of error.

Book example: 55% (.55 of sample of 1000) said they backed the politician)sd(phat) = square_root ((.55)*(.45)/1000) = .0157Multiply by z-score (e.g., 1.96 for a 95% confidence) to get margin of errorSo p is within the range: .550 (1.96)*(.0157) and .550 + (1.96)*(.0157) .519 to .581 or 51.9% to 58.1%

• Example, continued51.9% to 58.1% may round to 52% to 58% or may say 55% plus or minus 3 percent.

What is typically left out is that there is a 1/20 chance that the actual value is NOT in this range.

• 95% confident means95/100 probability that this is true5/100 chance that this is not true5/100 is the same as 1/20 so,There is only a 1/20 chance that this is not true.Only 1/20 truly random samples would give an answer that deviated more from the real

ASSUMING NO INTRINSIC QUALITY PROBLEMSASSUMING IT IS RANDOMLY CHOSEN

• 99% confidence means[Give fraction positive][Give fraction negative]

• WhyConfidence intervals given mainly for 95% and 99%??

History, tradition, doing others required more computing.

• Let's ask a questionHow many of you watched the last Super Bowl?Sample is whole classHow many registered to vote?Sample size is number in class 18 and older

????

• Excel: columns A & B

• Variation of book problemSay sample was 300 (not 1000).sd(phat) = square_root ((.55)*(.45)/300) = .0287Bigger number. The circle around the arrow is larger. The margin is larger because it was based on a smaller sample. Multiplying by 1.96 get .056, subtracting and adding from the .55 get .494 to .606 You/we are 95% sure that true value is in this range.Oops: may be better, but may be worse. The fact that the lower end is below .5 is significant for an election!

Divisor smaller

• Exercisesize of sample is nproportion in sample is phatconfidence level produces factor called the z-scoreCan be anything but common values are [80%], 90%, 95%, 99%) Use table. For example, 95% value is 1.96; 99% is 2.58Calculate margin of error m m = zscore * sqrt((phat)*(1-phat)/n)Actual value is >= phat m and
• Opportunity sampleCommon situationpeople assigned/asked to have a meter attached to their TVspeople asked/voluntarily sign up to have a meter (software) installed in their computers.people asked during a Web session to participate in surveystudents in a specific class!Practice is to determine categories (demographics) and project the sample results to the subpopulation to the populationFor example, if actual population was 52% female and 48% male, and sample (panel) is 60% male and 40% female, use proportions to adjust resultBut maybe this fact hides problem with the sampleHas negative features of any opportunity sampleAre these folks different than others in their (sub)population?

• RequirementsModel / Categories must be well-defined and validHispanic versus (Cuban, others) in Florida in 2000Need independent analysis of subpopulations representation in general populationThe sample sizes are the individual Ns, making the margin of errors larger

• Adjustment from panel dataPanel of 10: 6 females, 4 malesPopulation is 52% female and 48% maleFemale panelists: 5 liked interface, 1 didn't. Male panelists: 2 liked interface, 2 didn't.Estimate for whole population (size P) (5/6)* .52 * P + (2/4)*.48* P

• Critical part of surveysand survey analysis:Understand the exact wording of question.Understand definition of categories of population.Don't make assumptionsAdmire Michelle Obama exampleBelief in Holocaust example

• Usability researchOften aims for qualitative, not quantitative resultsIdeas, critical factors

Note: there are fields of studyNon-numeric statisticsQualitative researchStill necessary to be systematic.AD: consider taking Statistics!

• HomeworkContinue work on user observation studiesThis is qualitative work

Recommended