Date post: | 26-Dec-2015 |
Category: |
Documents |
Upload: | belinda-gordon |
View: | 218 times |
Download: | 1 times |
1
Ask Good Questions
Allan J. Rossman
Dept of Statistics
Cal Poly – San Luis Obispo (USA)
8th International Conference on Teaching Statistics
2Ask Good Questions (ICOTS-8) 2
Frank Sinatra
What’s the secret to success as a singer? “Sing Good Songs”
Rossman
33
My similarly succinct suggestion What’s the secret to success at
teaching statistics?
“Ask Good Questions”
Rossman Ask Good Questions (ICOTS-8)
4
No hidden message
My take-home message Ask Good Questions
Quiz at end of presentation Q: What was my point? A: Ask Good Questions
I may email you in 10 years Q: What do you remember? A: Ask Good Questions
Ask Good Questions (ICOTS-8)Rossman
55
George Cobb
“Judge a textbook by its exercises, and you cannot go far wrong.”
Rossman Ask Good Questions (ICOTS-8)
66
My corollary
“Judge a teacher by the questions that he/she asks of students, and you cannot go far wrong.”
Rossman Ask Good Questions (ICOTS-8)
77
What kinds/purposes of questions? Guide students to develop their understanding
and skills Learning activities
Assess how well students have learned Assessment questions
Rossman Ask Good Questions (ICOTS-8)
88
What makes a question “good”? I respectfully decline to answer
For now Instead I’ll present some examples of both
kind of questions (learning activities, assessment questions) Taken from “Stat 101” (introductory, algebra-
based, service course at tertiary level) Then I’ll revisit this question
Rossman Ask Good Questions (ICOTS-8)
1010
George Cobb (again)
“Shorn of all subtlety and led naked out of the protective fold of educational research literature, there comes a sheepish little fact: lectures don’t work nearly as well as many of us would like to think.”
Rossman Ask Good Questions (ICOTS-8)
11
Example 1: TVs and life expectancy Is there an association between a country’s
life expectancy and its number of televisions per person?
Rossman 11Ask Good Questions (ICOTS-8)
12
Example 1: TVs and life expectancy Does the scatterplot reveal an association?
Which direction? How strong? Linear? Based on these data, would you conclude
that sending TVs to Haiti would cause Haitians to start living longer?
Identify a confounding variable that explains the association
Does a strong association between variables imply a cause/effect relationship?
Rossman 12Ask Good Questions (ICOTS-8)
1313
Example 2: Reading cancer pamphlets Are pamphlets containing information for
cancer patients written at an appropriate level that cancer patients can understand?
Ask Good Questions (ICOTS-8)Rossman
14
Example 2: Reading cancer pamphlets Explain why the mean cannot be calculated
for patients’ reading levels Determine median reading level of patients,
median readability level of pamphlets Are the pamphlets’ readability levels well-
matched to the patients’ reading levels? What proportion of patients have reading
level below that of simplest pamphlet?
Rossman 14Ask Good Questions (ICOTS-8)
15
Example 2: Reading cancer pamphlets
0
0.05
0.1
0.15
0.2
0.25
0.3
unde
r 3 3 4 5 6 7 8 9
10 11 12
abov
e 12
level
prop
orti
on
patientspamphlets
Ask Good Questions (ICOTS-8)Rossman
16161616
Example 3: Sleep deprivation
Does sleep deprivation have harmful effects on cognitive functioning three days later? 21 subjects; random assignment
Identify type of study, observational units, explanatory and response variables
improvement
sleep c
onditio
n
4032241680-8-16
deprived
unrestricted
Ask Good Questions (ICOTS-8)Rossman
17171717
Example 3: Sleep deprivation
Students use simulation (tactile, then computer-based) to investigate core question of statistical inference: Is such an extreme difference unlikely to occur by
chance (random assignment) alone (if there were no treatment effect)?
Ask Good Questions (ICOTS-8)Rossman
18
Example 3: Sleep deprivation
Summarize conclusion, and explain reasoning process that supports your conclusion
Ask Good Questions (ICOTS-8)Rossman
19
Example 4: Which tire?
Legendary campus story Which tire would you pick?
Research question: Do people tend to pick right front tire more often than random chance? Again, students investigate the question of how
surprising the observed class result would be under the null model with ¼ probability
Rossman 19Ask Good Questions (ICOTS-8)
20
Example 4: Which tire?
What if 32% of a random sample selects right front? Is this a significant/convincing result? Or, what additional information would you need? Make prediction for how p-value will change (if at
all) as sample size increases Calculate p-value for many different sample sizes Summarize how sample size affects p-value,
strength of evidence Explain why this makes intuitive sense (in hindsight, if
not in foresight)
Rossman 20Ask Good Questions (ICOTS-8)
2222
Joan Garfield, Beth Chance
Because students learn to value what they know they will be tested on, we should assess what we value.” - JG
“The number one mantra to remember when designing assessment instruments is: Assess what you value.” - BC
Rossman Ask Good Questions (ICOTS-8)
23
Assessment example 1
For each of the following quantities, indicate whether it can NEVER be negative or can SOMETIMES be negative.
a) Sample size b) Sample proportion
c) Standard deviation d) Inter-quartile range
e) Difference in sample means f) Odds ratio
g) Total sum of squares h) Slope coefficient
i) Coefficient of determination j) Correlation coefficient
k) ANOVA F-test statistic l) p-value
Rossman 23Ask Good Questions (ICOTS-8)
24
Assessment example 2
The U.S. has slightly more than 300 million residents. In order to estimate the proportion of U.S. residents who have a facebook account to within about 3 percentage points with 95% confidence, about how many people should be randomly sampled?
100 1000 10,000
100,000 1,000,000 10,000,000
Rossman 24Ask Good Questions (ICOTS-8)
25
Assessment example 3
Are people more likely to lie with email than with pencil-and-paper communication? A recent study involved 48 graduate students in business at a particular university who participated in a “bargaining” game. Researchers kept track of whether the student lied about the amount of money involved when negotiating with another player. Some of the participants were randomly assigned to use email for their communication, while others used paper-and-pencil. It turned out that 24 of 26 who used email were guilty of lying about the amount of money involved, compared to 14 of 22 who used paper-and-pencil.
Rossman 25Ask Good Questions (ICOTS-8)
26
Assessment example 3 (cont.)Use a simulation analysis to investigate whether
these data provide strong evidence in support of the research conjecture in the first sentence above. Summarize the conclusions that you draw from your analysis. Be sure to address issues of statistical significance, causation, and generalizability. Also explain the reasoning process and justification for your conclusions.
Rossman 26Ask Good Questions (ICOTS-8)
27
Assessment example 4
State a research question, and describe a data collection plan to address it, for which a two-proportion z-test would be the relevant inference procedure. Identify the explanatory and response variables, and also state the relevant hypotheses, defining the parameter values appropriately. Be sure to indicate whether the data collection plan involves random sampling or random assignment.
Rossman 27Ask Good Questions (ICOTS-8)
28
Assessment example 5 (from 2009 AP Statistics exam) Consider the statistic mean / median. What
values of this statistic might indicate that the population distribution is skewed to the right?
Consider simulation results for values of mean / median, based on a normal population, and also the observed value of mean / median for given sample data. Do the simulation results suggest that the underlying population is skewed to the right? Explain.
Rossman 28Ask Good Questions (ICOTS-8)
29
Assessment example 5 (from AP Statistics, cont.)
Use only statistics in the five-number summary (min, Q1, median, Q3, max) to propose your own measure of skewness. Indicate values of this statistic that would suggest skewness to the right. Explain.
Rossman 29Ask Good Questions (ICOTS-8)
30
Oh, by the way ….
Notice that this talk is trying to model question-based pedagogy From these specific questions/examples we can
extract principles of “good” questions
Ask Good Questions (ICOTS-8)Rossman
3131
So, what makes a question “good”? Makes students think
Goes beyond their starting point Challenges without overwhelming students
Engages students’ interest To put forth effort to learn To care about the course material being studied
Addresses important ideas Indicates to students what’s valued
Rossman Ask Good Questions (ICOTS-8)
3232
What else makes a question “good”? Fits within logical progression
Builds on students’ knowledge Asks students to make/check predictions Confronts students’ misconceptions
Inspires students to ask their own questions About course material About “real-world” phenomena that can be
investigated using intellectual skills being learned
Rossman Ask Good Questions (ICOTS-8)
33
One vehicle for students posing and investigating own questions Students use gapminder software (Hans Rosling) to
Pose research questions about the world Investigate those questions with animated graphics Write report of their findings
Rossman 33Ask Good Questions (ICOTS-8)
3434
Some questions from students’ projects Are students more likely to recycle water bottles depending on the
proximity of recycle bin and waste basket? Can people better answer math problems if they are presented with
Roman letter notation as opposed to Greek letter notation? Are people who walk into a clothing store more likely to purchase
something when the weather is rainy as opposed to sunny? Is balsa wood less elastic after it has been immersed in water? Are students’ reaction times affected by whether they’ve just
completed exercising? Are faculty more likely to drive a foreign car than students? Do guests at a dinner respond more positively to spaghetti sauce
that has been sweetened or not sweetened? Do college students spend more money at a local fast food
restaurant if they are under the influence of alcohol?
Rossman Ask Good Questions (ICOTS-8)
35
More unsolicited advice
ICOTS: Teaching Statistics My experience at introductory tertiary level
Any advice for teachers of younger students? Ask Good Questions
Any advice for teachers of graduate students? Ask Good Questions
Any advice for statistics education researchers? Ask Good Questions
Ask Good Questions (ICOTS-8)Rossman
36
Examples of interest to me
Does presenting activities/examples based on real data from genuine studies Improve student learning of statistical ideas? Increase student interest in statistics? Enhance transferability of skills?
Do students learn more if tactile simulations are presented before technology ones?
Does it matter whether students construct or simply consume simulation results?
Ask Good Questions (ICOTS-8)Rossman
3737
Some final questions for you
Have you ever attended a presentation titled Ask Bad Questions, or Don’t Ask Questions?
Of course not! So, … Why am I wasting your time offering such obvious
advice as “Ask Good Questions”??
Rossman Ask Good Questions (ICOTS-8)
3838
Yeah, why am I wasting your time? I respectfully suggest that the next time you
or I feel compelled to invest some of our valuable time, thought, energy, and creativity on … Preparing crystal-clear lectures Writing lucid paragraphs of exposition Developing software illustrations Crafting beautiful presentations
Rossman Ask Good Questions (ICOTS-8)
41
So, what do I suggest instead? Instead we should invest these precious
commodities (time, thought, energy, creativity) on Developing engaging classroom activities Preparing thought-provoking assignments Writing lab or project expectations Crafting effective assessment items
In other words, we should focus more of our attention on making sure that we …
41Rossman Ask Good Questions (ICOTS-8)
43
Thanks very much!
[email protected] http://statweb.calpoly.edu/
arossman/AskGoodQuestionsICOTS.ppt
Ask Good Questions (ICOTS-8)Rossman