Post on 13-Feb-2016
description
transcript
AScILSAssessing Science Inquiry and
Leadership Skills
Research Questions:• How do activities implemented by support programs (especially research and mentoring) influence: - skills in science inquiry and scientific team leadership and membership
- beliefs in efficacy and collective efficacy regarding these skills- stage-appropriate education and career outcomes?
• Are these influences similar for minority and non-minority students?
“Inquiry” in the context of AScILS
Lisa HunterAssociate Director, Education & Human Resources, Center for Adaptive Optics
AScILS Research Team MemberInquiry Strand
Advisor/Mentor Focus Group, 9/13 & 9/15/05
What do we mean by “inquiry”
The processes or methods that scientists use to gain an understanding of the world, and to create a body of knowledge (“research”)
Which “Science Inquiry Skills?”
CfAOSurvey
Intern Advisor/Mentor Meeting (9/13 & 9/15)
Education & Assessment Lit.
AScILS AlumniSurvey
AScILS qualitativeStudies (interviews)
Team Members’Prior Experience
Current AScILS Inquiry Strand Focus: Which science inquiry skill(s) or knowledge should be assessed?
Elements of scientific inquiry• Skills or processes (what scientists do)• Knowledge about scientific inquiry (what
scientists know)
Scientific Inquiry Processes Chinn, et. al. “Cognitive processes”
America’s Lab Report “Scientists’ activities”
Generating research questions
Posing a research question
Designing studies Formulating hypotheses
Making observations Designing investigations
Explaining results Make observations, gather & analyze data
Developing theories Building and revising scientific models or theories
Studying research reports Evaluating, testing, or verifying models
1. Chinn, Clark, et. al. (2002). Epistemilogically Authentic Inquiry in Schools. Science Education, 86:2.
2. National Research Council (2005). America’s Lab Report: Investigations in High School Science. Washington, DC: The National Academies Press.
Knowledge about scientific inquiry or the nature of science
Lederman, et. al “Nature of science aspects”Tentativeness (of scientific knowledge)
Empirical basis
Subjectivity (influenced and driven by presently accepted theories)
Creativity
Observations and inference
Social and cultural influences
Nature of theories (how and why they change)
Lederman, N.G. et. al. (2002). Journal of Research in Science Teaching, 39(6), 497-521.
Why we are interested in “inquiry”
• Many (10 of 14) of the AScILS study programs include research experiences, and is one of the 3 criteria for inclusion as a study program
• Students engage in authentic researchexpect gains in inquiry (or research) skills and understandings about inquiry
• NOTE: Student researchers clearly gain specific technical skills, but this is outside the AScILS focus
How can we assess interns’ gains in inquiry skills and/or
understandings about inquiry?
If we could assess inquiry skills, there are many interesting questions• Do students gain inquiry skills and
understandings about inquiry more in a research experience than in classes?
• Do gains depend on the type of research experience?
• Is there a connection between gains in inquiry skills and students’ confidence (or self-efficacy)? Students’ identity as a scientist? Persistence in science?
What is known?
• Knowledge base on assessing inquiry skills gained from authentic research is very limited:– Papers reporting self-reported gains from interviews and
surveys• “Thinking and working like a scientist” is a major gain/benefit
from research experiences*– Papers reporting on how research experiences impacted
students understandings about the nature of science– Significant amount of work on assessing inquiry skills at K-
12 classroom level
*Seymour, Elaine, et. al. (2004). Establishing the Benefits of Resarch Experiences for Undergraduates: First Findings from a Three-Year Study. Science Education, 88: 493-534.
AScILS Inquiry Strand
• Determine the most fruitful inquiry skill to assess
• Design a performance assessment (or simulation) to assess those skills
Which “Science Inquiry Skills?”
CfAOSurvey
Intern Advisor/Mentor Meeting (9/13 & 9/15)
Education & Assessment Lit.
AScILS AlumniSurvey
AScILS qualitativeStudies (interviews)
Team Members’Prior Experience
Current AScILS Inquiry Strand Focus: Which science inquiry skill(s) or knowledge should be assessed?
Scientific Inquiry Processes
Chinn, et. al. “Cognitive processes”
America’s Lab Report “Scientists’ activities”
Generating research questions
Posing a research question
Designing studies Formulating hypotheses
Making observations Designing investigations
Explaining results Make observations, gather & analyze data
Developing theories Building and revising scientific models or theories
Studying research reports Evaluating, testing, or verifying models
Scientific Inquiry Processes
Chinn, et. al. “Cognitive processes”
America’s Lab Report “Scientists’ activities”
Engineering processes
Generating research questions
Posing a research question
Identifying a problem or design need
Designing studies Formulating hypotheses Designing solutions within constraints
Making observations Designing investigations Implementing
Explaining results Make observations, gather & analyze data
Testing
Developing theories Building and revising scientific models or theories
Explaining solution, including limitations and tradeoffs
Studying research reports Evaluating, testing, or verifying models
Studying research reports
As a result of doing research with your group, which skill(s) are most important
for your students to develop?Chinn, et. al. “Cognitive processes”
America’s Lab Report “Scientists’ activities”
Engineering processes
Generating research questions
Posing a research question
Identifying a problem or design need
Designing studies Formulating hypotheses Designing solutions within constraints
Making observations Designing investigations Implementing
Explaining results Make observations, gather & analyze data
Testing
Developing theories Building and revising scientific models or theories
Explaining solution, including limitations and tradeoffs
Studying research reports Evaluating, testing, or verifying models
Studying research reports
ASSESSMENT BASICS
AScILS Style
Jerome ShawAssistant Professor
Education Department
ASSESSMENT as a PROCESS of
REASONING from EVIDENCE
Data
Evide nc e
Infe re nc es
Source:
National Research Council. (2001). Knowing what students know: The science and design of educational assessment. Committee on the Foundations of Assessment. Pelligrino, J., Chudowsky, N., and Glaser, R., editors. Board on Testing and Assessment, Center for Education. Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.
ASSESSMENT MANTRA
• STANDARDS– what?
• TASKS– how?
• RUBRICS– how well?
Scientific Inquiry Processes
“Cognitive Processes”Chinn et al.
“Scientists’ activities”America’s Lab Report
Engineering processes
Generating research questions Posing a research question Identifying a problem or design need
Designing studies Formulating hypotheses Designing solutions within constraints
Making observations Designing investigations Implementing
Explaining results Make observations, gather & analyze data
Testing
Developing theories Building and revising scientific models or theories
Explaining solutions, including limitations and tradeoffs
Studying research reports Evaluating, testing, or verifying models
Studying research reports
Source:
Sandoval, W. A., & Reiser, B. J. (2004). Explanation-driven inquiry: Integrating conceptual and epistemic scaffolds for scientific inquiry. Science Education, 88, 345-372.
Explanation Rubric: Sample Criteria
Thoroughness and Clarity of Explanations
Use of Data
Ruling Out Alternative Explanations
Documenting the Limitations of Your Explanations
Scoring Guide: Using EvidenceScore
Response uses objective reason(s) based on relevant evidence to argue for or against a choice.
4Response accomplishes level 3, AND goes beyond in some relevant way, e.g., questioning or justifying the source, validity, and/or quantity of evidence.
3Provides major objective reasons AND supports each with relevant and accurate evidence.
2Provides some objective reasons AND some supporting evidence, BUT at least one reason is missing and/or part of the evidence is incomplete.
1Provides only subjective reasons (opinions) for choice; uses unsupported statements; OR uses inaccurate or irrelevant evidence from the activity.
Source:Science Education for Public Understanding Program (1995) as presented in… National Research Council. (2001). Classroom assessment and the National Science Education Standards. Committee on Classroom Assessment and the National Science Education Standards. J. Myron Atkin, Paul Black, and Janet Coffey (Eds.). Center for Education, Division of Behavioral and Social Sciences Education. Washington, DC: National Academy Press.
Which “Science Inquiry Skills?”
CfAOSurvey
Intern Advisor/Mentor Meeting (9/13 & 9/15)
Education & Assessment Lit.
AScILS AlumniSurvey
AScILS qualitativeStudies (interviews)
Team Members’Prior Experience
Current AScILS Inquiry Strand Focus: Which science inquiry skill(s) or knowledge should be assessed?