Date post: | 28-Dec-2015 |
Category: |
Documents |
Upload: | ira-griffith |
View: | 219 times |
Download: | 4 times |
SurveysCynthia J. Miller, Ph.D.
Assistant ProfessorDept. of Physiology & Biophysics
University of Louisville
Why should you use surveys?
To reach a large number
of participants
To obtain baseline data and monitor
changes
To gauge motivations, beliefs, and
attitudes
To evaluate the
effectiveness of teaching practices
Time-efficient,
convenient, and familiar
To gather demographic
s
To examine alignment
between faculty and students
Developing Surveys Search for established and/or
validated instrumentso Consult the literature in your fieldo Verify that the instrument is
beneficial for your useo Example: SALG (Student
Assessment of their Learning Gains) Survey
If an appropriate instrument does not exist:o AKA “back to square one”o Example: CREATE Surveyo Define your objectiveso Work backwards-determine the
data that you need to gather
Question Development Forced vs. open-response items
o Forced, Likert-scale most commonly usedo Forced responses can be easier to analyzeo Open responses can be useful due to multiple
definitions/perceptions of questions, but also increases difficulty of analysis
Try to ensure anonymity if possible, guarantee confidentiality
Use simple terminology and unambiguous language
Check for bias-ensure you are not asking leading questions
Question Development Ask questions from multiple perspectives and in
different parts of the survey to ensure validity Hoskins Paper
o “The scientific literature is difficult to understand”
o “I am confident in my ability to critically review scientific literature”
o Negative statements are reverse-scored when quantifying data
Question Development
Revisit original goals/hypotheses and ensure alignment
Pilot questions with your research team, peers, a sample of target respondents, and a statistician
Survey Distribution
Pen-and-Paper Benefits: Can ensure
completion by respondents
Limitations: Must compile data by hand, lower perception of anonymity (particularly with open-response questions)
Online Programs such as
SurveyMonkey, SurveyGizmo, Qualtrics
Most institutions have programs available for free or reduced prices
Benefits: Compile data, Produce graphs or charts, Statistical analysis, Larger potential sample sizes
Limitations: May lower response rates
Effective DistributionChallenges
Low Response Rateso “Survey Fatigue”o Poor timing
Respondents not carefully completing survey
Suggestions Consider providing incentives for
participation Emphasize importance of results Complete during class time or
scheduled meetings Endorsement by respected
authorities
Potential Limitations
Samples may not be representative of entire population
Self-reported data is often biasedo Past eventso Behaviors that may be
negatively perceived Respondents may feel the
need to “tell you what you want to hear”
Triangulation Use multiple methods of data collection to
analyze the same research question
Faculty Survey
s
Student Surveys
ExamScore
s
Student
Focus Groups
Faculty Surveys
Classroom Observation
s
CREATE Method Papers
Student Surveys (Pre/Post
)
Mentioned past studies on critical-thinking skills,
interviews, and validated surveys
Student
Surveys
Student Exams
(Pre/Post)
Student Article
Critiques
Segura-Totten and DalmanHoskins, Lopatto, and Stevens
Resources Survey Design
o www.statpac.com/surveys Describing & Measuring Undergraduate Stem Teaching Practices
o http://ccliconference.org/files/2013/11/Measuring-STEM-Teaching-Practices.pdf
Duke Initiative on Survey Methodologyo http://dism.ssri.duke.edu/question_design.php
General Introductiono Fink AG. How to Conduct Surveys: A Step-by-Step Guide. Sage: Thousand
Oaks, CA, 2008. Question Development and Distribution Methods
o DeVellis RF. Scale Development Theory and Application, 3rd Edition. Sage: Thousand Oaks, CA, 2012.
o Dillman DA. Mail and Internet Surveys: The Tailored Design Method, 2nd Edition. John Wiley & Sons, Inc.: Hoboken, NJ, 2007.
Triangulationo Creswell N, Clark V. Designing and Conducting Mixed Methods Research.
Sage: Thousand Oaks, CA, 2007.