Designing Experiments: Student Learning Experience and Behaviour in Undergraduate Physics Laboratories Bei Cai, 1, * Lindsay A. Mainhood, 2, † Ryan Groome, 1 Corinne Laverty, 3 and Alastair McLean 1, ‡ 1 Department of Physics, Engineering Physics, and Astronomy, Queen’s University, Kingston, Ontario, K7L 3N6, Canada 2 Faculty of Education, Queen’s University, Kingston, Ontario, K7M 5R7, Canada 3 Center for Teaching and Learning, Queen’s University, Kingston, Ontario, K7L 3N6, Canada (Dated: June 15, 2020) We investigated physics students’ learning experience and behaviour in a second-year laboratory by analyzing transcribed audio recordings of laboratory sessions. One student group was given both a problem and procedure and asked to analyze and explain their results. Another was provided with only the problem and asked to design and execute the experiment, interpret the data, and draw conclusions. These two approaches involved different levels of student inquiry and they have been described as guided and open inquiry respectively. The latter gave students more opportunities to practice “designing experiments,” one of the six major learning outcomes in the recommendations for the undergraduate physics laboratory curriculum by the American Association of Physics Teachers (AAPT). Qualitative analysis was performed of the audio transcripts to identify emergent themes and it was augmented by quantitative analysis for a richer understanding of students’ experiences. An important finding is that significant improvements can be made to undergraduate laboratories impacting both student learning experience and behaviour by increasing the level of inquiry in laboratory experiments. This is most easily achieved by requiring students to design their own experimental procedures. I. INTRODUCTION As physics is an experimental science, it is natural that physicists receive some of their education in a laboratory en- vironment where personal observation and physical experi- mentation can take place. AAPT identifies the foremost goals of physics laboratories as learning to think like a physicist, referred to as “habits of mind,” and constructing a view of the physical world through experimental design, data collec- tion, analysis, and interpretation [1]. The AAPT recommen- dations for the undergraduate physics laboratory curriculum outlines six learning outcomes that underpin achievement of these goals; these are: constructing knowledge, modeling, de- signing experiments, developing technical and practical lab- oratory skills, analyzing and visualizing data, and communi- cating physics. The traditional mode of instruction in physics laborato- ries frequently uses the “recipe” approach where students are given a detailed procedure. This has recently been described as a form of passive teaching, masquerading as active engage- ment [2]. When students follow detailed instructions without having to grapple with their own conceptual understanding, they are not challenged to think for themselves. Fraser et al. argue that, despite the fact they are using physics apparatus, they are solely “hands-on” and not “heads-on” [2]. There is emerging interest in student-driven inquiry labo- ratories that allow students to take ownership of their experi- mental work [3, 4]. The inquiry approach provides opportu- * [email protected]. Current address: Department of Physics, Carleton University, Ottawa, Ontario, K1S 5B6, Canada † [email protected]‡ [email protected]nities to build conceptual understanding and scientific skills through active participation in design and decision-making processes. Inquiry-based learning is endorsed as learner- focused pedagogy that reinforces achievement of learning outcomes including subject content and thinking skills. It has been shown to improve knowledge of subject content [5] and science process skills [6]. More specifically, it aids concep- tual understanding, analysis of experimental errors, interpre- tation and representation of data, and evaluation of results as well as enjoyment of learning [7, 8]. A review of 138 inquiry studies in elementary-secondary education concluded that inquiry is a means to prompt ac- tive thinking, increase conceptual understanding, and draw conclusions from data [9]. A meta-analysis of 22 studies over a ten-year span on the same population confirmed that inquiry prompts active engagement in higher-order thinking skills, such as proposing and evaluating results, which helps students understand science [10]. In a comparison of differ- ent levels of inquiry, Spronken-Smith and Walker [11] found that inquiry questions serve as a trigger for learning, student collaboration, teacher facilitation, and increased self-directed learning [11]. Although the inquiry approach is championed in many post-secondary science programs, characterization of the at- tributes of inquiry has proven difficult. This may in part be due to the fact that the term is used to describe both a teach- ing and learning approach as well as a process of investiga- tion [12, 13]. An inquiry scale, derived from an analysis of nearly 400 undergraduate experiments in 22 lab manuals, was recently introduced by Buck and colleagues [13]. They posited that inquiry is a continuum. Within this continuum they defined five levels, each with a different amount of student indepen- dence. As the amount of independence increases, the amount arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020
Transcript
Designing Experiments: Student Learning Experience and Behaviour in Undergraduate PhysicsLaboratories
Bei Cai,1, ∗ Lindsay A. Mainhood,2, † Ryan Groome,1 Corinne Laverty,3 and Alastair McLean1, ‡
1Department of Physics, Engineering Physics, and Astronomy,Queen’s University, Kingston, Ontario, K7L 3N6, Canada
2Faculty of Education, Queen’s University, Kingston, Ontario, K7M 5R7, Canada3Center for Teaching and Learning, Queen’s University, Kingston, Ontario, K7L 3N6, Canada
(Dated: June 15, 2020)
We investigated physics students’ learning experience and behaviour in a second-year laboratory by analyzingtranscribed audio recordings of laboratory sessions. One student group was given both a problem and procedureand asked to analyze and explain their results. Another was provided with only the problem and asked to designand execute the experiment, interpret the data, and draw conclusions. These two approaches involved differentlevels of student inquiry and they have been described as guided and open inquiry respectively. The lattergave students more opportunities to practice “designing experiments,” one of the six major learning outcomesin the recommendations for the undergraduate physics laboratory curriculum by the American Association ofPhysics Teachers (AAPT). Qualitative analysis was performed of the audio transcripts to identify emergentthemes and it was augmented by quantitative analysis for a richer understanding of students’ experiences. Animportant finding is that significant improvements can be made to undergraduate laboratories impacting bothstudent learning experience and behaviour by increasing the level of inquiry in laboratory experiments. This ismost easily achieved by requiring students to design their own experimental procedures.
I. INTRODUCTION
As physics is an experimental science, it is natural thatphysicists receive some of their education in a laboratory en-vironment where personal observation and physical experi-mentation can take place. AAPT identifies the foremost goalsof physics laboratories as learning to think like a physicist,referred to as “habits of mind,” and constructing a view ofthe physical world through experimental design, data collec-tion, analysis, and interpretation [1]. The AAPT recommen-dations for the undergraduate physics laboratory curriculumoutlines six learning outcomes that underpin achievement ofthese goals; these are: constructing knowledge, modeling, de-signing experiments, developing technical and practical lab-oratory skills, analyzing and visualizing data, and communi-cating physics.
The traditional mode of instruction in physics laborato-ries frequently uses the “recipe” approach where students aregiven a detailed procedure. This has recently been describedas a form of passive teaching, masquerading as active engage-ment [2]. When students follow detailed instructions withouthaving to grapple with their own conceptual understanding,they are not challenged to think for themselves. Fraser et al.argue that, despite the fact they are using physics apparatus,they are solely “hands-on” and not “heads-on” [2].
There is emerging interest in student-driven inquiry labo-ratories that allow students to take ownership of their experi-mental work [3, 4]. The inquiry approach provides opportu-
nities to build conceptual understanding and scientific skillsthrough active participation in design and decision-makingprocesses. Inquiry-based learning is endorsed as learner-focused pedagogy that reinforces achievement of learningoutcomes including subject content and thinking skills. It hasbeen shown to improve knowledge of subject content [5] andscience process skills [6]. More specifically, it aids concep-tual understanding, analysis of experimental errors, interpre-tation and representation of data, and evaluation of results aswell as enjoyment of learning [7, 8].
A review of 138 inquiry studies in elementary-secondaryeducation concluded that inquiry is a means to prompt ac-tive thinking, increase conceptual understanding, and drawconclusions from data [9]. A meta-analysis of 22 studiesover a ten-year span on the same population confirmed thatinquiry prompts active engagement in higher-order thinkingskills, such as proposing and evaluating results, which helpsstudents understand science [10]. In a comparison of differ-ent levels of inquiry, Spronken-Smith and Walker [11] foundthat inquiry questions serve as a trigger for learning, studentcollaboration, teacher facilitation, and increased self-directedlearning [11].
Although the inquiry approach is championed in manypost-secondary science programs, characterization of the at-tributes of inquiry has proven difficult. This may in part bedue to the fact that the term is used to describe both a teach-ing and learning approach as well as a process of investiga-tion [12, 13].
An inquiry scale, derived from an analysis of nearly 400undergraduate experiments in 22 lab manuals, was recentlyintroduced by Buck and colleagues [13]. They posited thatinquiry is a continuum. Within this continuum they definedfive levels, each with a different amount of student indepen-dence. As the amount of independence increases, the amount
of information provided by the instructor decreases. Our fo-cus is on guided and open inquiry. Guided inquiry providesthe research question and procedures, and the students are ex-pected to analyze and explain their results. Open inquiry pro-vides the research question and it is up to the students to de-sign and execute the experiment, interpret the data, and drawconclusions.
The purpose of this paper is to provide an evidence-baseddescription of the student learning experiences in two types ofinquiry experiments that are part of a second-year undergrad-uate physics laboratory. In one experiment a detailed proce-dure was given, while the other experiment provided the sameapparatus and experimental questions, but asked students todevelop their own procedure through written prompts. Wefollowed the inquiry scale defined by Buck [13] and, althoughnot an exact match, the closest levels of inquiry are guidedand open inquiry, respectively. The guided-inquiry experi-ment relied heavily on a lab manual for procedures and anal-ysis. The open-inquiry experiment focused on experimen-tal design. While it included the problem with the onus onstudents to establish procedures and generate explanations oftheir data, prompts in the lab manual were used to scaffoldthe learning experiences. Both experiments focused on pro-cess rather than product and engaged students in the stages ofdesign, experimentation, and analysis, with varying levels ofsupport.
We begin by describing the course context for the guided-and open-inquiry experiments. The methodology section out-lines the research approach and methods for qualitative datacollection and analysis. The results discuss the major themesthat capture students’ learning processes and affective learn-ing behaviours during guided- and open-inquiry experimentalexperiences. The discussion section compares the differentstudent experiences during the two experiment types. Finally,we offer considerations for the design of physics experimentsbased on our findings.
II. COURSE CONTEXT
The second-year undergraduate physics laboratory atQueen’s University comprises eleven experiments whose sub-ject matter overlaps the syllabus of the second-year lecturecourses: electromagnetism, waves and vibrations, modernphysics, and dynamics.
The first three weeks of the twelve-week semester are de-voted to tutorial laboratories that each cover a specific AAPTlearning outcome [1] shown in parenthesis: keeping a labora-tory record in a Jupyter Notebook [14] (practical laboratoryskills), calculating experimental uncertainties using Pythonand the QExPy Python package [15] (analyzing data), mak-ing measurements with an oscilloscope (technical laboratoryskills) and the creation of reports using LATEX and Overleaf(communicating physics). In the remaining nine weeks thestudents have to complete six experiments and they do that ingroups of two. In each three-week period, the groups perform
two experiments and, in one of the weeks, they analyze theirresults or repeat measurements.
The lab manuals inherited from previous instructors werewell-written and contained step-by-step procedures in recipe-like [1, 16] format. Using the scheme introduced above, wewould classify them as guided-inquiry experiments [13].
As part of a TRESTLE (multi-institution consortiumfunded by the National Science Foundation) intervention [17]and laboratory-redesign project that took place between 2016and 2019, two of the experiments were redesigned and a newexperiment on coupled oscillators was designed and imple-mented. All three experiments required the students to prac-tice open inquiry. Although the experimental task was stilldefined by the instructor, the students were required to designtheir own procedures.
The experiments selected for redesign were Young’s mod-ulus and compact disk (CD) diffraction. In the Young’s mod-ulus experiment, the students measured the Young’s modulusof steel using both a static and a dynamic method. The CDdiffraction experiment, which is the focus of this paper, wasintroduced to the second-year laboratory course by AM, oneof the authors of this paper, circa 1993. The original ver-sion of this experiment was based on a short paper by Ket-tler [18]. Students were given: a helium-neon laser, a CD,and a copy of Kettler’s paper containing the grating equation.They were asked to estimate the grating spacing (also calledline spacing) of the CD as precisely as possible using theirown procedures. Consequently, in its original form, this wasan open-inquiry experiment. In subsequent years, other in-structors had extended the scope of the experiment by askingthe students to use their estimate of the grating spacing to es-timate the wavelength of a second laser, and the lab manualhad become more guided.
Re-conceptualizing guided-inquiry experiments as open-inquiry experiments can, in some cases, involve relativelyminor modifications to the lab manual. Frequently, detailedprocedural instructions are replaced with a request for the stu-dents to design their own strategy or to evaluate a number ofdifferent strategies and choose one. The apparatus, of course,is configured to support at least one experimental strategyand the configuration conveys information about how the in-structor expects the experiment to be done. Consequently, wefound it necessary to add degrees of freedom (DOF) to sup-port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distancebetween the CD and the wall where the diffraction pattern ismeasured, rotate the CD, rotate the laser, and also use a laserwith a different wavelength. To avoid overwhelming the stu-dents with choices, the lab manual describes the options thatare available and the instructor and teaching assistant can helpthe students make informed choices.
From the instructor’s perspective, the following differencesbetween the guided- and open-inquiry student experienceshave been noted: (1) Students performing open-inquiry ex-periments devote time at the beginning of the laboratoryperiod to the design of the experiment and, consequently,
3
they start taking measurements later than those performingguided-inquiry experiments. However, both groups finish theexperiment during the three-hour laboratory period. (2) Instudents’ reports who perform open-inquiry experiments, af-ter reflecting on different procedural approaches, they do sug-gest how the apparatus might be improved. In some cases wehave been able to implement these suggestions. (3) When stu-dents are given apparatus with multiple DOF they occasion-ally design experiments which combine DOF in ways thatwere not necessarily anticipated by the instructor. (4) Wefound that when students are given the choice of selectingan experiment for a final or culminating report (data from the2019 session), the open-inquiry experiments are selected, onaverage, more frequently than the other experiments.
III. METHODOLOGY
After redesigning the CD diffraction experiment, we hadtwo different lab manuals at our disposal. We have includedthem in the supplementary material [19]. Although theyshared the same experimental goals, and had the same intro-duction and theory sections, they embodied different levels ofstudent inquiry. The lab manual for the guided-inquiry exper-iment had a recipe-like procedure with step-by-step instruc-tions. In contrast, the lab manual for the open-inquiry ex-periment contained a section called “design activities” withprompts asking students to design their own activities. Ta-ble I compares the experimental sections and the correspond-ing learning outcomes for the guided- and open-inquiry ex-periments.
These manuals were used in the following fashion inthe second-year laboratory course in the winter semester of2017. Eight groups of students (in pairs) performed the open-inquiry experiment in the middle of the semester and anotherfive groups (in pairs) performed the guided-inquiry experi-ment towards the end of the semester. We designed this studythis way so that the students who did the open-inquiry experi-ment did not have access to the guided-inquiry lab manual.Each group was given three hours to complete the experi-ment. Ethics approval was obtained for the study and studentconsent was acquired for audio-recording their conversationsduring the experiment.
On average, it took 2 hours and 5 minutes for a group tocomplete the guided-inquiry experiment, while it took 2 hoursand 21 minutes for a group to complete the open-inquiry ex-periment. The audio data files were transcribed using a com-mercial software called Transcribe [20] by two senior physicsundergraduate research assistants with previous transcribingexperiences. The two students in all groups were anonymousand identified as Student A and Student B in the transcripts.The transcribers made sure that they were familiar with thestudents’ voices in their randomly assigned audio files be-fore they started transcribing. Each conversation was stampedwith a time corresponding to the start of the audio file. Aquality assurance check of the transcripts revealed their high
accuracy in reflecting the audio recordings.This paper focuses on the qualitative and quantitative anal-
ysis of the transcripts by addressing the following researchquestions:
A. How do student learning experience and behaviour dif-fer in experiments that allow for either guided or openinquiry?
B. What level of inquiry better reinforces student experi-mental design skills?
Thematic analysis, using the coding procedure describedby Corbin and Strauss [21], was carried out to analyze the au-dio transcripts using a general inductive approach [22]. Gen-eral inductive analysis refers to an approach that primarilyuses reading of data to derive themes through interpretationsof the data made by the researcher. Strategies used in this ap-proach include questioning what core meanings are evident inthe text in relation to the research questions. This is achievedby carrying out open, axial, and selective coding steps. Opencoding involves identifying phrases within text and creatingcodes for these. The codes are then grouped in the axial cod-ing phase to create categories. Finally, selective coding iscompleted when the main themes emerging from the cate-gories are created. The themes addressing the research ques-tions are the findings of the study. Emergent themes havebeen identified from data of each of the guided- and open-inquiry experiments. These themes, compared to the intendedphysics laboratory learning outcomes, provide an illustrationof what influence the two different experimental experienceshave on student learning.
The guided- and open-inquiry experiment transcripts wereanalyzed separately, that is, the analysis processes for one setof experimental data was kept distinct from the other, in aneffort to reduce bias during the coding process. Before anal-ysis began, we kept bracketing notes for awareness of per-sonal biases and research biases. Since the research aimsto compare the learning occurring during two experiments,we treated the guided-inquiry transcripts as the control dataset, and so this data set was analyzed first before the open-inquiry transcripts. LM, one of the authors, independentlycompleted the open coding of the first guided-inquiry tran-script while BC independently completed the open coding ofthe first 15 minutes of the same transcript. We compared anddiscussed our codes. LM wrote a description for all her opencodes to ensure an accurate understanding of the data and toensure future transcripts would be approached with a thor-oughly developed initial set of codes. BC adopted LM’s opencodes, continued coding part of the first guided-inquiry tran-script, compared with LM’s, and discussed the differencesuntil 100% agreement was reached. This process continueduntil coding of the first guided-inquiry transcript from the twocoders was completely agreed upon. LM then coded the 4 re-maining guided-inquiry transcripts. New codes were gener-ated and a few existing codes were modified when necessary.The same coding process was followed for the 8 open-inquirytranscripts.
Axial and selective coding to identify categories and
4
TABLE I. Comparison of guided- and open-inquiry manuals and learning outcomes for the CD Diffraction experiment. Students are providedwith recipe-like procedures in the guided-inquiry manual while they are prompted to design some of the experimental activities themselvesin the open-inquiry manual.
Guided-Inquiry Experiment Open-Inquiry ExperimentManual Learning Outcomes Manual Learning OutcomesMake sure that the CD is parallel tothe wall and the green laser isperpendicular to the CD; the laserlight will be reflected off the CDback into the laser.
Students will align theexperimental setup byfollowing instructions.
Measure all the physical quantities you needto predict what the diffraction patternshould look like with the CD in the verticalgeometry and the laser light incidenthorizontally.
Students will decide what physicalproperties to measure, make themeasurements, and use the results topredict where the diffracted light willintercept the wall.
Measure the distance from the CD tothe wall. Measure the 1st and 2ndorder diffraction spots.
Students will make accuratemeasurements of physicalquantities.
Establish procedures for ensuring that theCD is parallel to the wall and the laser lightis horizontal. Measure the diffractionpattern.
Students will design experimentalprocedures to align the apparatus,decide what quantities to measure,and make the measurements.
Calculate line spacing d. Students will calculate dusing experimental data.
Calculate d and the experimentaluncertainty for this single measurement.
Students will calculate d anduncertainties using experimental data.
Rotate the CD to a few differentangles and make measurements ofthe diffracted beams. Replace withred laser and acquire more datapoints.
Students will make use ofdifferent apparatus andadjust them to makemeasurements and compareresults.
Draft a plan that will allow you to find dwith higher precision. Identify the physicalquantities you will measure, think abouthow you will measure them, and carry outyour plan.
Students will reflect on their initialresults, design a procedure that willminimize the experimentaluncertainties, and carry out theirprocedure.
Use the re-arranged grating equationto plot the data and fit for d.Calculate the uncertainty in yourestimate.
Students will use curvefitting to improve theirexperimental results anduncertainties.
Make use of all your data points to calculated and your experimental uncertainty. Canyou identify any assumptions you havemade that might produce systematicuncertainties in your result?
Students will evaluate the systematicuncertainties of their experimentaldesign.
themes for the guided- and then open-inquiry transcripts fol-lowed the initial coding phase.
IV. RESULTS
We report our findings in two sections under qualitativeand quantitative results. Qualitative data analysis revealed themajor themes associated with student learning and behaviourduring the laboratory experience. Quantitative data analysiscaptured the number of categories, codes, and references ineach of the themes in the guided- and open-inquiry experi-ments. Comparisons between theses themes are presented inthe discussion section.
A. Qualitative Analysis Results
Table II depicts qualitative coding analysis results. Shownseparately are the guided-inquiry data set and the open-inquiry data set with emergent themes and the categories thatcomprise each theme.
1. Guided Inquiry
Four major themes emerged from the qualitative analysisof the guided-inquiry data: 1) Carrying out the experiment; 2)Interpersonal learning; 3) Sense making; and 4) The affectiveexperience.
“Carrying out the experiment” is intended to representstudents’ experiences of progressing through the steps pre-scribed by the lab manual until the end is reached. Studentbehaviours included constructing apparatus, making predic-tions, making measurements, calculations and visual observa-tions, and engaging in problem solving and project manage-ment. We point to the nature of students’ progression throughsteps in the guided-inquiry experiment, which was found tobe oriented toward reaching the end of the lab manual. Forexample, while students took measurements:
Alright do one more, I want to get out of here.In similar spirit,
I don’t like the [labs] that make you do the lab inthe lab.
Other students showed recognition of the value of laboratorytime, but remained focused on executing the bare-minimumsteps:
As much as I’d love to get home we should atleast make sure our measurements are kind ofprecise before we [leave].
Progressing from step to step, students continually refer to thelab manual for direction:
Let’s see what we have to do next,and
Alright, repeat with the other laser.“Interpersonal learning” represents students’ experiences
learning from and with others, including their laboratory part-ner, other groups, and the instructor or teaching assistant. Ex-periences related to interpersonal learning included needing
5
TABLE II. Themes and categories that emerged from qualitative coding analysis of the guided-inquiry data set and the open-inquiry data set.
Making predictionsMeasurement and calculation ProcessProblem solving Measurement and analysisProject managementVisual observation
Interpersonal learning
Needing help
Self and interpersonalexperiences
Physics language AttitudeInterpersonal interactionsPeer interaction
Student interaction with instructorInstructor interaction with studentsApplicationConfirmationLearningQuestion asking Comprehension
experience
LearningQuestion askingSense making
Sense making Sense makingSense making strategiesSense making of: calculations ormeasurements, instructions, observations,peer’s ideas or suggestions, the task, andphysics concepts
The affective experienceReactions Experimental design
experienceCritical thinkingDesignMarvelling
Emotions
help, using physics language, and a host of specific peer in-teraction modes (e.g., directing, suggesting, informing, coop-eration). Student learning from and with others is exempli-fied in the way that a student’s peers and instructors provideda source for their learning. Learning what n is (n is the nth
order diffraction and λ is the wavelength of the laser light), inthis conversation:
A: nλ divided by sine of the θ.B: What’s n?A: n is just our number, like diffraction gratingwhen . . . it’s one.B: So one λ?A: Yeah.B: Oh okay.
In another example, reconciling understanding of measure-ments:
A: And then we’re measuring N1 and N2, andthat’s the difference that is going to give us the Y.B: I think you’re thinking about measuring L, likethe distance to the horizontal. We did that once.But the distance to this point is going to changeevery time we change the angle right?
“Sense making” represents students’ experiences rational-izing, figuring out, or giving meaning to the task at hand.Students made sense of calculations or measurements, in-structions, observations, physics concepts, their peer’s ideasor suggestions, and of the task they were given. Applica-tion of knowledge gained at another time or place occurred
as part of the sense making experience, as did behaviours ofconfirmation, asking questions, and using other sense makingstrategies. Other sense making strategies used by students inthe guided-inquiry experiment included asking the instructoror teaching assistant for assistance, looking at the diagramin the lab manual, or drawing a diagram themselves. Audiorecordings from the laboratories include, for example:
I feel like I’d prefer to draw this . . . I just thinkbetter on paper I guess, I just have to look at thepicture really quick.
“The affective experience” represents students’ feelings,emotions, moods, and attitudes related to the laboratory expe-rience. Specifically, this experience includes reactions, mar-veling, positive emotions such as confidence, determination,excited, hopeful, and having fun, and negative emotions suchas careless, confusion, frustration, disappointment, sarcasm,self-doubt, and stress. Students’ expressions of confusion, forexample, relate to their understanding of variables:
I understand what they’re asking us to do withthis equation but I don’t understand how, howour n value is supposed to change,
and
I’m still confused about which one is the, like,zero order.
On the opposite side of the affective experience, students feltconfident in following the procedure:
6
Well I’m pretty confident in our procedure so Iguess we can just take these measurements prettyquick.
As demonstrated by the range of aspects that were part ofthe students’ affective experience, students completing theguided-inquiry experiment communicated a breadth of emo-tions.
2. Open Inquiry
Four major themes emerged from the qualitative analysisof the open-inquiry data: 1) Experimental process and com-ponents; 2) Self and interpersonal experiences; 3) Compre-hension experience; and 4) Experimental design experience.
“Experimental process and components” represents stu-dents’ experiences working in a non-stepwise process of ex-perimentation in the open-inquiry experiment. The natureof this experimental process and its components for studentswas task-oriented, meaning that they were given the task toachieve accurate measurements and their learning experienceand behaviour were oriented toward accomplishing the task.The nature of this process in the open-inquiry experimentdid not resemble sequential completion of steps, or an end-oriented experience. The process included learning experi-ence and behaviour related to measurement and analysis andthe overall process, which was categorized by the following:configuring laboratory station, observations, being off task,recording activity, referring to resources, safety, and trou-bleshooting. With this nature of experimental process andcomponents, students accomplished the task they were givenin the open-inquiry experiment. Students were interpretingtheir measurements:
I feel like this is going to be worse of a readingor even greater experimental error on our part,
andGot a perfect range of values.
Students also considered how to graph the data they gathered:I’m trying to think of all the things we could plot,
andI was trying to calibrate that, some sort of rela-tionship where d was the slope of something.
“Self and interpersonal experiences” are those that relateto students’ own attitudes and their interactions with others.The aspects of their experience that relate to attitude includeidentity, interest, marvelling, and wanting to finish the exper-iment. The aspects of their experience that relate to interper-sonal experiences include interaction with other groups, theinstructor or teaching assistant, and their peers. Students hada certain positionality or frame of attitude in the open-inquiryexperiment; for example, only one student remarked on want-ing to finish the experiment, while other students marvelled,making comments such as:
Whoa, what are you guys doing? That looks socool.
Another student, reflecting on their experience of the experi-ment, remarked that their group were
such scientists.In terms of interpersonal experiences, students’ interactionwith the instructor or teaching assistant varied from ask-ing questions, asking for help, and explaining their design.When the instructor approached the students, interactions in-cluded explaining concepts or the purpose of the experiment,checking in, answering questions, encouragement, guidance,and prompting. Interaction with other groups involved com-parison or gaining understanding from them. Peer interac-tion within the laboratory groups ranged widely and includedthese behaviours: clarification, informing, dismissing, sug-gesting, showing peer, planning, reassurance, and others re-lated to language and cooperation.
“Comprehension experience” represents students’ experi-ence of coming to understand the task at hand that is relatedto students’ comprehension or understanding that occurredvia learning, question asking, and sense making. An exampleof a student’s learning experience in the open-inquiry experi-ment is:
Oh wow that was so . . . now I understand what’sgoing on. Took me, like, the first hour to figureout what we were, like, the theory behind all ofthis stuff. Once I understood . . .
Such learning experiences included reflection on the experi-ment design or process. For example,
A: We can just start trying things and then see,but I feel like . . .B: I feel like the original method we did waspretty good.
Questions that students asked ranged in nature, from calcu-lation and measurement questions to design questions andnext steps questions. Other types included questions relatedto data collection, using equipment, observations, and under-standing. Understanding question examples include:
How do lasers work?How does this give us the average of d?
andIs the . . . how is the . . . line spacing directly pro-portional to the distance between the diffractionlines?
Design question examples include:Okay, so what can we change?Why does it even have to be level, what differencedoes it make?
andHow did we know that this is going to hit hereand then go directly to the center, like it hit thecenter of this CD?
7
Sense making experiences included topics similar to thosethat students asked questions about, such as calculations,data, measurements, observations, and instructions. How-ever, students also made sense of diagrams, equations, tasks,and the apparatus.
“Experimental design experience” represents the students’intentional consideration, creation, and execution of plansin the open-inquiry experiment. These experiences weregrouped into two categories: critical thinking and design.Critical thinking behaviours are exemplified by commentssuch as:
Okay, so. Rotating the CD, how does that affect. . . that affects θ too, right? So let’s try this out,
andIf you look at this, the laser light is pointing al-most right back into the laser itself, right? Sothat would indicate, if this is already level, thenthat should be.
Design experience and behaviour were further grouped intotwo categories: constructing apparatus and designing experi-ment. Constructing apparatus involved making assumptions,being resourceful, concern, designing the apparatus, makingadjustments, trial, and using equipment. Designing exper-iment involved a variety of design-oriented learning expe-rience and behaviour, including foreseeing issues, decision-making, changing directions, improvement ideas, makingpredictions, interpreting, and testing these, and identifyingtasks, a goal or purpose, and known and unknown informa-tion. The following quotation broadly exemplifies the open-inquiry student experience of designing the experiment:
So I’m thinking we can . . . do several differenttrials by using different distance from the mirrorto the wall.
B. Quantitative Analysis Results
We quantified the codes, categories, and themes for eachof the guided- and open-inquiry data sets to broadly repre-sent the analysis results. Table III and IV show the numberof categories, codes, and references in each of the themes inthe guided- and open-inquiry experiments. These numbersmay provide insight about the differences in the nature of stu-dents’ learning experiences in the two different experiments.Student experiences in the guided-inquiry experiment spanalmost three times the number of categories when comparedto those in the open-inquiry experiment. Conversely, whenexamining by the smallest unit, the number of codes, studentexperiences in the open-inquiry experiment are representedby more than ten percent more codes when compared to thosein the guided-inquiry experiment. In the discussion sectionwe explore how these initial differences may suggest a moreconcentrated, deep-level experience in the open-inquiry ex-periment compared to that in the guided-inquiry experiment.
TABLE III. The number of categories, codes, and references in eachof the themes in the guided-inquiry experiment. Note that there are5 guided-inquiry data sets from the 5 groups of students who did theguided-inquiry experiment.
Guided-Inquiry Theme No.Categories
No.Codes
No.References
Carrying out the experiment 6 39 890Interpersonal learning 5 23 553Sense making 12 21 395The affective experience 3 16 130Total for guided inquiry 26 99 1968
TABLE IV. The number of categories, codes, and references in eachof the themes in the open-inquiry experiment. Note that there are8 open-inquiry data sets from the 8 groups of students who did theopen-inquiry experiment.
Open-Inquiry Theme No.Categories
No.Codes
No.References
Experimental process andcomponents 2 32 708
Self and interpersonalexperiences 2 31 366
Comprehension experience 3 26 353Experimental design experience 2 23 444Total for open inquiry 9 112 1871
As for the remaining quantitative analysis results, we havecreated plots to depict the frequencies of themes, categories,and codes for both the guided-inquiry and open-inquiry ex-periments. Figure 1 shows the frequency of each theme basedon the average number of references (coded segments of rawtext) contributing to it, calculated by dividing the total num-ber of references across all the codes in the theme by the num-ber of the corresponding data sets in the guided-inquiry oropen-inquiry experiment.
Figure 2 shows in more detail all the categories and codesin each theme with the frequency of the corresponding theme,category, and code represented by the area of the circle. Tointerpret the relative frequency at the level of codes, for ex-ample, readers can compare the areas of red circles. In an ef-fort to ensure the figure’s readability, we provide an example:“Asking for help,” a code within the category “Peer interac-tion” and the theme “Interpersonal learning,” is the smallestred circle among its fellow red circles. This means that “Ask-ing for help” was least frequently coded for during analysis incomparison to other codes in this category. Continuing withthe example, “Deliberation,” seen to the left of “Asking forhelp,” was more frequently coded for.
By depicting the qualitative analysis in terms of frequencyof codes, we are able to visually represent the student learn-ing experience and behaviour in each of the guided- andopen-inquiry experiments. In the guided-inquiry experiment,student learning experience and behaviour were most fre-quently coded as “Carrying out the experiment,” within whichstudents most frequently engaged in measurement and cal-
8
The affective experience
Sense making
Interpersonal learning
Carrying out the experiment
26
79
111
178
0 50 100 150 200
Average number of references
Experimental design experience
Comprehension experience
Self and interpersonal experiences
Experimental process and components
56
44
46
88
FIG. 1. Histograms depicting the frequency of experience acrossthemes in each experiment. Shown are the average number of ref-erences in each theme in the guided-inquiry experiment (top) and inthe open-inquiry experiment (bottom). The frequency is calculatedby dividing the total number of references in each theme by the num-ber of the corresponding data sets in the guided-inquiry (5 data sets)or open-inquiry experiment (8 data sets).
culation, and to lesser frequencies in constructing appara-tus, project management, and problem solving. The secondhighest frequency theme is “Interpersonal learning,” with thehighest frequency category being peer interaction, followedby other similarly lower frequency categories such as instruc-tor interaction with students, student interaction with instruc-tor, needing help, and physics language. The third highest fre-quency theme is “Sense making,” with its highest frequencycategory question asking, and lower frequency categories in-cluding sense making of calculations or measurements, ofinstructions, and using sense making strategies. The leastfrequent theme in the guided-inquiry experiment is “The af-fective experience,” with the high-frequency category beingemotion (including a variety of types such as confusion, frus-tration, and confidence) and the lower frequency category be-ing marvelling.
In the open-inquiry experiment, student learning experi-ence and behaviour were most frequently coded as “Experi-mental process and components,” within which students mostfrequently engaged in measurement and analysis and less fre-quently in process, which includes frequent experience andbehaviour such as referring to the lab manual, challenges, andobservations. The second highest frequency theme is “Ex-perimental design experience,” with the highest frequencycategory being design. Design includes a variety of expe-rience and behaviour, most frequent of which include usingequipment, making adjustments, next steps, and improve-ment ideas. The only other category in this theme is criti-cal thinking. The third highest frequency theme is “Self andinterpersonal interactions.” The higher frequency categoryin this theme is interpersonal interactions, which includesnumerous related behaviours such as instructor interactionwith students, interaction with other groups, and informing(peers). The least frequent theme in the open-inquiry experi-
ment is “Comprehension experience,” within which studentsmost frequently engaged with sense making, and to lesser fre-quencies in question asking and learning.
V. DISCUSSION
Overall, the guided- and open-inquiry experiments each re-vealed four different themes, which allowed us to understandthe student learning behaviours in each of these experiencesas unique. The extent to which and how the nature of theexperiences differed was challenging to capture. In an at-tempt to illuminate the differences of student learning expe-rience and behaviour between the two forms of inquiry andthe benefits to students’ learning in the open-inquiry exper-iment, we discuss how the themes compare between the ex-periments, including discussions about common and uniquecodes in each experiment.
A. Comparing Number of Codes, Categories and Themes
The number of codes, categories, and themes resultingfrom analysis of each experiment suggest a difference inthe nature of students’ learning experience and behaviourin the two different experiments. As reported earlier, theguided- and open-inquiry experiments both yielded 4 themes.The guided-inquiry experiment yielded 26 categories and 99codes, whereas the open-inquiry experiment yielded 9 cate-gories and 112 codes. What we observe by comparing thenumber of codes and categories in the open-inquiry exper-iment is that a high number of codes (112) reduced to alow number of categories (9) shows high instances of expe-rience and behaviour occurring at a low variety. What weobserve by comparing the number of codes and categoriesin the guided-inquiry experiment is that a high number ofcodes (99) reduced to a medium number of categories (26)shows high instances of experience and behaviour occurringat a high variety. Based on these results we suggest that theopen-inquiry experiment allowed students to have more con-centrated, deep-level experiences compared to those in theguided-inquiry experiment.
To provide an example that may illustrate our assertion, wecompare the guided-inquiry theme “Carrying out the experi-ment” and the open-inquiry theme “Experimental process andcomponents.” Within the former, there are 6 categories (indescending frequency: measurement and analysis, construct-ing apparatus, project management, problem solving, visualobservation, and making predictions) that describe how stu-dents carried out the experiment. These categories furtherdivide into 39 codes. Within the open-inquiry theme “Exper-imental process and components,” there are two categories(in descending frequency: measurement and analysis, andprocess) that describe students’ experimental process. Thesecategories further divide into 32 codes. The two themes con-tain similar number of codes, although in the open-inquiry
9
Carrying outthe experiment
Constructingapparatus
Makingpredictions
Measurementand calculation
Problemsolving
Projectmanagement
Visualobservation
Constructingapparatus Being
resourceful
Designingapparatus
Makingadjustments
PlanningTrial
Usingequipment
Calculation
Calculationcomparison
Data analysis aids
Assumptions
Caution
Measurementcomparison
Concern
Double-checkingmeasurements
Estimatinguncertainty
Interpretingmeasurements
Makingmeasurements
Math
Recordingactivity
CopyingResults
Using JupyterNotebook
Graphingdata
Challenges
Checking
IdentifyingneedIdentifying
problem
Success
Unsuccessful
Off task
Referring tojournal article
Referring tolab manual
Safety
Timemanagement
Organizingtasks
Progress
Interpersonallearning
Instructor interactionwith students
Needinghelp
Peer interaction
Physicslanguage
Student interactionwith instructor
Needinghelp
Employing otherstudents’ help
Peer interaction
Apologizing
Cooperation
Agreement
Asking forhelp
DeliberationDisagreement
Doubting orchecking
Sharingtasks
Correcting
Directing
Expressingthinking
Informing
Observation
Reassuring
Reconcilingunderstanding
Reflection
Suggesting
Sensemaking
Sensemaking
Application
Confirmation
Learning
Of calculationsor measurements
Of instructionsor resources
Of observations
Of peer’s ideaor suggestion
Of thetask
Physics concepts
Questionasking
Sense makingstrategies
Questionasking
Askingself
Calculationquestion
Data usequestion
Designquestion
Equipmentquestion
Measurementquestion
Procedurequestion
Understandingquestion
Visual observationquestion
The affectiveexperience
Emotions
Marvelling
Reactions
Careless
Confusion
Frustration
Making mistakes(disappointed)
Sarcasm
Self doubt
StressConfidence
Determination
Excited
Having fun
Hopeful
Marvelling
Relevance ofphysics
Interest
Experimental processand components
Process
Measurementand analysis
Configuringlab station
Observations
Off task
Recordingactivity
Saving orsharing documents
Using JupyterNotebook
Referring toassessment rubric
Referring tojournal paper
Referring tolab manual
SafetyTrouble-shooting
Challenges
Identifyingproblem
Problemsolving
Success
Calculation
Significantdigits
Unitconversions
Data analysisaids
Dataorganization
Difficultyusing aids
Graphingdata
Caution
Doublechecking
Estimatinguncertainty
Identifyingmeasurements
Interpretingmeasurements
Makingmeasurements
Measurementlimitations
Results
Discussingresults
Humanerror
Self and interpersonalexperiences
Attitude
Interpersonalinteractions Identity Interest
Marvelling Wanting tofinish lab
Instructor answeringquestions
Instructorchecking in
Instructor encouragement
Instructor explainingconcepts
Instructorexplaining
lab purpose
Instructorguidance
Instructorprompting
Interaction withother students
Comparison toother students
Gaining understandingfrom other students
Peer interactionClarification
CooperationHelping one
another Dismissing
Informing
Physicslanguage
Planning
Allocatingresponsibility
Planning forreport writing
Timemanagement
Reassuring
Showing peer
Suggesting
Student asking for help
Student asking questions
Student explaining design
Comprehensionexperience
Learning
Questionasking
Sensemaking
Learning
Reflection
Reflection on designor process
Reflection onlaboratory experience
Reactions
Calculationquestion
Data collectionquestion
Designquestion
Measurementquestion
Next stepsquestion
Visual observationquestion
Understandingquestion
Equipmentquestion
Application
Prior physicsknowledge
Confusion
Sense makingstrategies Of
apparatusOf calculations
or measurements
Of data
Of diagram
Of equations
Of instructionsor resources
Of observations
Of thetask
Purpose oflab experiment
Experimental designexperience
Criticalthinking
DesignAssumptions
Beingresourceful
Concern
Designingapparatus
Makingadjustments
Trial
Usingequipment
DesigningexperimentChanging
directions
Decisionmaking
Achieving bestaccuracy
Reasoning
Foreseeing issues
Identifying a goalor purpose
Identifyingknown
information
Identifyingtasks Identifying
unknowninformation
Improvementideas
Interpretingpredictions
Making predictions
Next steps
Testing predictions
Guided-Inquiry Experiment Open-Inquiry Experiment
FIG. 2. This figure represents the complete picture of the our qualitative and quantitative analysis results. All categories and codes in eachtheme are shown for the guided-inquiry experiment (left) and for the open-inquiry experiment (right). The area of each node scales withthe frequency of the theme (green), category (blue), and code (red). When a circle is seen half filled with blue and half filled with red, thisis an indication that the circle represents a term that is both a category and code. An example of this is “Physics language,” seen in the“Interpersonal learning” theme in the guided-inquiry experiment on the left.
10
experiment the students’ learning experience and behaviourcould be grouped into just two categories, only one-third ofthe number of categories in the guided-inquiry experimentwhen comparing these themes. Therefore, we assert that stu-dents had a different experience in the open-inquiry experi-ment than in the guided-inquiry experiment; their inquiry ismore concentrated in focus, which we attribute to the openlevel of inquiry.
B. Frequency Observations
To further our above assertion and suggest that the open-inquiry experiment is more supportive of students’ experi-mental design skills than the guided-inquiry experiment, wedescribe observations made of the frequency of themes (seeFigure 1). Student’s experiences in the open-inquiry experi-ment are most frequently experimental process- and design-related, whereas in the guided-inquiry experiment, student’sexperiences are most frequently related to carrying out the ex-periment and interpersonal learning. In addition, the themesaligning more closely with the AAPT’s guidelines for design-ing experiments are the open-inquiry experiment themes. Itsuggests that the open level of inquiry provides students witha design skill-reinforcing experience more so than the guidedlevel of inquiry. Interestingly, the average total referencesfor the open-inquiry experiment (234) is lower than that forthe guided-inquiry experiment (394), which may suggest stu-dents simply talked less and engaged in more thinking or do-ing. While it was beyond the scope of this paper, we suggestthese findings of relative quietness in the open-inquiry exper-iment could support students’ exploration, engagement, andthe experience of design.
We also compared student’s experiences in each experi-ment by the least frequent themes. The single-lowest fre-quency theme in the open-inquiry experiment was “Com-prehension experience,” which is only slightly less frequentthan the second-lowest frequency theme, “Self and inter-personal experience”; however, the lowest frequency themein the guided-inquiry experiment is “The affective experi-ence,” which is less than half as frequent as the second-lowesttheme, “Sense making.” These findings suggest a more bal-anced experience across themes in the open-inquiry exper-iment than in the guided-inquiry experiment. In terms ofhelping students meet learning outcomes intended to developtheir design skills, we find a more balanced laboratory expe-rience to be favourable because not one learning experienceor behaviour is favoured too highly over other supporting be-haviours. As is the case in the guided-inquiry experiment,the frequency of students’ experiences was concentrated on“Carrying out the experiment” while the frequency of otherexperience and behaviour suffered in comparison.
The idea that certain behaviours or learning experiencesmay happen in support of or at the expense of others is aninteresting topic. We have observed “The affective experi-ence” as part of the guided-inquiry experience, albeit the least
frequent theme, as a possible hindrance to students’ experi-encing other themes more frequently. Because of the natureof the affective experience which is unique to the guided-inquiry experiment, with students’ two most frequent emo-tions being confusion and frustration, it is possible that theaffective experience hindered students from having an expe-rience more frequently design-oriented (or otherwise). Theresults relating to theme frequency as they are, however, dosuggest that students’ experiences in the open-inquiry exper-iment are more frequently design-oriented compared to theguided-inquiry experiment and therefore we advise that theopen level of inquiry better reinforces students’ design skills.
C. Common Codes Across Themes
We examine the common codes that emerged in bothguided- and open-inquiry experiments in this section, i.e.,those that represent a specific learning experience and/or be-haviour that occurred for students in both data sets. We ex-plore interpretations of such results as a means to address howstudent learning experience and behaviour differ in guided-and open-inquiry experiments; this is our first research ques-tion. The common codes emerged in both experiments asexpected since a) the students did the CD diffraction exper-iment with the same set of provided apparatus; b) there arecommon learning outcomes shared within the two levels ofinquiry for this experiment as seen in table I. We list in detailthese common codes in table V with their frequencies of be-ing referenced in the corresponding themes in the two levelsof inquiry.
Most of the common codes emerged in the “Carry out theexperiment” theme in the guided-inquiry experiment, whichis also the theme that has the largest number of codes and ref-erences. These common codes are either in the “Experimen-tal process and components” or the “Experimental design ex-perience” theme in the open-inquiry experiment, except thatthe “Planning” code is in the “Self and interpersonal experi-ence” theme. Some of these common codes have similar fre-quencies in the two experiments while many have large dif-ferences in frequencies. For example, students made similaramount of efforts in “Being resourceful,” “Designing appara-tus,” making “Trial,” “Using equipment,” and “Double check-ing” measurements in the two experiments, and often talkedabout “Safety.” Notably, students did a lot more “Makingadjustments,” “Calculation,” “Estimating uncertainty,” “Mak-ing measurements,” “Identifying problem,” and ”Refering tolab manual” throughout the guided-inquiry experiment com-pared to the open-inquiry experiment. They also encounteredmore “Challenges” and went “Off task” a lot more often. Onthe other hand, students in the open-inquiry experiment didmore “Making predictions” and “Using Jupyter Notebook,”and used “Data analysis aids” more often than those in theguided-inquiry experiment.
Some common codes emerged both in the “Interpersonallearning” theme in guided inquiry and in the “Self and inter-
11
TABLE V. A list of the 52 common codes that emerged from qualitative coding analysis of the guided-inquiry data set and the open-inquirydata set. We list the 4 themes from the guided-inquiry experiment first, followed by the 4 themes from the open-inquiry experiment. Thenumbers shown in the table are how many times each code is referenced in the corresponding experiment and how many files each code isreferenced (shown in parentheses). Note that the total number of guided- and open-inquiry files is 5 and 8, respectively.
personal experiences” theme in open inquiry. Many of thestudent conversations in guided inquiry fell into the “Peer in-teraction” category and the students were most often “Inform-ing” and “Suggesting” ideas to each other. The natures of theconversations in open inquiry were quite different and wereoften more design based. Therefore many of the conversa-tions moved away from simply “Informing” or “Suggesting”and fell into the unique codes in the “Design” category inthe “Experimental design experience” theme, discussed in thenext section.
We also saw that students in guided inquiry had more ques-tions (a total of 227 references in 5 data sets) than thosein open inquiry (a total of 133 references in 8 data sets).The common codes in the “Sense making” theme in guidedinquiry that are more frequent than those in the “Compre-hension experience” theme in open inquiry are: “Calcula-tion question,” “Design question,” “Measurement question,”“Visual observation question,” “Sense making of calculationsor measurements,” and “Sense making of instructions or re-sources.” On the other hand, the “Equipment question” codewas much more frequent in open inquiry.
Lastly, the “Reactions” and “Confusion” codes emergedmuch more frequently in the “The affective experience”theme in guided inquiry than in the “Comprehension expe-rience” theme in open inquiry. For example, there were atotal of 35 references of “Confusion” in the 5 guided-inquirydata sets, and 25 references in the 8 open-inquiry data sets, in-dicating that students on average experienced confusion lessfrequently in the open-inquiry experiment.
D. Unique Codes Across Themes
Many unique codes emerged in either the guided- or open-inquiry experiment that highlight the differences in these twoexperiments. We have listed the 47 codes that are uniquein the guided-inquiry experiment and the 60 codes that areunique in the open-inquiry experiment in table VI and ta-ble VII.
There are 11 unique codes in the “Carrying out the experi-ment” theme in guided inquiry, with the most frequently ref-erenced ones being “Measurement comparison,” “Checking,”and “Visual observation.” There are 14 unique codes in the“Experimental process and components” theme in open in-quiry and the most frequent ones are “Observations,” “Identi-fying measurements,” and “Measurement limitations.” “Ob-servations” include visual observations in the experiment aswell as other general observations. “Identifying measure-ments” and “Measurement limitations” are unique in openinquiry, which indicates that this experiment successfullyprompted the students to think often as well as deeply aboutthe measurements they should make and the experimentallimitations of their measurements. We did not see these stu-dent behaviours in the guided-inquiry experiment.
There are a total of 16 unique codes in the “Interpersonallearning” theme in guided inquiry, out of which 12 unique
codes appeared in the “Peer interaction” category. The in-teractions in the guided-inquiry experiment were often be-tween the two students performing the experiment and thestudents were “Apologizing” to each other, being in “Agree-ment” with one another, “Doubting or checking,” “Correct-ing,” “Directing,” “Expressing thinking,” making “Observa-tion,” and “Reconciling understanding.” This is consistentwith the students more frequently “Informing” and “Suggest-ing” to each other in the guided-inquiry experiment, shownin the common codes table V. There are a total of 21 uniquecodes in the “Self and interpersonal experiences” theme inopen inquiry, out of which 19 unique codes appeared in the“Interpersonal interactions” category. The most frequent onesinclude “Instructor answering questions,” “Instructor check-ing in,” “Instructor guidance,” “Instructor prompting,” andstudents seeking “Clarification” and “Helping one another.”We saw richer and more in-depth interactions in the open-inquiry experiment between the instructor and students, andbetween students in different groups.
There are 8 unique codes in the “Sense making” themein guided inquiry including “Confirmation” and “Procedurequestions” shown in table VII. In open inquiry there are a to-tal of 10 unique codes in the “Comprehension experience”theme, including “Reflection on design or process” and “Re-flection on laboratory experience” in the “Learning” category.Here is an example of “Reflection on design or process”:
A: Well the other one is still valid though. It’sstill a valid method, we’re just taking the nextone, which makes sense why it’s not exactly.B: No no, I feel like, well our L is technically, ourL is actually what we wrote as d. L, it seems likeit’s the distance.
And an example of “Reflection on laboratory experience” is:
A: It really doesn’t seem like it’s gonna be verylong.B: No [be]cause I think a lot of like the lab isspent like kind of figuring out what to do.A: Yeah.B: So then your actual experiment is like if youcame back in and knowing exactly what to do.
Finally, there are 12 unique codes in the “The affective ex-perience” theme in guided inquiry all in the “Emotions” cat-egory except “Relevance of physics.” The negative emotionsinclude “Careless,” “Frustations,” “Making mistakes (disap-pointed),” “Sarcasm,” “Self doubt,” and “Stress.” The posi-tive emotions are “Confidence,” “Determination,” “Excited,”“Having fun,” and “Hopeful.” The most frequent emotion thestudents experienced was “Frustration,” for example,
We should have done the green and like the ad-justing, then we should have done the red, whydidn’t they just tell us to do that. You know whatI mean?
We did not see any of these emotions in open-inquiry exper-iment. Instead, we saw many unique codes in the “Exper-
13
TABLE VI. Open codes that are unique in the “Carrying out the experiment” and “Interpersonal learning” themes in the guided-inquiry dataset in comparison to the “Experimental process and components” and “Self and interpersonal experiences” themes correspondingly in theopen-inquiry data set. Note that there are a total of 5 guided-inquiry files and 8 open-inquiry files.
Planning for report writing 1 (1)Showing peer 8 (5)Student asking for help 6 (5)Student asking questions 5 (4)Student explaining design 4 (2)
imental design experience” theme including “Critical think-ing” and 15 codes in the “Design” category. Students didsome “Decision making”:
Okay so 10 degrees . . . What should we do, 10?Do something a bit more. Let’s do a 5 degree oneas well . . .
They were “Identifying tasks”:
A: We need . . .B: Oh the angle measurement, that’s the otherthing we need.A: Angle between that and the wall?B: Well we can do the angle between this and thevertical.
14
TABLE VII. Open codes that are unique in the “Sense making” and “The affective experience” themes in the guided-inquiry data set incomparison to the “Comprehension experience” and “Experimental design experience” themes correspondingly in the open-inquiry data set.Note that there are a total of 5 guided-inquiry files and 8 open-inquiry files.
The students were talking about “Improvement ideas”:
Um maybe it, maybe the red [laser] will be bet-ter, or maybe if it was closer [than] we can see.
And “Next steps”:
We’re gonna be able to get a bunch of data. Andwe’re gonna know like for each data point liken, the wavelength, sin θn, and sin θ0, and thenthat’s what’s gonna give us d.
E. Mapping onto Learning Outcomes and AAPT’s Guidelines
Our findings are consistent with the intended laboratorylearning outcomes we outlined in table I. Specifically, boththe guided- and open-inquiry experiments asked the studentsto align their apparatus, make measurements and calculations,and estimate uncertainties. These learning outcomes are re-flected in the common codes that are present in both exper-iments: “Making adjustments,” “Using equipment,” “Mak-
ing measurements,” “Calculations,” and “Estimating uncer-tainty.” In addition, the open-inquiry experiment promptedthe students to make predictions, decide what physical quan-tities to measure, and design experimental procedures. Thesewere reflected in the 27 references in the “Making predic-tions” code, 64 references in the unique “Identifying mea-surements” code, and a total of 222 references in the 15unique codes in the “Experimental design experience” theme,respectively.
Furthermore, the AAPT guidelines recommend that stu-dents should be able to design a procedure to make a mea-surement, should have a hands-on opportunity to constructan apparatus, should do basic troubleshooting, should under-stand the limitations of their experimental design, and reflecton their results and suggest ways to improve their design. Wesaw in our analysis results that, while the guided-inquiry ex-periment addressed some of these recommended learning out-comes, the open-inquiry experiment allowed a much deeperand broader coverage of all aspects of these recommenda-tions.
15
VI. CONCLUSION
In this study we analyzed audio transcripts of students’conversations that took place while they were performing ei-ther a guided-inquiry experiment, in which procedures wereprovided, or an open-inquiry experiment, in which studentswere required to design their own procedures. We comparedstudent learning experience and behaviour for both inquirylevels, by studying the results of qualitative and quantitativeanalysis of the transcripts, and found many differences.
First, we found that students in the guided-inquiry experi-ment, with the ultimate goal of completing all the prescribedtasks, focused on following the detailed instructions that wereprovided as evidenced by their frequent referencing of thelab manual. In contrast, the open-inquiry experiment pro-vided the students with the freedom to explore a range of ap-proaches and design their own experimental procedures.
Second, we saw richer and more in-depth interactions inthe open-inquiry experiment. Interactions during guided in-quiry were generally between students who asked each otherquestions and told each other what to do. In open inquiry,there were more interactions between the instructor and stu-dents, and between students in different groups, and the con-versations focused more on procedural design.
Third, students in guided inquiry expressed many emo-tions, often negative ones including frustration and confusion.This was somewhat surprising since the students were givendetailed procedures in the lab manual and should know verywell what to do in the next steps. We argue that the nega-tive emotions could be correlated with our first finding thatthe students came into the laboratory with the mindset thatthey would be able to carry out the experiment by simplyfollowing the instructions. When they experienced techni-cal or other unexpected problems, they often reverted to ex-press negative emotions including frustration and confusion.In comparison, students in open inquiry did not express anyfrustrations and they showed confusion less frequently. Thesestudents seemed to have come into the laboratory knowingthat the lab manual would not give them all the answers andthey were expected to figure out the next steps. With thismindset, these students in general had more positive experi-ences than those in guided inquiry.
Finally, the students in open inquiry had more opportuni-ties to develop their experimental design skills. Many uniquelearning experience and behaviour emerged in the “Compre-hension experience” and “Experimental design experience”themes, which indicate a richer and more comprehensive de-sign and learning experiences for these students. Althoughthe open-inquiry experiment did not ask the students to designall aspects of this experiment, the tasks required the studentsto develop skills in one of the core AAPT curriculum areas,experimental design. It provided an opportunity for studentsto configure their apparatus, troubleshoot their apparatus andmethod, reflect on their results, and evaluate their procedure,and consequently they started to think like a physicist.
We find that student learning experience and behaviourin physics undergraduate laboratory experiments can be sig-
nificantly improved by increasing the level of inquiry fromguided to open. In some cases, the experimental apparatusdoes not have to be changed; it is sufficient to replace recipe-like procedures with questions or prompts that give studentsthe freedom to design some of the experimental activitiesthemselves. Alternatively, students can be given an experi-mental problem or task and the apparatus can be configuredto accommodate a variety of experimental approaches. Usingthe analogy from mechanics introduced above, these experi-ments can be considered to have multiple DOF from whichthe students have to choose one. In the experiment describedhere, students had to design a strategy that resulted in the bestexperimental precision they could achieve, after a consider-ation of the experimental uncertainties and the available ap-proaches.
The increased level of inquiry promotes AAPT learningoutcomes and has the beneficial side-effect of reducing neg-ative affective experiences for students. An important find-ing being that detailed experimental procedures can, counter-intuitively, be the source of the negative affective experiences.
We suggest that when instructors are designing open-inquiry laboratory experiments or transforming a guided-inquiry experiment into an open-inquiry experiment, theyconsider whether instructions could be turned into questionsor prompts. Rather than telling the students what to do, stu-dents can be given the freedom to evaluate a number of exper-imental options. Clearly, for this to be successful, the appara-tus has to support more than one experimental strategy. It isnot always necessary to ask the students to design a completeexperimental procedure to raise the inquiry level. Neverthe-less, the instructor has to find a manageable balance betweenthe amount of detail given in the lab manual and the numberof design tasks that the students are required to execute.
This study describes the positive effect of increasing theinquiry level of undergraduate physics experiments on stu-dent behaviour and learning. Moreover, it provides guidelineson how to best design or redesign undergraduate experimentsto support open-inquiry, which we have posited as a way oftransferring agency from the instructor to the student. How-ever, we do recognize that a study of this sort has limitationsthat, for completeness, we delineate here: 1. It was basedon a specific experimental topic on CD diffraction; 2. Thestudents in this study were from a second-year undergraduatelaboratory course in one research institution, therefore maynot represent those who are new to universities or more se-nior in their undergraduate studies; 3. The students who didthe guided-inquiry and open-inquiry experiments are differ-ent and randomly selected from the class, which may not rep-resent a general population; and 4. While we categorized ourexperiments on two specific inquiry levels, we recognize thateach experiment remains unique. The design of each experi-ment must retain some degree of flexibility to support studentfamiliarity with the topic and expertise with a specific exper-imental process. As a result, the level of instructor guidance,in terms of the number and type of written lab manual andin-lab prompts, may shift accordingly.
16
ACKNOWLEDGMENTS
This work was funded by: the TRESTLE network [17](NSF DUE1525775); the Department of Physics, Engineer-
ing Physics and Astronomy; the Center for Teaching andLearning; and the Faculty of Engineering and Applied Sci-ence at Queen’s University. We would like to thank: C. Knap-per, R. Knobel, G. L. McLean and M. Swarthout for criticalreadings of this manuscript.
[1] D. MacIsaac, Report: AAPT recommendations for the Under-graduate Physics Laboratory Curriculum, The Physics Teacher53, 253 (2015), https://doi.org/10.1119/1.4914580.
[2] J. Fraser, A. L Timan, K. Miller, J. Dowd, L. Tucker, andE. Mazur, Teaching and physics education research: bridg-ing the gap, Reports on progress in physics, Physical Society(Great Britain) 77, 032401 (2014).
[3] F. Aparicio-Ting, D. Slater, and E. Kurz, Inquiry-based learn-ing (IBL) as a driver of curriculum: a staged approach, Paperson Postsecondary Learning and Teaching 3, 44 (2019).
[4] D. R. Dounas-Frazer, J. T. Stanley, and H. J. Lewandowski,Student ownership of projects in an upper-division opticslaboratory course: a multiple case study of successful ex-periences, Physical Review Physics Education Research 13,020136 (2017).
[5] D. B. Luckie, J. J. Maleszewski, S. D. Loznak, and M. Krha,Infusion of collaborative inquiry throughout a biology curricu-lum increases student learning: a four-year study of “teams andstreams”, Advances in Physiology Education 28, 199 (2004).
[6] J. Arantika, S. Saputro, and S. Mulyani, Effectiveness ofguided inquiry-based module to improve science process skills,Journal of Physics: Conference Series 1157, 042019 (2019).
[7] S. J. Husnaini and S. Chen, Effects of guided inquiry vir-tual and physical laboratories on conceptual understanding, in-quiry performance, scientific inquiry self-efficacy, and enjoy-ment, Physical Review Physics Education Research 15, 010119(2019).
[8] N. G. Holmes and D. A. Bonn, Quantitative comparisons topromote inquiry in the introductory physics lab, The PhysicsTeacher 53, 352 (2015).
[9] D. D. Minner, A. J. Levy, and J. Century, Inquiry-based sci-ence instruction-what is it and does it matter? Results from aresearch synthesis years 1984 to 2002, Journal of Research inScience Teaching: The Official Journal of the National Asso-ciation for Research in Science Teaching 47, 474 (2010).
[10] E. M. Furtak, T. Seidel, H. Iverson, and D. C. Briggs, Exper-imental and quasi-experimental studies of inquiry-based sci-ence teaching: a meta-analysis, Review of educational research82, 300 (2012).
[11] R. Spronken-Smith and R. Walker, Can inquiry-based learn-ing strengthen the links between teaching and disciplinary re-search?, Studies in Higher Education 35, 723 (2010).
[12] E. M. Furtak, R. J. Shavelson, J. T. Shemwell, and M. Figueroa,The journey from child to scientist: Integrating cognitive devel-opment and the education sciences, edited by S. M. Carver andJ. Shrager (Washington, DC: American Psychological Associ-ation, 2012) pp. 227–244.
[13] L. B. Buck, S. L. Bretz, and M. H. Towns, Characterizing thelevel of inquiry in the undergraduate laboratory, Journal of Col-lege Science Teaching 38, 52 (2008).
[14] Jupyter Notebook: an open-source programming languageused in the laboratory course in this study, https://jupyter.org/.
[15] R. Martin, Q. Cai, T. Garrow, and C. Kapahi, QExPy: a python-3 module to support undergraduate physics laboratories, Soft-wareX 10, 100273 (2019).
[16] M. P. Silverman, Self-directed learning: A heretical experi-ment in teaching physics, American Journal of Physics 63, 495(1995).
[18] J. E. Kettler, The compact disk as a diffraction grat-ing, American Journal of Physics 59, 367 (1991),https://doi.org/10.1119/1.16553.
[19] See supplementary material for the CD diffraction lab manuals(2020).
[20] Transcribe: software used to transcribe audio recordings in thisstudy, https://transcribe.wreally.com/.
[21] J. Corbin and A. Strauss, Basics of Qualitative Research: Tech-niques and Procedures for Developing Grounded Theory, 4thed. (Newbury Park, CA: SAGE, 2015).
[22] D. R. Thomas, A general inductive approach for analyzingqualitative evaluation data, American Journal of Evaluation 27,237 (2006).
![Page 1: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/1.jpg)
![Page 2: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/2.jpg)
![Page 3: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/3.jpg)
![Page 4: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/4.jpg)
![Page 5: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/5.jpg)
![Page 6: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/6.jpg)
![Page 7: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/7.jpg)
![Page 8: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/8.jpg)
![Page 9: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/9.jpg)
![Page 10: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/10.jpg)
![Page 11: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/11.jpg)
![Page 12: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/12.jpg)
![Page 13: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/13.jpg)
![Page 14: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/14.jpg)
![Page 15: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/15.jpg)
![Page 16: arXiv:2006.06725v1 [physics.ed-ph] 11 Jun 2020 · port open inquiry. For example, in the CD diffraction exper-iment it is now possible for students to: change the distance between](https://reader036.documents.pub/reader036/viewer/2022071021/5fd5258f9c85f73bf52e3f29/html5/thumbnails/16.jpg)
![arXiv:2105.02897v1 [physics.ed-ph] 6 May 2021](https://static.documents.pub/doc/80x56/621b445d67369419b774c475/arxiv210502897v1-6-may-2021.jpg)
![arXiv:1907.08154v2 [physics.ed-ph] 26 Jul 2019](https://static.documents.pub/doc/80x56/61c446030921364fcd0d1e12/arxiv190708154v2-26-jul-2019.jpg)
![arXiv:1804.05748v2 [physics.ed-ph] 17 Oct 2018](https://static.documents.pub/doc/80x56/61cc2df435889f6b2405ce45/arxiv180405748v2-17-oct-2018.jpg)
![ELATIONS CANVAS TM ED OUTCOME HYPOTHESIS … · hypothesis iest assumption iment iment canvas] 2015 measure measure measure tm. ed outcome assumptions hypothesis iest assumption iment](https://static.documents.pub/doc/80x56/5f0d30cf7e708231d4391e1e/elations-canvas-tm-ed-outcome-hypothesis-hypothesis-iest-assumption-iment-iment.jpg)
![arXiv:2008.10931v2 [physics.ed-ph] 26 Aug 2020](https://static.documents.pub/doc/80x56/62588c33d31e6809834f2668/arxiv200810931v2-26-aug-2020.jpg)
![Abstract arXiv:1608.00718v1 [physics.ed-ph] 2 Aug 2016](https://static.documents.pub/doc/80x56/62079d644e01fe1fee3d12ed/abstract-arxiv160800718v1-2-aug-2016.jpg)
![arXiv:1510.07147v1 [physics.ed-ph] 24 Oct 2015](https://static.documents.pub/doc/80x56/6181f9d255ae5c357147e971/arxiv151007147v1-24-oct-2015.jpg)
![arXiv:2003.00896v1 [physics.ed-ph] 28 Feb 2020](https://static.documents.pub/doc/80x56/61ac85687d7bd35bbd323c24/arxiv200300896v1-28-feb-2020.jpg)
![1 arXiv:1712.06295v1 [physics.ed-ph] 18 Dec 2017](https://static.documents.pub/doc/80x56/61ef7bbd8dc6660ae83e4b62/1-arxiv171206295v1-18-dec-2017.jpg)
