Student Evaluation Differences between Different Physics by
Inquiry CoursesGordon J. Aubrecht, II,
Ohio State University PERGAbstract: The setting of Physics by Inquiry (PbI) classes is the laboratory. Students do experiments as suggested by the text as well as doing their own experiments to test predictions they have
made about nature’s behavior. This is guided inquiry, and students assessments of PbI classes reflect important aspects of inquiry, while not supporting others. We report here on results from different versions of Physics by Inquiry courses (properties of matter, electric circuits, and astronomy by sight and optics) using the Laboratory Program Variables Inventory (LPVI), a Q-sort
instrument.
The Q-sort mechanism, devised by William Stephenson in 1935, consists in organizing the statements or pictures into a ranking scheme.
Q-sort
Q-sort instruments can get first-hand data from large numbers of students in a short time. Our Q-sort instrument is called the Laboratory Program Variables Inventory (LPVI). The LPVI contains 25 statements characteristic of most hands-on lab environments; the statements describe common lab activities.It measures what individual students think are the most and least characteristic features of classroom activities, not what they most like or most dislike about the course.
Method
Students were instructed to sort the 25 statements of the modified LPVI into five groups. Group I is considered most descriptive of the course; Group V least descriptive. Students are forced to rank the statements into groups of size 2, 6, 9, 6, 2, forming a quasi-normal distribution.
Most
descriptive
to
the
Least
descriptive
1. Students follow the step-by-step instructions in the laboratory manual.
2. Questions in the laboratory manual require the interpretation
of data.
3. The instructor is concerned with the correctness of the data.
4. Students are allowed to go beyond regular laboratory exercises and do experiments on their own.
5. Laboratory activities are used to develop concepts.
6. The instructor lectures to the whole class.
7. Students are asked to design their own experiments.
8. During laboratory students record information requested by
the instructor or the laboratory manual.
9. Laboratory sessions raise new problems or result in data that cannot be immediately explained.
10. The instructor or laboratory manual identifies the problem to be investigated. 11. Laboratory activities require students to solve problems. 12. The laboratory manual requires that specific questions be answered. 13. The instructor or laboratory manual requires that students explain why certain things happen. 14. Laboratory is used to investigate a problem that comes up in class. 15. Laboratory experiments develop skill in the techniques or procedures of physics. 16. Questions in the laboratory manual require that students use evidence to back up their conclusions
17. Students discuss their data and conclusions with each other. 18. The instructor or laboratory manual asks students to state alternative explanations of observed phenomenon. 19. During laboratory students record the information they feel is important. 20. Students propose their own explanations for observed phenomenon. 21. Students identify problems to be investigated. 22. During laboratory students check the correctness of their work with the instructor. 23. In discussion with the instructor, assumptions are challenged and conclusions must be justified. 24. Students usually know the general outcome of an experiment before doing the experiment. 25. The instructor gives information to individuals in small groups.
Circuits PbI instructors N = 12
average score
matrix score
17 1.09 17 0.761 1.03 1 0.765 0.83 8 0.648 0.64 5 0.5623 0.62 16 0.54
21, 24 -0.32 21 -0.2514 -0.40 14 -0.297 -0.47 7 -0.374 -0.55 4 -0.396 -1.07 6 -0.65
The 2 analysis
We perform a 2 analysis on M’ that allows us to see which statements are significantly different from randomly chosen values. These are highlighted below. Items are considered significant if confidence level > 95%, or, in other words, if P(2) < 0.05 that the result could occur randomly.
N = 12average score P(2)
P(2)
matrix score
P(2)
17 1.09 0.0072 17 0.76 0.0072
1 1.03 0.039 1 0.76 0.039
5 0.83 8 0.64 0.0205
8 0.64 0.0205 5 0.56
23 0.62 16 0.54 0.039
16 0.504 0.039
18 -0.18 0.0283 18 -0.22 0.0283
21, 24 -0.32 21 -0.25
14 -0.40 14 -0.29
7 -0.47 7 -0.37
4 -0.55 4 -0.39
6 -1.07 <0.0001
6 -0.65 <0.0001
Ranking of Statements
The table shows how few of the Physics 107 instructors’ choices are significant. In addition, some non-outlier scores are significant: statements 16 and 18.
Distribution of matrix elements for the two greatest outliers and the two non-outlier statements for physics instructors.
Statement
I II III IV V
6 -0.10
-0.20
-0.36 0.12 0.56
17 0.28 0.03 -0.09 -0.24 0.01Statemen
tI II III IV V
16 -0.08
0.40 -0.18 -0.15 0.01
18 0.01 -0.24
-0.18 0.40 0.01
We have taken LPVI data for all three versions of our PbI class over several years. They are known as 106 (properties of matter), 107 (electric circuits), and 108 (optics and astronomy by sight).
Do all three classes’ students see the
courses (all taught the same way) as
acting identically?
Student significant LPVI statements appearing at least in two courses (red denotes among the top/bottom five)
Physics 106 Physics 107 Physics 1081 14 45 5 56 6 67 7 713 13 1314 14 14
16 1617 17 1718 18 1821 21 2122 22 2223 23 2324 24 24
Remember, only significant statements are shown. Statements 1, 4, 5, 6, 7, 13, 14, 16, 17, 18, 21, 22, 23, and 24 are commonly seen as very characteristic or very uncharacteristic.
5. Laboratory activities are used to develop concepts.
17. Students discuss their data and conclusions with each other.
22. During laboratory students check the correctness of their work with the instructor.
23. In discussion with the instructor, assumptions are challenged and conclusions must be justified.
6. The instructor lectures to the whole class.
7. Students are asked to design their own experiments.
18. The instructor or laboratory manual asks students to state alternative explanations of observed phenomenon.
21. Students identify problems to be investigated.
24. Students usually know the general outcome of an experiment before doing the experiment.
Statements 1, 4, and 16 are characteristic of just two classes each.
106, 107
1. Students follow the step-by-step instructions in the laboratory manual.
106, 108
4. Students are allowed to go beyond regular laboratory exercises and do experiments on their own.
107, 108
16. Questions in the laboratory manual require that students use evidence to back up their conclusions.
The LVPI is a valuable tool for use because it provides instructors information about how students perceive what actually happened in a course without the need for lengthy classroom observations.This work supports the usefulness of Q methodology for assessing laboratory courses and shows that it can be used to provide instructors with formative assessment of their classes.