EDCI858 Mathematics Research Synthesis
Michael Mazzarella
George Mason University
Paper (40/ 40)
Nice job pulling the synthesis together on the important variables you are examining through computer based algebra assessment. The focus on tem Format and Question Type will be of great interest to the mathematics education community as we continue to examine ways to assess students’ conceptual understanding and find the most valid assessment design. I look forward to following your work especially the dissertation that you gear up for next. I hope that the opportunity to spend some time on this research synthesis moved you closer to honing in on your proposal. Great to have you in our seminar!
Category Exceeds/Meets Expectations
Needs Revision/Unacceptable
Research Question (RQ) (5 points)
5/5 Question is clearly and coherently describedNo posting or posting is late
Analysis of Previous Studies(15 points)
15/15 Advanced organizer is given to connect major areas of investigation for previous studies and the RQ Analysis includes reference to studies that are clearly connected to the question and from multiple aspects of the question (e.g., students and teachers)Analysis synthesizes research as well as reporting or summarizing results of studies
No advanced organization is providedNo references are cited Incomplete explanation of previous research
Question is not situated within mathematics education research.Related studies are reported but not synthesized.
Theoretical Framework (TF)(15 points)
15/15 TF is clearly and coherently described including significant characteristics Analysis includes multiple references reflecting the chosen framework for the chosen question.
Analysis synthesizes research as well as reporting or summarizing results of studiesDescription includes implications for research methodology Appropriate, cohesive rationale is provided to connect TF with RQ
Characteristics of TF are not clearly described.Related studies are reported but not synthesized.
Very few studies are provided. Limited rationale is given to connect TF and RQ
References (5 points) 5/5 References are cited with APA 6
th formattingMinimal
grammar and stylistic errors
References are not cited, many references are missing Significant grammar and stylistic errors throughout
Algebra Achievement in a Computer-Based Environment: Item Format, Question Type, and Non-Cognitive Factors in High School Students
Within the last decade, many schools and school districts across the country have begun
administering end-of-the-year tests and other standardized assessments on the computer. Taking
these exams on the computer can impact how students perform on tests, even when the content is
the same as that of a paper-and-pencil test (Scherer & Siddiq, 2015). This change can drastically
impact student achievement, especially for high-stakes assessments. For example, in the state of
Virginia, the Standards of Learning (SOL) Assessment has been well-known by teachers,
students, and parents alike. Passing this assessment in almost all of the core high school subjects
is required for graduation (Virginia Department of Education, 2012). In the 2011-2012 school
year, the state of Virginia changed the format of all of the SOL exams in order to make them
more technology-based. All mathematics SOL assessments now include fill-in-the-blank, drag-
and-drop, and graph plotting questions, instead of only multiple choice questions. In the first
year, passing rates for all mathematics SOL exams significantly dropped. Many who are familiar
with the test, however, wonder whether this drop can be attributed to a change in item format, the
introduction to computer-based testing, an increase in difficulty of the exam, or a combination of
those factors.
The purpose of this study is to analyze the relationships among non-cognitive factors
(e.g. interest, task value, and perceived difficulty), computer-based algebra assessments, different
item formats and questions types on tests, and students’ perceptions of the assessments. The
complexity of the problem presented in this study is one that requires extensive review of the
literature. Literature on algebraic thinking encompasses students’ cognitive processes, but
achievement on algebra assessments is also highly influenced by social factors as well.
Expectancy-value literature attempts to define specific factors from the larger Social Cognitive
Theory, and offers measures to quantify these variables. The research on question type also
naturally relates to the algebraic thinking literature in that students often apply knowledge
transfer during tests. Finally, the specific idea of item format provides an added element of
thinking for students, and warrants research on the literature. Thus, it is important to conduct a
widespread review of the literature in order to understand and interpret the results of this study as
best as possible.
Algebra and Algebraic Thinking
From a mathematics perspective, algebraic thinking is a construct that is vital to student
success on mathematics assessments. Kaput (1998) categorized algebraic thinking into five
strands:
1. Algebra as generalizing and formalizing patterns and regularities, in particular,
algebra as generalized arithmetic
2. Algebra as syntactically guided manipulations of symbols
3. Algebra as the study of structure and systems abstracted from computations and
relations
4. Algebra as the study of functions, relations, and joint variations
5. Algebra as modelling
Kaput argued that, while all strands should be incorporated into learning to maximizing algebraic
thinking, much of the algebra curriculum in classrooms today focuses only on the second and
fourth strands. He further studied the extent to which “spontaneous algebraic reasoning” (SAR)
and “planned algebraic reasoning” (PAR) were used effectively in an activity-based urban
classroom. For instance, students took part in a lengthy discussion on the results of adding two
odd, two even, or an odd and even number. The authors found that when algebra was presented
as authentic problem solving, the number of SAR instances outweighed the number of PAR
instances. Furthermore, students were able to learn and understand concepts that were well
beyond their grade level, such as the representation of an odd number of 2n + 1 (Blanton &
Kaput, 2005). The authors theorized that algebraic thinking is enhanced when the content is
presented in realistic and relevant problems.
One important aspect of algebraic thinking comes from students making algebraic
meaning from the mathematics. Finding meaning within mathematics is vital in order for
students to understand abstract algebraic ideas and make connections among the numbers and
symbols (Sfard & Linchevski, 1994). Radford (2004) sorted meaning into three categories:
algebraic structure, problem context, and exterior of the problem context. Kieran (2007) split that
first category down further into two subsections: meaning from the letter-symbolic form of the
algebra, and meaning from multiple representations. For example, in terms of symbolic and
multiple representations, it is important that a traditional curriculum, such as literal expressions
and equations, be merged with authentic problems that show graphs, tables, and pictures (Kaput,
1989). This corresponds with Kaput’s argument that only one strand of algebra is being used in
today’s classroom.
Another viewpoint of algebraic thinking is that of seeing algebra as “activity.” Lee (1997)
offered several definitions of algebraic activity. One such definition is performing an action on
algebraic symbols, while another definition uses the notion of building algebraic objects.
Nevertheless, Lee emphasizes the idea of algebra as an active process. According to Kieran
(1996), algebra activity can be modeled using three types: generational, transformational, and
global/meta-level. On the generational level, algebra is the use of language to solve unknown
variables or express patterns and relationships among numbers and symbols. Transformational
activity refers to the manipulation of expressions, such as factorization, substitution, expansion,
or performing operations on polynomials. The global/meta-level of algebra refers to activities
that are not specific to algebra, but rather overall mathematical processes. These three levels of
activity are all interconnected according to Kieran’s “GTG” model of activity. The importance of
these activities, according to Kieran, as vital to developing student algebraic thinking,
particularly in young mathematics students.
Using technology in the algebra classroom has become a growing trend in the past
decade. On an upper algebra level, graphing calculators are commonplace for all students and
teachers. Doerr and Zangor (2000) claimed that students use graphing calculators in five
different ways: computation, transformation, visualization, verification, and data collection and
analysis. O’Callaghan (1998) studied the effects of graphing calculators on students’
understanding of algebra. The results showed that students who used graphing calculator during
the lessons showed a stronger conceptual understanding of higher level functions based on the
researcher’s test of functions. The same students scored higher on their end-of-the-year
assessment than students who did not used the calculators.
Social Cognitive Theory
According to Albert Bandura (1986), social cognitive theory states that students’
cognition is influenced by their behaviors, which are caused by personal and environmental
factors. Bandura, a pioneer researcher on the theory, students’ values impact the ways in which
they approach their academic careers. Furthermore, students frequently set academic and social
goals for themselves based on how important they perceive a task to be. These goals are often
based on the goals of their peers and the social environment around them (Bandura & Schunk,
1981). There is also a relationship between students’ values and their behavior (Wigfield, Tonks
& Eccles, 2004). In summary, Schunk (2012) stated that students learn a tremendous amount
from their social environment, and that social environment also influences their academic
success.
One of the most important pieces of conceptual frameworks of social cognitive theory is
triadic reciprocality. Triadic reciprocality describes the interaction among three influences:
person, behavior, and environment. The person in this model describes the students’ cognitive
skills and beliefs. Each factor is influenced by one another, both within and outside of the
classroom. For example, a student’s behavior can be influenced by both his personal beliefs and
the social environment around him, but conversely, his behaviors also affect those around him
and how he perceives himself (Bandura, 1977). This triadic influence can also happen at a group
level. For instance, a group of teachers may decide to adjust curricula and lessons if there is an
overall low achievement in a school (Schunk, 2012). While many constructs are at play within
this model, the relationships described by triadic reciprocality adequately summarize the main
tenants of social cognitive theory.
Schunk (2012) proposes that there are five central symbolic processes that occur within
social cognitive theory, particularly at the mathematics level. These processes, which help
describe student cognition during problem solving, include self-efficacy, goals, outcome
expectations, values, and social comparisons. Self-efficacy refers to a student’s beliefs in their
abilities to perform on a given task. This construct is a part of the “person” aspect of triadic
reciprocality in that a student’s self-efficacy influences his/her behavior and environment, and is
also influenced by these two as well (Bandura, 1997). Goals can refer to the progress and
outcomes that students achieve throughout their education. One important note is that there is a
difference between learning goals and performance goals. While learning goals are defined as
what knowledge students hope to achieve regardless of grades, performance goals refer to how
well students want to do on assessments and classroom grades (Anderman & Wolters, 2006).
Outcome expectations are often formed based on prior performance and are content specific; that
is, a student’s outcome expectation in one subject is not strongly influenced by their outcome
expectation in another class (Schunk & Zimmerman, 2006). Value is a broad term that can refer
to the importance of a task in different contexts. For instance, a student may view a particular
math task as valuable because doing well in school is valuable (i.e. attainment value) but does
not see the value in that task in terms of their career goals (i.e. utility value; Wigfield & Eccles,
2000). Social comparison is important not in the mere fact that it occurs, but in with whom
students compare themselves. It is most beneficial for students to compare themselves to those
who have similar abilities (Wheeler & Suls, 2005). In the present study, variations of values and
outcome expectations will be measured and analyzed.
Social cognitive characteristics of students have been found to impact mathematics
achievement. Lopez and colleagues (1997) measured math performance and social cognitive
constructs, such as interest, self-efficacy, and outcome expectations, in algebra students. The
path analysis models produced for the variables showed that social persuasion and perceived past
performance were both significant predictors of self-efficacy, which was a significant predictor
of students’ grades. Additionally, self-efficacy fully mediated the relationship between a
standardized mathematics test and classroom grades. In terms of interest, self-efficacy and
outcome expectations were both found to significantly predict interest (Lopez et al., 1997). In a
similar study, Pajares and Miller (1994) state that self-concept and self-efficacy both have a
direct effect on mathematics performance. This study also reported gender differences; despite
similar prior experience, men reported higher self-efficacy and had a higher math performance.
The results of these studies justify the need for further research on non-cognitive factors and
mathematics achievement.
Expectancy-Value Theory, Attainment Value, and Achievement
Social cognitive theory is a very broad theory that encompasses several other smaller
theories, each of which consist of several non-cognitive constructs. One smaller theory that lies
within social cognitive theory is expectancy-value theory. Schunk (2012) claims that
“expectancy-value theory bears much similarity to the social cognitive emphasis on goals, self-
evaluation of progress, self-efficacy, and outcome expectations” (p. 117). Atkinson (1957), one
of the founders of this theory, explained that expectations and values influence students’ beliefs
toward a task. He also popularized the term “task value,” which is a broader construct under
which attainment value exists, and argued that task value was intrinsic and could be measured in
terms of students’ pride in succeeding (Atkinson, 1957). While expectancy-value theory became
less popular after Atkinson’s publications on the topic, a “modern” version of the theory was
revived in the late 1900s. Some emerging questions in the field of educational psychology
regarding the “modern” expectancy-value theory included the relationships among expectations,
task value, and achievement-related behavior, as well as whether or not these beliefs change over
time (Graham & Weiner, 2012). Eccles and Wigfield (1995), some of the most important
researchers in the revival of the theory, claimed that task value consisted of four aspects:
attainment value, interest, utility value, and cost. Attainment value and interest, two constructs
that will be used in the present study, are both positively correlated with achievement in a given
task (Wigfield & Eccles, 2002).
Attainment value can be defined as how important a student feels it is for him/her to
succeed on an assessment, perform well in a subject, or apply a subject to their lives (Eccles &
Wigfield, 1992). Attainment value is a specific aspect of the expectancy-value theory. It is a
motivation construct that is content-specific. Bong (2002) studied the task value, self-efficacy,
performance goal, and mastery goal of several students across several subjects in middle school
and high school. The results showed that task value in high school was one of the most domain-
specific variables in the study. Specifically, in terms of mathematics, task value was correlated
with Korean, English, and science task value in high school at .06, .17, and .44, respectively, and
in middle school at .36, .37, and .45, respectively. However, as expected, task value was
significantly correlated with self-efficacy, performance goals, and mastery goals (Bong, 2002).
These findings suggest that task value for mathematics is a unique construct and, thus, it is
important to distinguish it from other similar constructs when measuring it.
Task value can also differ between males and females. Gaspard and her colleagues (2014)
measured several distinct constructs related to motivation in mathematics for ninth grade
students. After a factor analysis confirmed the distinction between interest, attainment, utility,
and cost constructs, which are the four aspects of task value according to Eccles and Wigfield
(1995), the results of this study showed that there are significant differences between males and
females in intrinsic value (i.e. interest), but not importance in achievement (i.e. attainment
value). Despite this difference, the components of the factor analysis of intrinsic and attainment
were significantly correlated (Gaspard, Dicke, Flunger, Schreier, Hafner, Trautwein, &
Nagengast, 2014). In addition to analyzing the relationship between the two motivational
constructs used in the present study, the aforementioned research also suggests that it is
necessary to further analyze the differences in gender for these constructs.
Personal Mathematics Interest and Achievement
In addition to attainment value, personal interest in a certain subject is another
motivational construct used in predicting achievement. Dewey (1913) defined interest as an
object, subject, or idea that becomes an accompanying part of one’s identity. In the context of
education, many researchers use “interest” and “intrinsic value” interchangeably. Intrinsic value
is another aspect that Eccles and Wigfield (1995) pinpointed in their four aspects of task value. It
is often found that intrinsic value and attainment value are strongly correlated, and when both
constructs are measured as high, achievement is also high (Durik, Vida, & Eccles, 2006).
Mitchell (1993) specified educational interest as interest directly tied to the content of
instruction. The present study will use the same definition as that of Mitchell (1993). Students’
interest may also differ based on the different types of problems. Renninger, Ewen, and Lasher
(2002) studied three cases of students with varying levels of mathematics interest and analyzed
how that interest related to mathematics word problems. The researchers found that even when
word problems were personalized to students’ mathematics abilities, high interest resulted in
higher achievement. Furthermore, higher interest can also result in students rereading problems
to understand their contexts or checking their answers to make sure that they makes sense in the
context of the problems (Renninger, Ewen, & Lasher, 2002).
Personal interest in mathematics varies throughout a student population. Trautwein,
Ludtke, Marsh, Koller, and Baumert (2006) measured interest in ninth grade students on
different mathematics tracks. The researchers found that interest in mathematics was
significantly higher in students in the upper track, but there was little difference in interest for
students in the middle or lower tracks. Interest was also found to be significantly correlated to
both individual achievement on standardized mathematics tests and the overall school
achievement on the same standardized tests (Trautwein, Ludtke, Marsh, Koller, & Baumert,
2006). The same researchers also found marginally significant results stating that there were
reciprocal effects on mathematics interest and self-concept; in other words, rather than one
construct impacting the other, both interest and self-concept impact each other. These results
were also found to be generalizable across gender (Marsh, Trautwein, Ludtke, Koller, and
Baumert, 2005). Other studies have also found personal interest to be related to other
psychological constructs. Ozyurek (2005) found statistically significant correlations between
interest in a mathematics class and self-efficacy, subject preference, previous mathematics
performance, and class expectations. These results were also consistent for undergraduate
students who were mathematics majors and not mathematics majors (Ozyurek, 2005). Therefore,
many factors contribute to one’s personal interest, including mathematics achievement, which is
a variable to be used in the current study.
Perceived Task Difficulty and Achievement
The way that a student personally views how complicated a certain subject or assignment
is can be referred to as perceived task difficulty (Midgley, Feldlaufer, & Eccles, 1989). Perceived
task difficulty is also categorized as a construct under the expectancy-value theory of Atkinson
(1957). Specifically, students’ expectations of a task impact their motivation and, thus, their
achievement. Another early definition of this construct explains it as the subject probability that
the task will result in success or failure (Karabenick & Youssef, 1968). Moreover, perceived task
difficulty can fall under the realm of social cognitive theory. Students engage in tasks in which
their expected outcome is high; however, when the task is expected to be difficult, it will impact
their performance in that specific task (Schunk, 1995). Midgley, Feldlaufer, and Eccles (1989)
found that perceived task difficulty can be affected by teacher efficacy, expectancies, previous
performance in a subject, or current classroom grades in a subject. Therefore, it is worthwhile to
study the relationships among perceived task difficulty, interest, attainment value, and algebra
achievement.
As mentioned previously, many motivational constructs are strongly correlated with one
another. Fulmer and Tulis (2013) analyzed the interest and perceived difficulty of middle school
students before, during, and after a difficult reading task. The results of the study showed that
interest was not significantly correlated with perceived difficulty until the final interest measure
after the reading, which, at the point, was negatively correlated. The researchers also measured
affect, which was defined in the study as a spectrum of positive and negative emotions (Fulmer
& Tulis, 2013). The study found that situational affect, which was defined as a student’s
emotional response to a particular task, was significantly correlated with perceived task difficulty
during and after the task, but not before it (Fulmer & Tulis, 2013). Not only do the researchers
introduce an interesting concept of measuring motivational constructs at different times during a
task, but they also supported prior research that these constructs were positively correlated.
Senko and Harackiewicz (2005) performed a similar study, which compared interest with not
only perceived task difficulty, but also the variable of introduced perceived goal difficulty, which
was defined as the extent to which students believe they can achieve goals that they set for
themselves. The researchers found that perceived goal difficulty was a mediating variable
between several other motivational constructs (Senko & Harackiewicz, 2005).
Similar to the studies done on attainment value, research suggests that there are gender
differences in perceived task difficulty. Parsons, Adler, and Meece (1984) conducted a
groundbreaking study that compared differences in motivational constructs between gender and
subject (English and mathematics). Results showed that all students, regardless of gender,
perceived mathematics as more difficult than English. However, despite this significant
difference between subjects, there was very little difference in perceived task difficulty between
males and females (Parsons, Adler, & Meece, 1984). Although this study was designed well and
had measures relevant to the present study, it was conducted over thirty years ago, so it should be
replicated to analyze present-day implications.
Research shows that the three motivational constructs outlined above (personal interest,
attainment value, and perceived task difficulty) all individually impact mathematics
achievement. However, there is less research regarding the relationship among the three as they
relate to computer-based mathematics assessments, particularly with different item formats and
question types.
Assessments: Item Format
In the present study, item format refers to the ways in which students can respond to the
prompts. Some item formats include multiple choice, true or false, selecting multiple correct
answers, and fill-in-the-blank. Many research studies have been conducted regarding the
relationship between item format and achievement. Ozuru, Best, Bell, Witherspoon, and
McNamara (2010) measured achievement on a reading assessment. Items were separated into
multiple choice and open ended questions. Overall, students scored the multiple choice questions
correctly more often than the open ended questions. The average score for the multiple choice
section without and with available text was 68% and 79%, respectively, while the average score
for the open ended section without and with available text was 48% and 60%, respectively
(Ozuru, Best, Bell, Witherspoon, & McNamara, 2010). The large gap between the scores of the
multiple choice and open ended questions suggests that students think about each type of
question differently, which affects achievement on assessments.
Item format may be particularly important when the exams are computer-based. For
instance, Jodoin (2003) studied responses on an engineering exam that used multiple choice and
“innovative” items, such as drag-and-drop and selecting multiple answers. Results showed that
examinees answered multiple choice items with more accuracy, even when items of different
formats were asking about the same subject (Jodoin, 2003).
Not only can different item formats produce different achievement scores, but they may
result in different strategies used. For example, Katz, Bennett, and Berger (2000) had students
record the strategies used for each item on an assessment with multiple item formats. Traditional
strategies included using formulas and solving algebraic equations, while nontraditional
strategies included estimation and guess-and-check. Results showed that while item format for
some questions changed their levels of difficulty, the strategies used for different questions did
not affect the achievement (Katz, Bennett, & Berger, 2000).
It is important to note that measures with different item formats can threaten validity in
several ways. For example, in one of the first articles looking at computer-based assessments
with different formats, Martinez and Bennett (1992) analyzed several types of mathematics skills
(e.g. algebraic reasoning, computer science) that were tested using different item formats.
Psychometric analysis determined that there was little discrepancy between computerized raters
and human raters, but in some cases, scores differed by up to 1.2 points on a 16-point scale
(Martinez & Bennett, 1992). Because of this small difference, computerized grading gained more
trust; nevertheless, even a small difference such as this highlights the importance of calculating
validity and reliability of computer-based assessments.
In a similar article, Pomplun and Omar (1997) outlined four different threats to the
validity of an assessment: lack of familiarity of the item format, omitting alternatives,
dependency among alternatives, and guessing. In their study, two specific types of item formats
used were “multiple mark” (a multiple choice question with more than one correct answer) and a
“multiple true-false question” (similar to multiple mark, but only with T/F as possible choices).
The results of this article stated that omitting answers or not following directions did not
seriously threaten validity, but students tended to leave choices blank instead of guessing
(Pomplun & Omar, 1997). Further threats to validity will be discussed in the “Limitations”
section of this study.
Due to the recent transition from paper-and-pencil to computer-based assessments, the
research on item format is minimal, especially in the mathematics field. Further research is
needed to determine how item format affects different content areas. Additionally, non-multiple
choice questions are often grouped into one “constructed response” category, but their
differences on a computer-based test are not distinguished. There is also little research on how
item format impacts different types of mathematical questions.
Assessments: Question Type
Assessing mathematics knowledge is a broad idea that can be categorized into multiple
aspects. Thus, it is important to study differences in the ways that questions are asked on
mathematics assessments. In particular, two such types of mathematics problems are
computational (i.e. straightforward) problems and word problems. Fuchs et al. (2008) studied
whether or not different aspects of cognition (e.g. language, concept formation, and working
memory) were used in different types of problems. Results showed that correlations between
computational and problem-solving skill was only moderate. Cognitively, processing speed was
highly correlated with computational skill, but not with problem-solving skill. Working memory,
on the other hand, was more highly correlated with problem-solving skill than with
computational skill (Fuchs et al., 2008). The same study also compared demographics to
achievement on different question types. For example, poverty and race had little effect on
difficulty with computational problems, but those who had difficulty with solving word problems
were poorer and more likely to be African-American (Fuchs et al., 2008). Therefore, it is
certainly worthwhile to study question type as it relates to mathematics achievement.
When studying question type, one must take into consideration the construct of
knowledge transfer. Knowledge transfer can be defined as the ability to relay knowledge that
students learned one way to a task presented in a different way (Belenky & Nokes-Malach,
2013). In the present study, knowledge transfer is relevant regarding question type because
students may have learned a mathematics procedure in a straightforward manner, but may not be
able to apply that knowledge to a word problem, for example. Knowledge transfer also relates to
assessments from an information processing viewpoint. Working memory is necessary for
students to be able to transfer, organize, and apply their knowledge to a given task (Belenky &
Nokes-Malach, 2013). Day and Goldstone (2012) stated that despite cognitive load or item
difficulty, student transfer was high when students were shown several examples of the problem.
They went on to claim the following: “Contextual similarity between the situations themselves
seems to play a much larger role in determining whether transfer will actually occur” (Day &
Goldstone, 2012, p. 155).
Self-regulation can also play a role in students’ transfer abilities. According to
Zimmerman (1986), an early pioneer of the construct, self-regulation is defined as actively
participating in one’s own learning. This can take place cognitively, metacognitively,
motivationally, or behaviorally (Zimmerman, 1986). Kramarski, Weiss, and Sharon (2013)
experimented with an intervention that attempted to increase students’ transfer abilities among
mathematics tasks by increasing self-regulation. The researchers gave students a survey
measuring three aspects of self-regulation (planning, monitoring, and evaluation), procedural
knowledge algebra tasks, and verbal algebra problem-solving tasks, which they classified as
“long-term transfer to novel tasks.” The results showed that while there was no significant
difference between the two learning approaches for the procedural knowledge tasks, there were
significant differences between the two learning approaches for the problem-solving tasks,
including all three specific types of mathematical categories (algebraic, number sense, and
visualization; Kramarski, Weiss, & Sharon, 2013). Those who were part of the intervention
group tended to be higher in self-regulation, which led to high scores on mathematical tasks that
were classified as “far transfer” tasks (number sense and visualization; Kramarski et al., 2013).
Overall, knowledge transfer is another construct to consider in all aspects of assessments,
especially question type and item format.
Gaps in the Literature
Despite previous research on these topics, there are several gaps in the literature. First,
although Renninger, Ewen, and Lasher (2002) claim that mathematics interest can impact
achievement on different question types, their study was done using a paper-and-pencil
assessment. For that reason, it is worthwhile to study whether similar results will occur with a
computer-based assessment. Second, while there is much research on item format and question
type individually as they relate to achievement on a computer-based test, there is little research
comparing the effects of the interactions between item format and question type on achievement
on a computer-based test. Third, more research is needed on the relationships among
motivational constructs and items formats. While many research studies have been done
comparing interest, task value, and perceived difficulty with mathematics achievement, little is
known about their impact on achievement on a computer-based assessment with different item
formats. Finally, while there has been much research on the results of computer-based
mathematics assessments, there is little research how students view and perceive these types of
exams, especially at the high-stakes level.
Research Questions
The following research questions hope to address the gaps in the literature regarding
computer-based mathematics assessments, mathematics interest, attainment value, and perceived
task value:
1. To what extent do personal interest, attainment value, and perceived task difficulty
predict algebra achievement on a computer-based assessment with different item formats
and question types?
2. What are students’ perceptions on a computer-based standardized math test, in terms of
relevance, value, and importance?
Researcher Identity
When conducting qualitative or mixed-methods research, it is important to consider the
researcher’s experiences, beliefs, and identity. Knowing this information can help the reader
understand the knowledge that the researcher comes into the study with, as well as their purpose
in conducting the study. As a math teacher myself, I have tremendous experience with teacher
identity and standardized testing. For four years, I have had to prepare students for the end-of-
the-year assessment in my courses. This assessment was not only important for the students, but
it also reflected on my “success” as their teacher; over half of my professional evaluation was
based on these exams. This shaped my identity as a teacher in several ways. The most prevalent
way is that I found myself teaching to the test, especially in my first year on the job, rather than
focusing on students conceptually understanding the material. This is partially due to my belief
that the new test is, in fact, difficult for students. I was so nervous about how I would get graded,
that I focused less on what was truly important: student understanding. I continue to find a
balance between preparation for a test and teaching students critical thinking skills, but it
certainly has influenced my role as a teacher. Nevertheless, I have often wondered if this is how
the teaching profession has always been. I was aware of the changes that took place my first
year, but I wondered how different it actually was. Within the school, I see myself as a team
player who shares materials and strategies with other teachers. I also see myself as somewhat of
a leader; other teachers come to me for help or advice regarding students or lessons. My identity
as a teacher shapes this research because I have a better understanding of the school environment
in which the participants work.
In addition to being a high school mathematics teacher, I am also currently pursuing a
Ph.D. in Education degree at George Mason University. My primary concentration is educational
psychology, and my secondary concentration is mathematics leadership. At the time of this
study, I am beginning my third year in the program. My research interests include secondary
mathematics, computer-based standardized tests with different item formats, and motivation
constructs based on the expectancy-value theory (e.g. personal interest, task value). Before the
time of this study, the only analysis that I had conducted was quantitative. I had taken several
upper-level statistics classes and felt very comfortable conducting quantitative analysis.
Furthermore, I had even created and conducted a quantitative pilot study. As a result, I came into
this study feeling very uncomfortable with qualitative analysis. Therefore, I am still not fully
comfortable with qualitative or mixed-methods studies, their procedures, and the analysis that is
conducted from qualitative data.
Method
Participants
The participants of this study will be 225 high school students (n = 225) currently
enrolled in an on-level Algebra II course in a large suburban school district during the 2014-2015
school year. These students will be selected based on a sample of convenience. The 225 students
will come from a total of nine on-level high school Algebra II classes, four of which are taught
by the researcher and five of which are taught by three other teachers in the same school. The
three teachers assisting the researcher with this study are on-level Algebra II teachers at the same
school. These teachers have two, seven, and twenty years of experience teaching within the same
school, while the researcher has four years of teaching experience within the same school. The
present study has been fully explained to the other three teachers, and they have agreed to help in
the data collection process and allow their students to participate in the study.
The sample will consist of 113 females (n = 113) and 112 males (n = 112). The ages of
the students at the time of the study will range from fifteen years to nineteen years. The ethnicity
of the students is expected to be similar to that of the school demographic: About 33% of
students will be Hispanic, 25% of students will be Asian, 24% of students will be white (not of
Hispanic origin), 18% of students will be African-American (not of Hispanic origin), and 2% of
students will be listed as “other.” Approximately twenty students will be categorized as limited
English proficiency. About 55% of students will receive free or reduced lunch on a daily basis.
Approximately 5% of students will be categorized as special education. The school at which the
data is being collected is a public school within a large, diverse, suburban community. The
school has approximately 2,300 students, and the ethnicity of those students will be
approximately the same as that of the sample (source not cited to ensure confidentiality).
Measures
Personal mathematics interest (Mitchell, 1993). The self-report measure will consist of
items measuring interest, attainment value, and perceived task difficulty. On this survey, there
will be four items measuring personal mathematics interest. The survey consists of four Likert-
scale items measuring students’ personal interest in mathematics. One example of an item asks
students to evaluate this statement: “Compared to other subjects, mathematics is exciting to me.”
Students will respond to each item by circling “strongly agree,” “agree,” “slightly agree,”
“slightly disagree,” “disagree,” or “strongly disagree.” The internal consistency coefficient of
this measure was found to be .92, which suggests that the measure is very reliable (Mitchell,
1993).
Attainment value (Eccles & Wigfield, 1995). On the same survey containing the four
interest items, there will be three items measuring attainment value. One example of an item
measuring attainment value is the following: “It is important for me to get good grades in math.”
These items will be answered on the same Likert-scale as the interest items. The internal
consistency of these items is .70 (Eccles & Wigfield, 1995).
Perceived task difficulty (Eccles & Wigfield, 1995). On the same survey containing the
interest and attainment value items, there will be three items measuring perceived task difficulty.
One example of an item measuring perceived task difficulty is the following: “In general,
mathematics is hard for me.” These items will be answered on the same Likert-scale as the
interest and attainment value items. The internal consistency of these items is .80 (Eccles &
Wigfield, 1995).
Demographics. In addition to the ten Likert-scale items, students will also record their
gender, grade level, and age on the survey. The items of this survey can be found at the end of
this document in Appendix B.
Mathematics achievement (Mazzarella, 2015). Students will be given a set of thirty
mathematics items on the computer through the program Horizon, which is a commonly used
computer-based assessment program in the county in which the present study will be being
conducted. This program allows teachers to create items of different formats, and teachers will
receive the students’ results for each question when the assessment is complete. The items for
this measure can be found in Appendix C.
The curriculum used to create these mathematics items aligns with the standards set forth
by the county in which this study will be conducted. These standards also match the standards
used to create the end-of-year state assessment that students in Algebra II are required to take.
Some examples of standards included in the mathematics measure are solving radical, absolute
value, and rational equations, finding the domain and range of various functions, simplifying
rational and radical expressions, identifying properties of a normal distribution, and recognizing
and solving permutations and combinations. According to the state in which the study will be
taking place, there are four strands (i.e. standards) that categorize Algebra II test questions:
Expressions and Operations, Equations and Inequalities, Functions, and Statistics (Virginia
Department of Education, 2012). It is also worthwhile to note that students in the county are
required to complete and pass Algebra II in order to graduate. This is important to keep in mind
because it implies that students with a variety of skill levels will be measured in the present
study, rather than only students who choose to enroll in an Algebra II course without it being
required.
Procedures
In order to obtain assent from students and consent from the students’ parents or
guardians, the researcher will speak with all classes about the purpose of this study, the measures
being used, the potential of being chosen randomly to be interviewed, the types of questions that
would be asked on interviews, the students’ optional participation in this study, and the
confidential data being collected. Students will then be given the consent and assent forms, and
they will be instructed to read and sign the assent forms and have their parents read and sign the
consent forms. Once the forms have been read and signed, the students will be instructed to
return both forms to their teacher. The signed forms will be kept in a secure file cabinet in the
researcher’s room until the study has been completed.
Participants will first take a survey in which they self-report their levels of interest,
attainment value, and perceived task difficulty in mathematics. The survey consists of ten Likert-
scale items and three questions pertaining to demographics, all of which will take approximately
two minutes to complete. Students will be asked to put their student ID number, but not their
name, on the survey. A teacher other than the students’ Algebra II teacher will be in the room
while the survey is administered to ensure confidentiality.
One class period after taking the self-report survey, students will take a measure of
mathematics achievement. Students will have one full class period (90 minutes) to complete this
test. Once students complete the test, the responses and scores will be linked to a student ID
number that matches that written on the survey. A teacher other than the researcher will match
the responses of the survey and the math measure, enter the data into a Microsoft Excel file,
replace the student ID numbers with a different unidentifiable ID number, and give the data to
the researcher.
Finally, after all measures have been administered, five students will be randomly
selected to participate in a one-hour long interview with the researcher. From the beginning of
this process, participants were informed that the interview would be recorded, but the recording
would only be used for transcription purposes and their voices would not be used. Additionally, I
would use a pseudonym for their name, school, and any other identifying information. The
questions used in the interview can be found in Appendix C. These questions will act more as
guidelines rather than strict required questions. As an incentive, all students participating in the
study (whether or not they were interviewed) will be entered into a raffle to win one of four $25
Target gift cards. Each of the three teachers assisting with the study will also receive a $25
Target gift card.
Research Design
All students will take the same self-report survey. The items in the survey are staggered
based on construct. For example, the first item measures interest, the second item measures
attainment value, the third item measures perceived task difficulty, and the remaining items
follow that same pattern. Items are staggered in this way in an effort to separate similar
questions.
Classes will be randomly assigned to take one of four versions of the math measure: Test
A, Test B, Test C, or Test D. Test A and Test B have the same prompts in the same order. On
Test A, the odd-numbered questions will be multiple choice questions, and the even-numbered
questions will be technology-enhanced questions (either fill-in-the-blank or selecting multiple
correct answers). On Test B, the even numbered questions will be multiple choice questions, and
the odd numbered questions will be technology-enhanced questions. For example, on both tests,
Question #1 asks students to solve a radical equation. On Test A, students will answer that
question in multiple-choice format, while on Test B, students will answer that same question in
fill-in-the-blank format. Test questions are staggered this way so that data will be collected for
every prompt in both a multiple choice and technology-enhanced format. On both Test A and
Test B, there will be twenty straightforward mathematics problems (ten multiple choice and ten
technology-enhanced) and ten real-world application problems (five multiple choice and five
technology-enhanced). Creating the assessments in this way yields four categories:
straightforward multiple choice, straightforward technology-enhanced, word problem multiple
choice, and word problem technology-enhanced. Both tests have the same number of questions
in each category. These tests were created in this way because the data collected will provide a
comparison of the same prompts with different item formats, as well as a comparison between
straightforward questions and word problems. The items in Test A and Test B can be found at
the end of this document in Appendix B.
Test C will have the same exact questions and formats as Test A, but the questions will
be in reverse order. Test D will have the same exact questions and formats as Test B, but the
questions will be in reverse order. Test C and Test D were created to account for test fatigue.
Tyrrell and colleagues (1995) support that visual and mental fatigue can occur as students take
assessments, especially when the assessments are taken on the computer. As students work
through the thirty-question math measure, some may become less motivated or energized toward
the end of the test. Thus, it is important to test whether or not this occurs in the measure before
drawing conclusions about specific test questions or overall achievement.
As mentioned previously, the results of the mathematics measure will be available online
to the student’s teacher once the test is complete. Results will include the number of correct
answers and the responses to each multiple choice and technology-enhanced question. The
results will also be separated based on question type (i.e. whether the question was
straightforward or a word problem). All students will have access to a graphing calculator during
the mathematics measure.
After the students have completed the measures, the five students that were chosen
randomly from the sample to be interviewed will be contacted. The interviews will be scheduled
at a time and place of convenience for the participant. Communication will primarily be
conducted via e-mail to schedule the interview, but the interviews themselves will actually be
conducted in person. The interviews will be recorded using an iPhone 6.
Data Analysis
All data analysis will be conducted using the computer programs Microsoft Excel, SPSS,
jMetrik, Mplus or NVivo. A data analysis matrix organizer can be found at the end of this
document in Appendix D. The first research question asks the following: To what extent do
personal interest, attainment value, and perceived task difficulty predict algebra achievement on
a computer-based assessment with different item formats and question types? Several types of
analysis will be used to address this question. First, descriptive statistics will be calculated for
each of the four versions of the mathematics measure. These descriptive statistics, which include
mean and standard deviation, will compare students’ overall achievement on each test. Second,
correlation coefficients will first be calculated to look at the relationships among personal
interest, attainment value, and perceived task difficulty. Ideally, the correlation coefficients will
be less than .80 to avoid multicollinearity (Shieh & Fouladi, 2003). Next, a confirmatory factor
analysis will be run using the survey items in order to confirm that three constructs are being
measured and that the model is an acceptable fit. If the model is not an acceptable fit, certain
items may be removed, and further analysis will be run with the adjusted number of items.
Finally, two separate multiple regression analyses will be run. The first regression will test how
well the three motivational constructs and question types predict achievement on multiple choice
questions. The second regression will test how well the three motivational constructs and
question types predict achievement on technology-enhanced items. In both regression models,
question type will be coded as 0 for straightforward problems and 1 for real-world problems.
The second research question asks: What are students’ perceptions on a computer-based
standardized math test, in terms of relevance, value, and importance? In order to answer this
question, student interviews will be conducted, transcribed, and coded. During the interview, the
researcher will take notes based on the interviewee’s responses. The purpose of these notes will
be to record initial reactions, mark unanticipated themes, and note when certain responses
correspond to those of other participants. The coding of the transcripts, which will be done by
hand using Microsoft Word, will be approached with some initial codes that the researcher will
look for (e.g. item format, pressure, value, importance, relevance), but will mostly be organized
accordingly once the researcher looks for patterns among the responses. The NVivo program will
allow the researcher to create “nodes,” which act as codes, as well as “sub-nodes,” which are
used if there are several common responses regarding a larger theme. This will allow the coding
to be organized in an outline format. Once the coding is complete, the researcher will consolidate
and combine nodes on NVivo in order to create a more organized system of codes. From this
method of organization, the researcher will then look for themes, connections, and conclusions
that can be found in the data.
Limitations, Further Research, and Educational Implications
There are several limitations in regard to this study. One such limitation is the validity of
a self-report survey. It is important to take into account the validity of each measure when
collecting data. In particular, the measure of interest should be carefully examined. Tracey
(2012) claims that single interest scales contain two types of error: systematic error and general
factor variance. Furthermore, these scales do not take into account bias that can influence
students’ interest. For example, interest is often correlated with students’ mathematics scores,
which can cause a problem with validity (Tracey, 2012). For that reason, it is important to
analyze the results of the interest survey to determine whether any of these problems can
compromise the research questions.
Another limitation of this study is the fact that students are taught by several different
teachers. Although the teachers whose students are participating in the study collaborate
throughout the year and use common lessons and assessments, teaching styles and effectiveness
may differ, which may result in a significant difference in achievement, motivation, or both.
Future research should control for this difference and use teachers who may have similar
teaching styles.
Third, the present study is specific in its content and demographics and, therefore, cannot
be generalized across all subjects, grade levels, or skill levels. In addition to a lack of
generalizability to other content areas, this study may only be applicable to only on-level
Algebra II students. As mentioned, Algebra II is more complex than Algebra I and, thus, requires
more skills to achieve mastery. Similarly, achievement in Geometry requires a different
mathematical skill set. Furthermore, this study was conducted with participants in only high
school, and the results cannot be generalized to elementary or middle school students. Further
research is needed to determine whether the results of this study will be similar across these
different demographics and subjects.
Next, the participant selection of this study is that of a convenient sample. The students
of this study attend the school at which the researcher teaches. Some of the participants are even
the researcher’s students. This can create bias and a potential of interest in the research because
of a personal connection to the researcher. Additionally, some of the responses to the self-report
survey may be a result of socially desirable responding; that is, students’ answers may be what
they think the teacher and researcher want to hear rather than an accurate representation of their
motivational beliefs. To prevent this, the study should be replicated using students outside of the
researcher’s school.
Finally, another limitation of this study is the researcher bias. As discussed previously,
my experience as a math teacher provided a foundation for this study, but also may have
impacted the interview and interpretation of responses. Because I have beliefs about the test
myself, it is possible that some of my personal thoughts influenced the results. Of course, I have
been making an effort to minimize this bias, but it should be noted that it cannot be fully
eliminated from the study.
Further research could add to the findings of this study in several ways. First, including
additional motivational constructs would give an added element to the study. For example,
measuring students’ task value, self-efficacy, and outcome expectations could provide a wealth
of additional information about students’ motivation as it relates to achievement on computer-
based assessments. Second, future research can also replicate this study in other subjects or
demographics. For instance, it would be worthwhile to determine whether the same results would
occur with geometry students or in a different school setting (e.g. rural or urban). Third, while
interviewing students provides valuable information regarding the assessments, interviewing the
teachers as well would allow the researcher to triangulate data and find common responses
among all those involved with the tests. Interviewing teachers would provide a different set of
factors regarding the exams, such as pressures put on by administration and autonomy within the
classroom.
The present study contains many practical implications for educational psychology and
secondary mathematics education. Overall, this study may bring to light certain aspects of
computer-based testing that have not previously been analyzed. With more counties and states
changing their standardized testing systems from paper-and-pencil to technology-enhanced, it is
important to investigate whether or not these assessments are testing students accurately and
fairly. The present study will not only determine the differences between item format and
question type, but it will also determine how personal interest, task value, and perceived
difficulty will impact achievement. Furthermore, the data of this study may also provide valuable
information to teachers on how to best prepare their students for these tests. It is the hope of the
researcher that testing companies will take into consideration the findings of this study and make
the necessary adjustments to the tests so that all students have a fair opportunity to succeed.
References
Anderman, E.M. & Wolters, C.A. (2006). Goals, values, and affects: Influences on student
motivation. In P.A. Alexander & P.H. Winne (Eds.), Handbook of educational
psychology (2nd ed., pp. 369-389). Mahwah, NJ: Erlbaum.
Atkinson, J. W. (1957). Motivational determinants of risk-taking behaviors. Psychological
Review, 64, 359-372.
Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory.
Englewood Cliffs, NJ: Prentice Hall.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman.
Bandura, A. & Schunk, D. H. (1981). Cultivating competence, self-efficacy, and intrinsic interest
through proximal self-motivation. Journal of Personality and Social Psychology, 41,
586-598. doi: 10.1037/0022-3514.41.3.586
Belenky, D. M., & Nokes-Malach, T. J. (2013). Mastery-approach goals and knowledge transfer:
An investigation into the effects of task structure and framing instruction. Learning and
Individual Difference, 25, 21-34. doi: 10.1016/j.lindif.2013.02.004
Blanton, M.L. & Kaput, J.J. (2005). Characterizing a classroom practice that promotes algebraic
reasoning. Journal for Research in Mathematics Education, 36(5), 412-446. Retrieved
from http://www.jstor.org/stable/30034944
Bong, M. (2004). Between- and within-domain relations of academic motivation among middle
and high school students: Self-efficacy, task-value, and achievement goals. Journal of
Educational Psychology, 93(1), 23-34. doi: 10.I037//0022-0663.93.1.23
Day, S. B. & Goldstone, R. L. (2012). The import of knowledge export: Connecting findings and
theories of transfer of learning. Educational Psychologist, 47(3), 153-176. doi:
10.1080/00461520.2012.696438
Dewey, J. (1913). Interest and effort in education. Cambridge, MA: Riverside Press.
Doerr, H.M. & Zangor, R. (2000). Creating meaning for and with the graphing calculator.
Educational Studies in mathematics, 41, 143-163.
Durik, A. M., Vida, M., & Eccles, J. S. (2006). Task values and ability beliefs as predictors of
high school literacy choices: A developmental analysis. Journal of Educational
Psychology, 98, 382-393. doi: 10.1037/0022-0663.98.2.382
Eccles, J. S. & Wigfield, A. (1992). The development of achievement task values: A theoretical
analysis. Developmental Review, 12, 265-310. doi: 10.1016/0273-2297(92)90011-P
Eccles, J. S. & Wigfield, A. (1995). In the mind of the actor: The structure of adolescents'
achievement task values and expectancy-related beliefs. Personality and Social
Psychology Bulletin, 21(3), 215-225. doi: 10.1177/0146167295213003
Fuchs, L. S., Fuchs, D., Stuebing, K., Fletcher, J. M., Hamlett, C. L., & Lambert, W. (2008).
Problem solving and computational skill: Are they shared or distinct aspects of
mathematical cognition? Journal of Educational Psychology, 100(1), 30-47. doi:
10.1037/0022-0663.100.1.30
Fulmer, S. M. & Tulis, M. (2013). Changes in interest and affect during a difficult reading task:
Relationships with perceived difficulty and reading fluency. Learning and Instruction,
27, 11-20. doi: 10.1016/j.learninstruc.2013.02.001
Gaspard, H., Dicke, A.-L., Flunger, B., Schreier, B., Häfner, I., Trautwein, U., & Nagengast, B.
(2014, October 27). More value through greater differentiation: Gender differences in
value beliefs about math. Journal of Educational Psychology. Advance online
publication. http://dx.doi.org/10.1037/edu0000003
Graham, S. & Weiner, B. (2012). Motivation: Past, present, and future. In Harris, K. R., Graham,
S., Urdan, T., McCormick, C. B., Sinatra, G. M., & Sweller, J. (Eds.), APA educational
psychology handbook, Vol 1: Theories, constructs, and critical issues. Washington, DC:
American Psychological Association.
Jodoin, M. G. (2003). Measurement efficiency of innovative item formats in computer-based
testing. Journal of Educational Measurement, 40(1), 1-15. doi: 10.1111/j.1745-
3984.2003.tb01093.
Kaput, J.J. (1989). Linking representations in the symbol systems of algebra. In S. Wagner & C.
Kieran (Eds.), Research issues in the learning and teaching of algebra (Vol. 4 of
Research agenda for mathematics education, pp. 167-194). Reston, VA: National
Council of Teachers of Mathematics.
Kaput, J. J. (1998). Transforming algebra from an engine of inequity to an engine of
mathematical power by “algebrafying” the K-12 curriculum. In the National Council of
Teachers of Mathematics & the Mathematical Sciences Education Board (Eds.), The
nature and role of algebra in the K-14 curriculum: Proceedings of a national symposium
(pp. 25-26). Washington, DC: National Research Council, National Academy Press.
Karabenick, S. A. & Youssef, Z. I. (1968). Performance as a function of achievement motive
level and perceived difficulty. Journal of Personality and Social Psychology, 10(4), 414-
419. doi: 10.1037/h0026735
Katz, I. R., Bennett, R. E., & Berger, A. E. (2000). Effects of response format on difficulty of
SAT-mathematics items: It’s not the strategy. Journal of Educational Measurement,
37(1), 39-57. doi: 10.1111/j.1745-3984.2000.tb01075.x
Kieran, C. (1996). The changing face of school algebra. In C. Alsina, J. Alvared, B. Hodgson, C.
Laborde, & A. Perez (Eds.), Eighth International Congress on Mathematical Education:
Selected lectures (pp. 271-290). Seville, Spain: S.A.E.M. Thales.
Kieran, C. (2007). Learning and teaching algebra at the middle school through college levels:
Building meaning for symbols and their manipulation. In Lester, F.K. (Ed.), Second
Handbook of Research on Mathematics Teaching and Learning (pp. 707-762). Charlotte,
NC: Information Age Publishing.
Kramarski, B., Weiss, I., & Sharon, S. (2013). Generic versus context-specific prompts for
supporting self-regulation in mathematical problem solving among students with low or
high prior knowledge. Journal of Cognitive Education and Psychology, 12(2), 197-214.
doi: 10.1891/1945-8959.12.2.197
Lee, L. (1997). Algebraic understanding: The search for a model in the mathematics education
community. Unpublished doctoral dissertation, Universite du Quebec a Montreal.
Lent, R.W., Brown, S.D., & Hackett, G. (1994). Towards a unifying social cognitive theory of
career and academic interest, choice, and performance. Journal of Vocational Behavior,
45, 79-122. doi: 10.1006/jvbe.1994.1027
Lopez, F.G., Lent, R.W., Brown, S.D., & Gore, P.A. (1997). Role of social-cognitive
expectations in high school students’ mathematical-related interest and performance.
Journal of Counseling Psychology, 44(1), 44-52. doi: 10.1037/0022-0167.44.1.44
Marsh, H. W., Trautwein, U., Ludtke, O., Koller, O., & Baumert, J. (2005). Academic self-
concept, interest, grades, and standardized test scores: Reciprocal effects models causal
ordering. Child Development, 76(2), 397-416. doi: 10.1111/j.1467-
8624.2005.00853.x
Martinez, M. E. & Bennett, R. E. (1992). A review of automatically scorable constructed-
response item types for large-scale assessment. Applied Measure in Education, 5(2), 151-
169. doi: 10.1207/s15324818ame0502_5
Mazzarella, M. (2015). Algebra II Review. Unpublished mathematics assessment, George Mason
University, Fairfax, VA.
Midgley, C., Feldlaufer, H., & Eccles, J. S. (1989). Change in teacher efficacy and student self-
and task-related beliefs in mathematics during the transition to junior high school.
Journal of Educational Psychology, 81(2), 247-258. doi: 10.1037/0022-0663.81.2.247
Mitchell, M. (1993). Situational interest: Its multifaceted structure in the secondary school
mathematics classroom. Journal of Educational Psychology, 85(3), 424-436. doi:
10.1037/0022-0663.85.3.424
O’Callaghan, B.R. (1998). Computer-intensive algebra and students’ conceptual knowledge of
function. Journal of Research in Mathematics Education, 29, 21-40.
Ozuru, Y., Best, R., Bell, C., Witherspoon, A. & McNamara, D. S. (2010). Influence of question
format and text availability on the assessment of expository text comprehension.
Cognition and Instruction, 25(4), 399-438. doi: 10.1080/07370000701632371
Ozyurek, R. (2005). Informative sources of math-related self-efficacy expectations and their
relationship with math-related self-efficacy, interest, and preference. International
Journal of Psychology, 40(3), 145-156. doi: 10.1080/00207590444000249
Pajares, F. & Miller, M.D. (1994). Role of self-efficacy and self-concept beliefs in mathematics
problem solving: A path analysis. Journal of Educational Psychology, 86(2), 193-203.
doi: 10.1037/0022-0663.86.2.193
Parsons, J. E., Adler, T., & Meece, J. L. (1984). Sex differences in achievement: A test of
alternate theories. Journal of Personality and Social Psychology, 46(1), 26-43. doi:
10.1037/0022-3514.46.1.26
Pomplun, M. & Omar, M. H. (1997). Multiple-mark items: An alternative objective item format?
Educational and Psychological Measurement, 57(6), 949-962. doi:
10.1177/0013164497057006005
Radford, L. (2004) Syntax and meaning In M.J. Hoines & A.B. Fuglestad (Eds.), Proceedings of
the 28th Conference of the International Group for the Psychology of Mathematics
Education (Vol. 1, pp. 161-166). Bergen, Norway.
Renninger, K. A., Ewen, L., & Lasher, A. K. (2002). Individual interest as context in expository
text and mathematical word problems. Learning and Instruction, 12, 467-491. doi:
10.1016/S0959-4752(01)00012-3
Scherer, R & Siddiq, F. (2015). The Big-Fish-Little-Pond-Effect revisited: Do different types of
assessments matter? Computers & Education, 80, 198-210. doi:
10.1016/j.compedu.2014.09.003
Schunk, D. H. (1995). Self-efficacy and education and instruction. In J. E. Maddux (Ed.), Self-
efficacy, adaptation, and adjustment: Theory, research, and application (pp. 281-303).
New York, NY: Plenum Press.
Schunk, D. S. (2012). Social cognitive theory. In Harris, K. R., Graham, S., Urdan, T.,
McCormick, C. B., Sinatra, G. M., & Sweller, J. (Eds.), APA educational psychology
handbook, Vol 1: Theories, constructs, and critical issues. Washington, DC: American
Psychological Association.
Schunk, D.H. & Zimmerman, B.J. (2006). Competence and control beliefs: Distinguishing the
means and ends. In P.A. Alexander & P.H. Winne (Eds.), Handbook of educational
psychology (2nd ed., pp. 349-367). Mahwah, NJ: Erlbaum.
Senko, C. & Harackiewicz, J. M. (2005). Achievement goals, task performance, and interest:
Why perceived goal difficulty matters. Personality and Social Psychology Bulletin,
31(12), 1739-1753. doi: 10.1177/0146167205281128
Sfard, A. & Linchevski, L. (1994). The gains and the pitfalls of reification- The case of algebra.
Educational Studies in Mathematics, 26, 191-228.
Shieh, Y. & Fouladi, R. T. (2003). The effect of multicollinearity on multilevel modeling
parameter estimates and standard errors. Educational and Psychological Measurement,
63(6), 951-985. doi: 10.1177/0013164403258402
Tracey, T. J. G. (2012). Problems with single interest scales: Implications of the general factor.
Journal of Vocational Behavior, 81, 378-384. doi: 10.1016/j.jvb.2012.10.001
Trautwein, U., Ludtke, O., Marsh, H. W., Koller, O., & Baumert, J. (2006). Tracking, grading,
and student motivation: Using group composition and status to predict self-
concept and interest in ninth-grade mathematics. Journal of Educational Psychology, 98(4),
788-806. doi: 10.1037/0022-0663.98.4.788
Tyrrell, R., Holland, K., Dennis, D., & Wilkins, A. J. (1995). Coloured overlays, visual
discomfort, visual search and classroom reading. Journal of Research in Reading, 181,
10-23.
Virginia Department of Education (2012). Standard of learning (SOL) and testing. Retrieved
from Virginia Department of Education website: http://www.doe.virginia.gov.
Wheeler, L. & Suls, J. (2005). Social comparisons and self-evaluations of competence. In A.J.
Elliot & C.S. Dweck (Eds.), Handbook of competence and motivation (pp. 566-578).
New York, NY: Guilford Press.
Wigfield, A. & Eccles, J.S. (2000). Expectancy-value theory of motivation. Contemporary
Educational Psychology, 25, 68-81. doi:10.1006/ceps.1999.1015
Wigfield, A. & Eccles, J.S. (2002). The development of competence beliefs, expectancies for
success, and achievement values from childhood through adolescence. In A. Wigfield &
J.S. Eccles (Eds.), Development of achievement motivation (pp. 91-120). San Diego, CA:
Academic Press. doi:10.1016/B978-012750053-9/50006-1
Wigfield, A., Tonks, S., & Eccles, J. S. (2004). Expectancy value theory in cross-cultural
perspective. In D. M. McInerney & S. Van Etten (Eds.), Big theories revisited (pp. 165-
198). Greenwich, CT: Information Age.
Zimmerman, B. J. (1986). Development of self-regulated learning: Which are the key
subprocesses? Contemporary Educational Psychology, 11, 307-313. doi: 10.1016/0361-
476X(86)90027-5
Appendix A: Demographics and Motivational Constructs Measure
Student ID _______________________________________ DO NOT PUT YOUR NAME ON THIS SURVEY
1. Circle your gender: Male Female
2. Circle your grade: 9 10 11 12
3. Write your age: ___________________
For items 4 – 13, circle the appropriate response based on the statement.
4. Mathematics is enjoyable to me.
Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree
5. The amount of effort it takes to do well in high school mathematics courses is worthwhile to me.
Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree
6. In general, mathematics is hard for me.
Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree
7. I have always enjoyed studying mathematics in school.
Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree
8. I feel that, to me, being good at solving problems which involve mathematics is important.
Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree
9. Compared to most other students, mathematics is hard for me.
Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree
10. Compared to other subjects, I feel relaxed studying math.
Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree
11. It is important for me to get good grades in math.
Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree
12. Compared to other subjects, mathematics is hard for me.
Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree
13. Compared to other subjects, mathematics is exciting to me.
Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree
Appendix B: Mathematics Measures
Test A
Question 1 :What is the solution set for this equation?
A:
B:
C:
D:
Question 2 :Type your answer into the box. You must give your answer in integer form.
The following sequence is given in recursive form.
What is the value of the fourth term of this sequence?
Question 3 :Which of the following situations involves a permutation?
A: Determining how many different groups of 3 employees can be chosen from 9 employees.
B: Determining how many different ways 7 runners can be assigned lanes on a track for a race.
C: Determining how many different ways to choose 10 students to attend a field trip from a group of 25 students.
D: Determining how many different ways 4 cashiers can be chosen to work from a group of 7 cashiers.
Question 4 :Click on a box to choose each y-coordinate you want to select. You must select all correct answers.
What are the y-coordinates for the solution to this system of equations?
A: y = -9
B: y = -3
C: y = -2
D: y = 1
E: y = 2
F: y = 6
G: y = 8
H: y = 9
Question 5 :The number of combinations of 7 objects taken 2 at a time is
A: 3B: 7
C: 21D: 42
Question 6 :Type your answer into the box. You must enter your answer in integer form.
The shoe sizes of a large population are normally distributed with a mean of 8.9 inches and a standard deviation of 0.705 inches. What percentage of the population has a shoe size greater than 9.8 inches? ROUND TO THE NEAREST INTEGER.
Question 7 :
Factor:
A: (2x+3) (3x-7)
B: (2x-3) (3x+7)
C: (3x+2) (2x-7)
D: (3x-2) (2x+7)
Question 8 :Select a box for each correct part of the expression. You must select each correct expression.
Select each part of the simplified expression .
A:
B:
C:
D:
E:
F:
Question 9 :The area of a triangle varies jointly with the product of the base and the height. A triangle has a base of 12 feet, a height of 3 feet, and an area of 18 square feet. What is the base of a triangle with a height of 4 feet and an area of 36 square feet?
A: 0.5 feet
B: 9 feet
C: 12 feet
D: 18 feet
Question 10 :Type the answer into the box.The number of permutations of 9 objects taken 3 times is
Question 11 :
Which is a solution of ?
A: x = -5
B:x
= C: x = -
1
D: x = 1
Question 12 :Click on the box to select the value. You must select each correct value.
Two baseballs were thrown on a field as the same time. One ball follows the
path of the function , and the other ball follows the path of
the function , where x is the time in seconds, and f(x) and g(x) are the heights in feet. At what two times, in seconds, are the two balls the same height?
A: 0.7 seconds
B: 1.0 seconds
C: 4.0 seconds
D: 5.14 seconds
E: 5.45 seconds
F: 7.13 seconds
Question 13 :What is the sum of this infinite series?
72 - 36 + 18 - 9 + ...
A: -144
B: -48C: 48
D: 144
Question 14 :Type your answer into the box. You must enter your answer in integer form.
Let and , what is ?
Question 15 :A new rollercoaster at an amusement park follows the path of the
function , where x is the time, in seconds, after the rollercoaster begins, and f(x) is the height of the rollercoaster, in yards. Between which two times, in seconds, is the rollercoaster increasing in height?
A:
B:
C:
D:
Question 16 :Select each expression that is equivalent. You must select all correct expressions.
Identify each expression that is equivalent to 1.
A:
B:
C:
D:
E:
F:
G:
H:
Question 17 :
Which of the following describes the end behavior of as x approaches infinity?
A: y approaches negative infinity
B: y approaches -2
C: y approaches 3
D: y approaches infinity
Question 18 :Type your answer into the box. You must enter your answer in integer form.
The heights of Galapagos penguins are normally distributed with a mean of 49 cm and a standard deviation of 1.82 cm. If a scientist measures the heights of 300 penguins, how many penguins are expected to be between 48.4 cm and 50.1 cm tall? ROUND YOUR ANSWER TO THE NEAREST INTEGER.
Question 19 :Which is the equation of an asymptote of the graph of the following equation?
A: x = -3
B: y = 3
C: x = 6
D: y = 6
Question 20 :Click on the box to select an interval. You must select each correct interval.
Indicate each intervals where the graph is only increasing.
A:
B:
C:
D:
E:
F:
G:
H:
Question 21 :A mathematics class consists of 10 girls and 8 boys. The teacher
wants to choose 2 girls and 2 boys to go on a trip. How many different groups could the teacher choose?
A: 73
B: 146
C: 1260
D: 5040
Question 22 :Type your answer into the box. You must enter your answer in integer form.
If y varies directly with the square root of x, what is the constant of variation if y = 36 when x = 9?
Question 23 :
What are the zeros of the function ?
A: x = -16 and x = 0
B: x = -16, x = -8, and x = 2
C: x = -8 and x = 2
D: x = -2 and x = 8
Question 24 :Click on the box to select the correct equation. You must select each correct equation.
A baseball was thrown by a player, and hit the ground after exactly 5 seconds. If x represents the time in seconds and y represents the height of the ball, which of the following functions could represent the path of the
ball?
A:
B:
C:
D:
E:
F:
Question 25 :A normally distributed set of numbers has a mean of 75 and a standard deviation of 7.97. What percentage of values lies between 70 and 85?
A: 11%B: 37%C: 63%D: 89%
Question 26 :Click on the correct box to select each value. You must select each correct value.
The domain of the function is all real numbers except -
A: -7B: -4C: -3D: 0E: 3
F: 4G: 7
Question 27 :In a school, 12 students are running for 4 class officers- a president, a vice president, a secretary, and a treasurer. If each position is to be held by one person and no person can hold more than one position, in how many ways can those positions be filled?
A: 48
B: 495
C: 11880
D: 20736
Question 28 :Click on each box to select each function. You must select each correct function.
Which of the following functions are in the same family as the
function ?
A:
B:
C:
D:
E:
F:
Question 29 :The steps to simplify an expression are shown below:
Step 1: 4(x+3) - 3x + 1Step 2: 4x + 12 - 3x + 1Step 3: 4x - 3x + 12 + 1Step 4: x + 13
Which of the following properties justifies getting from Step 2 to Step 3?
A: Associative Property
B: Commutative Property
C: Distributive Property
D: Transitive Property
Question 30 :Type your answer into the box. ROUND YOUR ANSWER TO THE NEAREST TENTH.
A baseball player throws a ball from one end of the field to the other. A fan measures the path of the ball and determines that it follows the
function , where x is the time in seconds and f(x) is the height in feet. What is the maximum height of the ball, in feet? ROUND YOUR ANSWER TO THE NEAREST TENTH.
Test B
Question 1 :Type your answer into the box. Enter your answer as a whole number.
What is the solution set for this equation?
Question 2 :The following sequence is given in recursive form.
What is the value of the fourth term of this sequence?
A: 29B: 33C: 61
D: 125
Question 3 :Click on the box to choose the situation. You must select all correct situations.
Which of the following situations involve a permutation?
A: Determining how many different ways to choose 3 employees from a group of 9 employees.
B: Determining how many different ways 7 runners can be assigned lanes on a track for a race.
C: Determining how many different seating charts can be made placing 6 people around a table.
D: Determining how many 5-letter passwords can be made using the
word "graph."
E: Determining how many different groups of 10 students can be chosen to go on a field trip from a group of 25 students.
F: Determining how many different ways 4 cashiers can be chosen to work from a group of 6 cashiers.
Question 4 :What are the y-coordinates for the solutions to this system of equations?
A: y = 1 and y = 9
B: y = -3 and y = -9
C: y = -2 and y = 6
D: y = 2 and y = 8
Question 5 :Type the answer into the box.
The number of combinations of 7 objects taken 2 at a time is
Question 6 :The shoe sizes of a large population are normally distributed with a mean of 8.9 inches and a standard deviation of 0.705 inches. What percentage of the population has a shoe size greater than 9.8 inches? Round to the nearest integer.
A: 5%B: 10%C: 20%
D: 34%
Question 7 :Click on a box for each factor you want to select. You must select all correct factors.
Select all of the factors of:
A: (2x+3)
B: (2x-3)
C: (3x+2)
D: (3x-2)
E: (2x+7)
F: (2x-7)
G: (3x+7)
H: (3x-7)
Question 8 :
Which of the following expressions is equivalent to ?
A:
B:
C:
D:
Question 9 :Type your answer into the box. You must enter your answer in integer form.
The area of a triangle varies jointly with the product of the base and the height. A triangle has a base of 12 feet, a height of 3 feet, and an area of 18 square feet. What is the base of a triangle with a height, in feet, of 4 feet and an area of 36 square feet?
Question 10 :The number of permutations of 9 objects taken 3 times is
A: 27B: 84
C: 504
D: 729
Question 11 :Click on each solution to the equation. You must select each correct solution.
Select all the solutions of .
A: x = -5
B:x
=
C: x = -1
D: x = 1
E:x
=
F: x = 5
Question 12 :Two baseballs were thrown on a field as the same time. One ball
follows the path of the function , and the other ball
follows the path of the function , where x is the time in seconds, and f(x) and g(x) are the heights in feet. At what two times, in seconds, are the two balls the same height?
A: 0.70 seconds and 5.45 seconds
B: 7.13 seconds and 5.14 seconds
C: 0.70 seconds and 7.13 seconds
D: 5.14 seconds and 5.45 seconds
Question 13 :Type your answer into the box.
What is the sum of this infinite series?
72 - 36 + 18 - 9 + ...
Question 14 :
Let and . What is ?
A: -29B: 35C: 75
D: 152
Question 15 :Click on the box to select the interval. You must select each correct interval.
A new rollercoaster at an amusement park follows the path of the
function , where x is the time, in seconds, after the rollercoaster begins, and f(x) is the height of the rollercoaster, in yards. Between which two times, in seconds, is the rollercoaster increasing in height?
A:
B:
C:
D:
E:
F:
Question 16 :Which expression is equivalent to 1?
A:
B:
C:
D:
Question 17 :Type your answer into the box. You must give your answer in integer form.
What value does y approach in the function as x approaches infinity?
Question 18 :The heights of Galapagos penguins are normally distributed with a mean of 49 cm and a standard deviation of 1.82 cm. If a scientist measures the heights of 300 penguins, how many penguins are expected to be between 48.4 cm and 50.1 cm tall? Round your answer to the nearest integer.
A: 84
B: 107
C: 168
D: 204
Question 19 :Click on each box to choose each asymptote. You must select all correct asymptotes.
Which are the equations of the asymptotes of the graph of the following function?
A: x = -3
B: y = -3
C: x = 3
D: y = 3
E: x = 6
F: y = 6
Question 20 :
Throughout which of the following intervals is only increasing?
A:
B:
C:
D:
Question 21 :Type your answer into the box. You must enter your answer in integer form.
A mathematics class consists of 10 girls and 8 boys. The teacher wants to choose 2 girls and 2 boys to go on a trip. How many different groups could the teacher choose?
Question 22 :If y varies directly as the square root of x, what is the constant of variation if y = 36 and x = 9?
A: 1.5
B: 2C: 4D: 12
Question 23 :Click on the box to select the zeros. You must select each correct zero.
Which of the following are zeros of the function ?
A: x = -16
B: x = -8
C: x = -2
D: x = 0
E: x = 2
F: x = 8
Question 24 :A baseball was thrown by a player, and hit the ground after exactly 5 seconds. If x represents the time in seconds and y represents the height of the ball, which of the following functions could represent the path of the ball?
A:
B:
C:
D:
Question 25 :Type your answer into the box. You must enter your answer in
integer form.
A normally distributed set of numbers has a mean of 75 and a standard deviation of 7.97. What percentage of values lies between 70 and 85? ROUND TO THE NEAREST INTEGER.
Question 26 :
The domain of the function is all real numbers except -
A: -7, -4, 4
B: -7, 4
C: -4, 7
D: 4
Question 27 :Type your answer into the box. You must enter your answer in integer form.
In a school, 12 students are running for 4 class officers- a president, a vice president, a secretary, and a treasurer. If each position is to be held by one person and no person can hold more than one position, in how many ways can those positions be filled?
Question 28 :Which of the following is in the same family as the
function ?
A:
B:
C:
D:
Question 29 :Click on the box to select a property. You must select each correct property.
The steps to simplify an expression are shown below:
Step 1: 4(x+3) - 3x + 1Step 2: 4x + 12 - 3x + 1Step 3: 4x - 3x + 12 + 1Step 4: x + 13
Which of the following properties justify Step 2, Step 3, and Step 4?
A: Associative Property
B: Commutative Property
C: Distributive Property
D: Inverse Property
E: Substitution Property
F: Transitive Property
Question 30 :A baseball player throws a ball from one end of the field to the other. A fan measures the path of the ball and determines that it
follows the function , where x is the time in seconds and f(x) is the height in feet. What is the maximum height of the ball, in feet?
A: 2 feet
B: 5.4 feet
C: 6 feet
D: 9.2 feet
Appendix C: Interview Questions
I. Algebra 2 Coursea. Do you find the Algebra 2 course difficult?
i. If so, what aspects?1. Content, rigor, pace, teacher, etc.?
ii. If not, why not?1. Teacher, content, etc.?
b. How important is taking Algebra 2 for you?i. For your academic career?
ii. For your future goals?iii. Why?
1. Content?a. How relevant is the content?
2. Critical thinking?c. What value, in general, do you see in the Algebra 2 course?
i. Preparation1. For what?
ii. Mathematical skills1. Which?
iii. Critical thinking skillsII. Algebra 2 SOL
a. Did you find the Algebra 2 SOL difficult?i. If so, what aspects?
1. Content, length, item format, question types, etc.?ii. If not, why not?
1. Preparation, similar to classroom tests, content, etc.?b. How important is it for you to do well on this test?
i. If so, why?1. College goals, content, career goals, etc.?
ii. If not, why not?1. Irrelevance, content, etc.?
c. If you were to improve the test, how would you do it?i. Change content, length, difficulty, types of questions, etc.?
d. What value, in general, do you see in passing the Algebra 2 SOL?i. Test of mathematical skills?
ii. Critical thinking skills?iii. Comparison to other students?iv. College?