EDCI858 Mathematics Research Synthesis

Michael Mazzarella

George Mason University

Paper (40/ 40)

Nice job pulling the synthesis together on the important variables you are examining through computer based algebra assessment. The focus on tem Format and Question Type will be of great interest to the mathematics education community as we continue to examine ways to assess students’ conceptual understanding and find the most valid assessment design. I look forward to following your work especially the dissertation that you gear up for next. I hope that the opportunity to spend some time on this research synthesis moved you closer to honing in on your proposal. Great to have you in our seminar!

Category Exceeds/Meets Expectations

Needs Revision/Unacceptable

Research Question (RQ) (5 points)

5/5 Question is clearly and coherently describedNo posting or posting is late

Analysis of Previous Studies(15 points)

15/15 Advanced organizer is given to connect major areas of investigation for previous studies and the RQ Analysis includes reference to studies that are clearly connected to the question and from multiple aspects of the question (e.g., students and teachers)Analysis synthesizes research as well as reporting or summarizing results of studies

No advanced organization is providedNo references are cited Incomplete explanation of previous research

Question is not situated within mathematics education research.Related studies are reported but not synthesized.

Theoretical Framework (TF)(15 points)

15/15 TF is clearly and coherently described including significant characteristics Analysis includes multiple references reflecting the chosen framework for the chosen question.

Analysis synthesizes research as well as reporting or summarizing results of studiesDescription includes implications for research methodology Appropriate, cohesive rationale is provided to connect TF with RQ

Characteristics of TF are not clearly described.Related studies are reported but not synthesized.

Very few studies are provided. Limited rationale is given to connect TF and RQ

References (5 points) 5/5 References are cited with APA 6

th formattingMinimal

grammar and stylistic errors

References are not cited, many references are missing Significant grammar and stylistic errors throughout

Algebra Achievement in a Computer-Based Environment: Item Format, Question Type, and Non-Cognitive Factors in High School Students

Within the last decade, many schools and school districts across the country have begun

administering end-of-the-year tests and other standardized assessments on the computer. Taking

these exams on the computer can impact how students perform on tests, even when the content is

the same as that of a paper-and-pencil test (Scherer & Siddiq, 2015). This change can drastically

impact student achievement, especially for high-stakes assessments. For example, in the state of

Virginia, the Standards of Learning (SOL) Assessment has been well-known by teachers,

students, and parents alike. Passing this assessment in almost all of the core high school subjects

is required for graduation (Virginia Department of Education, 2012). In the 2011-2012 school

year, the state of Virginia changed the format of all of the SOL exams in order to make them

more technology-based. All mathematics SOL assessments now include fill-in-the-blank, drag-

and-drop, and graph plotting questions, instead of only multiple choice questions. In the first

year, passing rates for all mathematics SOL exams significantly dropped. Many who are familiar

with the test, however, wonder whether this drop can be attributed to a change in item format, the

introduction to computer-based testing, an increase in difficulty of the exam, or a combination of

those factors.

The purpose of this study is to analyze the relationships among non-cognitive factors

(e.g. interest, task value, and perceived difficulty), computer-based algebra assessments, different

item formats and questions types on tests, and students’ perceptions of the assessments. The

complexity of the problem presented in this study is one that requires extensive review of the

literature. Literature on algebraic thinking encompasses students’ cognitive processes, but

achievement on algebra assessments is also highly influenced by social factors as well.

Expectancy-value literature attempts to define specific factors from the larger Social Cognitive

Theory, and offers measures to quantify these variables. The research on question type also

naturally relates to the algebraic thinking literature in that students often apply knowledge

transfer during tests. Finally, the specific idea of item format provides an added element of

thinking for students, and warrants research on the literature. Thus, it is important to conduct a

widespread review of the literature in order to understand and interpret the results of this study as

best as possible.

Algebra and Algebraic Thinking

From a mathematics perspective, algebraic thinking is a construct that is vital to student

success on mathematics assessments. Kaput (1998) categorized algebraic thinking into five

strands:

1. Algebra as generalizing and formalizing patterns and regularities, in particular,

algebra as generalized arithmetic

2. Algebra as syntactically guided manipulations of symbols

3. Algebra as the study of structure and systems abstracted from computations and

relations

4. Algebra as the study of functions, relations, and joint variations

5. Algebra as modelling

Kaput argued that, while all strands should be incorporated into learning to maximizing algebraic

thinking, much of the algebra curriculum in classrooms today focuses only on the second and

fourth strands. He further studied the extent to which “spontaneous algebraic reasoning” (SAR)

and “planned algebraic reasoning” (PAR) were used effectively in an activity-based urban

classroom. For instance, students took part in a lengthy discussion on the results of adding two

odd, two even, or an odd and even number. The authors found that when algebra was presented

as authentic problem solving, the number of SAR instances outweighed the number of PAR

instances. Furthermore, students were able to learn and understand concepts that were well

beyond their grade level, such as the representation of an odd number of 2n + 1 (Blanton &

Kaput, 2005). The authors theorized that algebraic thinking is enhanced when the content is

presented in realistic and relevant problems.

One important aspect of algebraic thinking comes from students making algebraic

meaning from the mathematics. Finding meaning within mathematics is vital in order for

students to understand abstract algebraic ideas and make connections among the numbers and

symbols (Sfard & Linchevski, 1994). Radford (2004) sorted meaning into three categories:

algebraic structure, problem context, and exterior of the problem context. Kieran (2007) split that

first category down further into two subsections: meaning from the letter-symbolic form of the

algebra, and meaning from multiple representations. For example, in terms of symbolic and

multiple representations, it is important that a traditional curriculum, such as literal expressions

and equations, be merged with authentic problems that show graphs, tables, and pictures (Kaput,

1989). This corresponds with Kaput’s argument that only one strand of algebra is being used in

today’s classroom.

Another viewpoint of algebraic thinking is that of seeing algebra as “activity.” Lee (1997)

offered several definitions of algebraic activity. One such definition is performing an action on

algebraic symbols, while another definition uses the notion of building algebraic objects.

Nevertheless, Lee emphasizes the idea of algebra as an active process. According to Kieran

(1996), algebra activity can be modeled using three types: generational, transformational, and

global/meta-level. On the generational level, algebra is the use of language to solve unknown

variables or express patterns and relationships among numbers and symbols. Transformational

activity refers to the manipulation of expressions, such as factorization, substitution, expansion,

or performing operations on polynomials. The global/meta-level of algebra refers to activities

that are not specific to algebra, but rather overall mathematical processes. These three levels of

activity are all interconnected according to Kieran’s “GTG” model of activity. The importance of

these activities, according to Kieran, as vital to developing student algebraic thinking,

particularly in young mathematics students.

Using technology in the algebra classroom has become a growing trend in the past

decade. On an upper algebra level, graphing calculators are commonplace for all students and

teachers. Doerr and Zangor (2000) claimed that students use graphing calculators in five

different ways: computation, transformation, visualization, verification, and data collection and

analysis. O’Callaghan (1998) studied the effects of graphing calculators on students’

understanding of algebra. The results showed that students who used graphing calculator during

the lessons showed a stronger conceptual understanding of higher level functions based on the

researcher’s test of functions. The same students scored higher on their end-of-the-year

assessment than students who did not used the calculators.

Social Cognitive Theory

According to Albert Bandura (1986), social cognitive theory states that students’

cognition is influenced by their behaviors, which are caused by personal and environmental

factors. Bandura, a pioneer researcher on the theory, students’ values impact the ways in which

they approach their academic careers. Furthermore, students frequently set academic and social

goals for themselves based on how important they perceive a task to be. These goals are often

based on the goals of their peers and the social environment around them (Bandura & Schunk,

1981). There is also a relationship between students’ values and their behavior (Wigfield, Tonks

& Eccles, 2004). In summary, Schunk (2012) stated that students learn a tremendous amount

from their social environment, and that social environment also influences their academic

success.

One of the most important pieces of conceptual frameworks of social cognitive theory is

triadic reciprocality. Triadic reciprocality describes the interaction among three influences:

person, behavior, and environment. The person in this model describes the students’ cognitive

skills and beliefs. Each factor is influenced by one another, both within and outside of the

classroom. For example, a student’s behavior can be influenced by both his personal beliefs and

the social environment around him, but conversely, his behaviors also affect those around him

and how he perceives himself (Bandura, 1977). This triadic influence can also happen at a group

level. For instance, a group of teachers may decide to adjust curricula and lessons if there is an

overall low achievement in a school (Schunk, 2012). While many constructs are at play within

this model, the relationships described by triadic reciprocality adequately summarize the main

tenants of social cognitive theory.

Schunk (2012) proposes that there are five central symbolic processes that occur within

social cognitive theory, particularly at the mathematics level. These processes, which help

describe student cognition during problem solving, include self-efficacy, goals, outcome

expectations, values, and social comparisons. Self-efficacy refers to a student’s beliefs in their

abilities to perform on a given task. This construct is a part of the “person” aspect of triadic

reciprocality in that a student’s self-efficacy influences his/her behavior and environment, and is

also influenced by these two as well (Bandura, 1997). Goals can refer to the progress and

outcomes that students achieve throughout their education. One important note is that there is a

difference between learning goals and performance goals. While learning goals are defined as

what knowledge students hope to achieve regardless of grades, performance goals refer to how

well students want to do on assessments and classroom grades (Anderman & Wolters, 2006).

Outcome expectations are often formed based on prior performance and are content specific; that

is, a student’s outcome expectation in one subject is not strongly influenced by their outcome

expectation in another class (Schunk & Zimmerman, 2006). Value is a broad term that can refer

to the importance of a task in different contexts. For instance, a student may view a particular

math task as valuable because doing well in school is valuable (i.e. attainment value) but does

not see the value in that task in terms of their career goals (i.e. utility value; Wigfield & Eccles,

2000). Social comparison is important not in the mere fact that it occurs, but in with whom

students compare themselves. It is most beneficial for students to compare themselves to those

who have similar abilities (Wheeler & Suls, 2005). In the present study, variations of values and

outcome expectations will be measured and analyzed.

Social cognitive characteristics of students have been found to impact mathematics

achievement. Lopez and colleagues (1997) measured math performance and social cognitive

constructs, such as interest, self-efficacy, and outcome expectations, in algebra students. The

path analysis models produced for the variables showed that social persuasion and perceived past

performance were both significant predictors of self-efficacy, which was a significant predictor

of students’ grades. Additionally, self-efficacy fully mediated the relationship between a

standardized mathematics test and classroom grades. In terms of interest, self-efficacy and

outcome expectations were both found to significantly predict interest (Lopez et al., 1997). In a

similar study, Pajares and Miller (1994) state that self-concept and self-efficacy both have a

direct effect on mathematics performance. This study also reported gender differences; despite

similar prior experience, men reported higher self-efficacy and had a higher math performance.

The results of these studies justify the need for further research on non-cognitive factors and

mathematics achievement.

Expectancy-Value Theory, Attainment Value, and Achievement

Social cognitive theory is a very broad theory that encompasses several other smaller

theories, each of which consist of several non-cognitive constructs. One smaller theory that lies

within social cognitive theory is expectancy-value theory. Schunk (2012) claims that

“expectancy-value theory bears much similarity to the social cognitive emphasis on goals, self-

evaluation of progress, self-efficacy, and outcome expectations” (p. 117). Atkinson (1957), one

of the founders of this theory, explained that expectations and values influence students’ beliefs

toward a task. He also popularized the term “task value,” which is a broader construct under

which attainment value exists, and argued that task value was intrinsic and could be measured in

terms of students’ pride in succeeding (Atkinson, 1957). While expectancy-value theory became

less popular after Atkinson’s publications on the topic, a “modern” version of the theory was

revived in the late 1900s. Some emerging questions in the field of educational psychology

regarding the “modern” expectancy-value theory included the relationships among expectations,

task value, and achievement-related behavior, as well as whether or not these beliefs change over

time (Graham & Weiner, 2012). Eccles and Wigfield (1995), some of the most important

researchers in the revival of the theory, claimed that task value consisted of four aspects:

attainment value, interest, utility value, and cost. Attainment value and interest, two constructs

that will be used in the present study, are both positively correlated with achievement in a given

task (Wigfield & Eccles, 2002).

Attainment value can be defined as how important a student feels it is for him/her to

succeed on an assessment, perform well in a subject, or apply a subject to their lives (Eccles &

Wigfield, 1992). Attainment value is a specific aspect of the expectancy-value theory. It is a

motivation construct that is content-specific. Bong (2002) studied the task value, self-efficacy,

performance goal, and mastery goal of several students across several subjects in middle school

and high school. The results showed that task value in high school was one of the most domain-

specific variables in the study. Specifically, in terms of mathematics, task value was correlated

with Korean, English, and science task value in high school at .06, .17, and .44, respectively, and

in middle school at .36, .37, and .45, respectively. However, as expected, task value was

significantly correlated with self-efficacy, performance goals, and mastery goals (Bong, 2002).

These findings suggest that task value for mathematics is a unique construct and, thus, it is

important to distinguish it from other similar constructs when measuring it.

Task value can also differ between males and females. Gaspard and her colleagues (2014)

measured several distinct constructs related to motivation in mathematics for ninth grade

students. After a factor analysis confirmed the distinction between interest, attainment, utility,

and cost constructs, which are the four aspects of task value according to Eccles and Wigfield

(1995), the results of this study showed that there are significant differences between males and

females in intrinsic value (i.e. interest), but not importance in achievement (i.e. attainment

value). Despite this difference, the components of the factor analysis of intrinsic and attainment

were significantly correlated (Gaspard, Dicke, Flunger, Schreier, Hafner, Trautwein, &

Nagengast, 2014). In addition to analyzing the relationship between the two motivational

constructs used in the present study, the aforementioned research also suggests that it is

necessary to further analyze the differences in gender for these constructs.

Personal Mathematics Interest and Achievement

In addition to attainment value, personal interest in a certain subject is another

motivational construct used in predicting achievement. Dewey (1913) defined interest as an

object, subject, or idea that becomes an accompanying part of one’s identity. In the context of

education, many researchers use “interest” and “intrinsic value” interchangeably. Intrinsic value

is another aspect that Eccles and Wigfield (1995) pinpointed in their four aspects of task value. It

is often found that intrinsic value and attainment value are strongly correlated, and when both

constructs are measured as high, achievement is also high (Durik, Vida, & Eccles, 2006).

Mitchell (1993) specified educational interest as interest directly tied to the content of

instruction. The present study will use the same definition as that of Mitchell (1993). Students’

interest may also differ based on the different types of problems. Renninger, Ewen, and Lasher

(2002) studied three cases of students with varying levels of mathematics interest and analyzed

how that interest related to mathematics word problems. The researchers found that even when

word problems were personalized to students’ mathematics abilities, high interest resulted in

higher achievement. Furthermore, higher interest can also result in students rereading problems

to understand their contexts or checking their answers to make sure that they makes sense in the

context of the problems (Renninger, Ewen, & Lasher, 2002).

Personal interest in mathematics varies throughout a student population. Trautwein,

Ludtke, Marsh, Koller, and Baumert (2006) measured interest in ninth grade students on

different mathematics tracks. The researchers found that interest in mathematics was

significantly higher in students in the upper track, but there was little difference in interest for

students in the middle or lower tracks. Interest was also found to be significantly correlated to

both individual achievement on standardized mathematics tests and the overall school

achievement on the same standardized tests (Trautwein, Ludtke, Marsh, Koller, & Baumert,

2006). The same researchers also found marginally significant results stating that there were

reciprocal effects on mathematics interest and self-concept; in other words, rather than one

construct impacting the other, both interest and self-concept impact each other. These results

were also found to be generalizable across gender (Marsh, Trautwein, Ludtke, Koller, and

Baumert, 2005). Other studies have also found personal interest to be related to other

psychological constructs. Ozyurek (2005) found statistically significant correlations between

interest in a mathematics class and self-efficacy, subject preference, previous mathematics

performance, and class expectations. These results were also consistent for undergraduate

students who were mathematics majors and not mathematics majors (Ozyurek, 2005). Therefore,

many factors contribute to one’s personal interest, including mathematics achievement, which is

a variable to be used in the current study.

Perceived Task Difficulty and Achievement

The way that a student personally views how complicated a certain subject or assignment

is can be referred to as perceived task difficulty (Midgley, Feldlaufer, & Eccles, 1989). Perceived

task difficulty is also categorized as a construct under the expectancy-value theory of Atkinson

(1957). Specifically, students’ expectations of a task impact their motivation and, thus, their

achievement. Another early definition of this construct explains it as the subject probability that

the task will result in success or failure (Karabenick & Youssef, 1968). Moreover, perceived task

difficulty can fall under the realm of social cognitive theory. Students engage in tasks in which

their expected outcome is high; however, when the task is expected to be difficult, it will impact

their performance in that specific task (Schunk, 1995). Midgley, Feldlaufer, and Eccles (1989)

found that perceived task difficulty can be affected by teacher efficacy, expectancies, previous

performance in a subject, or current classroom grades in a subject. Therefore, it is worthwhile to

study the relationships among perceived task difficulty, interest, attainment value, and algebra

achievement.

As mentioned previously, many motivational constructs are strongly correlated with one

another. Fulmer and Tulis (2013) analyzed the interest and perceived difficulty of middle school

students before, during, and after a difficult reading task. The results of the study showed that

interest was not significantly correlated with perceived difficulty until the final interest measure

after the reading, which, at the point, was negatively correlated. The researchers also measured

affect, which was defined in the study as a spectrum of positive and negative emotions (Fulmer

& Tulis, 2013). The study found that situational affect, which was defined as a student’s

emotional response to a particular task, was significantly correlated with perceived task difficulty

during and after the task, but not before it (Fulmer & Tulis, 2013). Not only do the researchers

introduce an interesting concept of measuring motivational constructs at different times during a

task, but they also supported prior research that these constructs were positively correlated.

Senko and Harackiewicz (2005) performed a similar study, which compared interest with not

only perceived task difficulty, but also the variable of introduced perceived goal difficulty, which

was defined as the extent to which students believe they can achieve goals that they set for

themselves. The researchers found that perceived goal difficulty was a mediating variable

between several other motivational constructs (Senko & Harackiewicz, 2005).

Similar to the studies done on attainment value, research suggests that there are gender

differences in perceived task difficulty. Parsons, Adler, and Meece (1984) conducted a

groundbreaking study that compared differences in motivational constructs between gender and

subject (English and mathematics). Results showed that all students, regardless of gender,

perceived mathematics as more difficult than English. However, despite this significant

difference between subjects, there was very little difference in perceived task difficulty between

males and females (Parsons, Adler, & Meece, 1984). Although this study was designed well and

had measures relevant to the present study, it was conducted over thirty years ago, so it should be

replicated to analyze present-day implications.

Research shows that the three motivational constructs outlined above (personal interest,

attainment value, and perceived task difficulty) all individually impact mathematics

achievement. However, there is less research regarding the relationship among the three as they

relate to computer-based mathematics assessments, particularly with different item formats and

question types.

Assessments: Item Format

In the present study, item format refers to the ways in which students can respond to the

prompts. Some item formats include multiple choice, true or false, selecting multiple correct

answers, and fill-in-the-blank. Many research studies have been conducted regarding the

relationship between item format and achievement. Ozuru, Best, Bell, Witherspoon, and

McNamara (2010) measured achievement on a reading assessment. Items were separated into

multiple choice and open ended questions. Overall, students scored the multiple choice questions

correctly more often than the open ended questions. The average score for the multiple choice

section without and with available text was 68% and 79%, respectively, while the average score

for the open ended section without and with available text was 48% and 60%, respectively

(Ozuru, Best, Bell, Witherspoon, & McNamara, 2010). The large gap between the scores of the

multiple choice and open ended questions suggests that students think about each type of

question differently, which affects achievement on assessments.

Item format may be particularly important when the exams are computer-based. For

instance, Jodoin (2003) studied responses on an engineering exam that used multiple choice and

“innovative” items, such as drag-and-drop and selecting multiple answers. Results showed that

examinees answered multiple choice items with more accuracy, even when items of different

formats were asking about the same subject (Jodoin, 2003).

Not only can different item formats produce different achievement scores, but they may

result in different strategies used. For example, Katz, Bennett, and Berger (2000) had students

record the strategies used for each item on an assessment with multiple item formats. Traditional

strategies included using formulas and solving algebraic equations, while nontraditional

strategies included estimation and guess-and-check. Results showed that while item format for

some questions changed their levels of difficulty, the strategies used for different questions did

not affect the achievement (Katz, Bennett, & Berger, 2000).

It is important to note that measures with different item formats can threaten validity in

several ways. For example, in one of the first articles looking at computer-based assessments

with different formats, Martinez and Bennett (1992) analyzed several types of mathematics skills

(e.g. algebraic reasoning, computer science) that were tested using different item formats.

Psychometric analysis determined that there was little discrepancy between computerized raters

and human raters, but in some cases, scores differed by up to 1.2 points on a 16-point scale

(Martinez & Bennett, 1992). Because of this small difference, computerized grading gained more

trust; nevertheless, even a small difference such as this highlights the importance of calculating

validity and reliability of computer-based assessments.

In a similar article, Pomplun and Omar (1997) outlined four different threats to the

validity of an assessment: lack of familiarity of the item format, omitting alternatives,

dependency among alternatives, and guessing. In their study, two specific types of item formats

used were “multiple mark” (a multiple choice question with more than one correct answer) and a

“multiple true-false question” (similar to multiple mark, but only with T/F as possible choices).

The results of this article stated that omitting answers or not following directions did not

seriously threaten validity, but students tended to leave choices blank instead of guessing

(Pomplun & Omar, 1997). Further threats to validity will be discussed in the “Limitations”

section of this study.

Due to the recent transition from paper-and-pencil to computer-based assessments, the

research on item format is minimal, especially in the mathematics field. Further research is

needed to determine how item format affects different content areas. Additionally, non-multiple

choice questions are often grouped into one “constructed response” category, but their

differences on a computer-based test are not distinguished. There is also little research on how

item format impacts different types of mathematical questions.

Assessments: Question Type

Assessing mathematics knowledge is a broad idea that can be categorized into multiple

aspects. Thus, it is important to study differences in the ways that questions are asked on

mathematics assessments. In particular, two such types of mathematics problems are

computational (i.e. straightforward) problems and word problems. Fuchs et al. (2008) studied

whether or not different aspects of cognition (e.g. language, concept formation, and working

memory) were used in different types of problems. Results showed that correlations between

computational and problem-solving skill was only moderate. Cognitively, processing speed was

highly correlated with computational skill, but not with problem-solving skill. Working memory,

on the other hand, was more highly correlated with problem-solving skill than with

computational skill (Fuchs et al., 2008). The same study also compared demographics to

achievement on different question types. For example, poverty and race had little effect on

difficulty with computational problems, but those who had difficulty with solving word problems

were poorer and more likely to be African-American (Fuchs et al., 2008). Therefore, it is

certainly worthwhile to study question type as it relates to mathematics achievement.

When studying question type, one must take into consideration the construct of

knowledge transfer. Knowledge transfer can be defined as the ability to relay knowledge that

students learned one way to a task presented in a different way (Belenky & Nokes-Malach,

2013). In the present study, knowledge transfer is relevant regarding question type because

students may have learned a mathematics procedure in a straightforward manner, but may not be

able to apply that knowledge to a word problem, for example. Knowledge transfer also relates to

assessments from an information processing viewpoint. Working memory is necessary for

students to be able to transfer, organize, and apply their knowledge to a given task (Belenky &

Nokes-Malach, 2013). Day and Goldstone (2012) stated that despite cognitive load or item

difficulty, student transfer was high when students were shown several examples of the problem.

They went on to claim the following: “Contextual similarity between the situations themselves

seems to play a much larger role in determining whether transfer will actually occur” (Day &

Goldstone, 2012, p. 155).

Self-regulation can also play a role in students’ transfer abilities. According to

Zimmerman (1986), an early pioneer of the construct, self-regulation is defined as actively

participating in one’s own learning. This can take place cognitively, metacognitively,

motivationally, or behaviorally (Zimmerman, 1986). Kramarski, Weiss, and Sharon (2013)

experimented with an intervention that attempted to increase students’ transfer abilities among

mathematics tasks by increasing self-regulation. The researchers gave students a survey

measuring three aspects of self-regulation (planning, monitoring, and evaluation), procedural

knowledge algebra tasks, and verbal algebra problem-solving tasks, which they classified as

“long-term transfer to novel tasks.” The results showed that while there was no significant

difference between the two learning approaches for the procedural knowledge tasks, there were

significant differences between the two learning approaches for the problem-solving tasks,

including all three specific types of mathematical categories (algebraic, number sense, and

visualization; Kramarski, Weiss, & Sharon, 2013). Those who were part of the intervention

group tended to be higher in self-regulation, which led to high scores on mathematical tasks that

were classified as “far transfer” tasks (number sense and visualization; Kramarski et al., 2013).

Overall, knowledge transfer is another construct to consider in all aspects of assessments,

especially question type and item format.

Gaps in the Literature

Despite previous research on these topics, there are several gaps in the literature. First,

although Renninger, Ewen, and Lasher (2002) claim that mathematics interest can impact

achievement on different question types, their study was done using a paper-and-pencil

assessment. For that reason, it is worthwhile to study whether similar results will occur with a

computer-based assessment. Second, while there is much research on item format and question

type individually as they relate to achievement on a computer-based test, there is little research

comparing the effects of the interactions between item format and question type on achievement

on a computer-based test. Third, more research is needed on the relationships among

motivational constructs and items formats. While many research studies have been done

comparing interest, task value, and perceived difficulty with mathematics achievement, little is

known about their impact on achievement on a computer-based assessment with different item

formats. Finally, while there has been much research on the results of computer-based

mathematics assessments, there is little research how students view and perceive these types of

exams, especially at the high-stakes level.

Research Questions

The following research questions hope to address the gaps in the literature regarding

computer-based mathematics assessments, mathematics interest, attainment value, and perceived

task value:

1. To what extent do personal interest, attainment value, and perceived task difficulty

predict algebra achievement on a computer-based assessment with different item formats

and question types?

2. What are students’ perceptions on a computer-based standardized math test, in terms of

relevance, value, and importance?

Researcher Identity

When conducting qualitative or mixed-methods research, it is important to consider the

researcher’s experiences, beliefs, and identity. Knowing this information can help the reader

understand the knowledge that the researcher comes into the study with, as well as their purpose

in conducting the study. As a math teacher myself, I have tremendous experience with teacher

identity and standardized testing. For four years, I have had to prepare students for the end-of-

the-year assessment in my courses. This assessment was not only important for the students, but

it also reflected on my “success” as their teacher; over half of my professional evaluation was

based on these exams. This shaped my identity as a teacher in several ways. The most prevalent

way is that I found myself teaching to the test, especially in my first year on the job, rather than

focusing on students conceptually understanding the material. This is partially due to my belief

that the new test is, in fact, difficult for students. I was so nervous about how I would get graded,

that I focused less on what was truly important: student understanding. I continue to find a

balance between preparation for a test and teaching students critical thinking skills, but it

certainly has influenced my role as a teacher. Nevertheless, I have often wondered if this is how

the teaching profession has always been. I was aware of the changes that took place my first

year, but I wondered how different it actually was. Within the school, I see myself as a team

player who shares materials and strategies with other teachers. I also see myself as somewhat of

a leader; other teachers come to me for help or advice regarding students or lessons. My identity

as a teacher shapes this research because I have a better understanding of the school environment

in which the participants work.

In addition to being a high school mathematics teacher, I am also currently pursuing a

Ph.D. in Education degree at George Mason University. My primary concentration is educational

psychology, and my secondary concentration is mathematics leadership. At the time of this

study, I am beginning my third year in the program. My research interests include secondary

mathematics, computer-based standardized tests with different item formats, and motivation

constructs based on the expectancy-value theory (e.g. personal interest, task value). Before the

time of this study, the only analysis that I had conducted was quantitative. I had taken several

upper-level statistics classes and felt very comfortable conducting quantitative analysis.

Furthermore, I had even created and conducted a quantitative pilot study. As a result, I came into

this study feeling very uncomfortable with qualitative analysis. Therefore, I am still not fully

comfortable with qualitative or mixed-methods studies, their procedures, and the analysis that is

conducted from qualitative data.

Method

Participants

The participants of this study will be 225 high school students (n = 225) currently

enrolled in an on-level Algebra II course in a large suburban school district during the 2014-2015

school year. These students will be selected based on a sample of convenience. The 225 students

will come from a total of nine on-level high school Algebra II classes, four of which are taught

by the researcher and five of which are taught by three other teachers in the same school. The

three teachers assisting the researcher with this study are on-level Algebra II teachers at the same

school. These teachers have two, seven, and twenty years of experience teaching within the same

school, while the researcher has four years of teaching experience within the same school. The

present study has been fully explained to the other three teachers, and they have agreed to help in

the data collection process and allow their students to participate in the study.

The sample will consist of 113 females (n = 113) and 112 males (n = 112). The ages of

the students at the time of the study will range from fifteen years to nineteen years. The ethnicity

of the students is expected to be similar to that of the school demographic: About 33% of

students will be Hispanic, 25% of students will be Asian, 24% of students will be white (not of

Hispanic origin), 18% of students will be African-American (not of Hispanic origin), and 2% of

students will be listed as “other.” Approximately twenty students will be categorized as limited

English proficiency. About 55% of students will receive free or reduced lunch on a daily basis.

Approximately 5% of students will be categorized as special education. The school at which the

data is being collected is a public school within a large, diverse, suburban community. The

school has approximately 2,300 students, and the ethnicity of those students will be

approximately the same as that of the sample (source not cited to ensure confidentiality).

Measures

Personal mathematics interest (Mitchell, 1993). The self-report measure will consist of

items measuring interest, attainment value, and perceived task difficulty. On this survey, there

will be four items measuring personal mathematics interest. The survey consists of four Likert-

scale items measuring students’ personal interest in mathematics. One example of an item asks

students to evaluate this statement: “Compared to other subjects, mathematics is exciting to me.”

Students will respond to each item by circling “strongly agree,” “agree,” “slightly agree,”

“slightly disagree,” “disagree,” or “strongly disagree.” The internal consistency coefficient of

this measure was found to be .92, which suggests that the measure is very reliable (Mitchell,

1993).

Attainment value (Eccles & Wigfield, 1995). On the same survey containing the four

interest items, there will be three items measuring attainment value. One example of an item

measuring attainment value is the following: “It is important for me to get good grades in math.”

These items will be answered on the same Likert-scale as the interest items. The internal

consistency of these items is .70 (Eccles & Wigfield, 1995).

Perceived task difficulty (Eccles & Wigfield, 1995). On the same survey containing the

interest and attainment value items, there will be three items measuring perceived task difficulty.

One example of an item measuring perceived task difficulty is the following: “In general,

mathematics is hard for me.” These items will be answered on the same Likert-scale as the

interest and attainment value items. The internal consistency of these items is .80 (Eccles &

Wigfield, 1995).

Demographics. In addition to the ten Likert-scale items, students will also record their

gender, grade level, and age on the survey. The items of this survey can be found at the end of

this document in Appendix B.

Mathematics achievement (Mazzarella, 2015). Students will be given a set of thirty

mathematics items on the computer through the program Horizon, which is a commonly used

computer-based assessment program in the county in which the present study will be being

conducted. This program allows teachers to create items of different formats, and teachers will

receive the students’ results for each question when the assessment is complete. The items for

this measure can be found in Appendix C.

The curriculum used to create these mathematics items aligns with the standards set forth

by the county in which this study will be conducted. These standards also match the standards

used to create the end-of-year state assessment that students in Algebra II are required to take.

Some examples of standards included in the mathematics measure are solving radical, absolute

value, and rational equations, finding the domain and range of various functions, simplifying

rational and radical expressions, identifying properties of a normal distribution, and recognizing

and solving permutations and combinations. According to the state in which the study will be

taking place, there are four strands (i.e. standards) that categorize Algebra II test questions:

Expressions and Operations, Equations and Inequalities, Functions, and Statistics (Virginia

Department of Education, 2012). It is also worthwhile to note that students in the county are

required to complete and pass Algebra II in order to graduate. This is important to keep in mind

because it implies that students with a variety of skill levels will be measured in the present

study, rather than only students who choose to enroll in an Algebra II course without it being

required.

Procedures

In order to obtain assent from students and consent from the students’ parents or

guardians, the researcher will speak with all classes about the purpose of this study, the measures

being used, the potential of being chosen randomly to be interviewed, the types of questions that

would be asked on interviews, the students’ optional participation in this study, and the

confidential data being collected. Students will then be given the consent and assent forms, and

they will be instructed to read and sign the assent forms and have their parents read and sign the

consent forms. Once the forms have been read and signed, the students will be instructed to

return both forms to their teacher. The signed forms will be kept in a secure file cabinet in the

researcher’s room until the study has been completed.

Participants will first take a survey in which they self-report their levels of interest,

attainment value, and perceived task difficulty in mathematics. The survey consists of ten Likert-

scale items and three questions pertaining to demographics, all of which will take approximately

two minutes to complete. Students will be asked to put their student ID number, but not their

name, on the survey. A teacher other than the students’ Algebra II teacher will be in the room

while the survey is administered to ensure confidentiality.

One class period after taking the self-report survey, students will take a measure of

mathematics achievement. Students will have one full class period (90 minutes) to complete this

test. Once students complete the test, the responses and scores will be linked to a student ID

number that matches that written on the survey. A teacher other than the researcher will match

the responses of the survey and the math measure, enter the data into a Microsoft Excel file,

replace the student ID numbers with a different unidentifiable ID number, and give the data to

the researcher.

Finally, after all measures have been administered, five students will be randomly

selected to participate in a one-hour long interview with the researcher. From the beginning of

this process, participants were informed that the interview would be recorded, but the recording

would only be used for transcription purposes and their voices would not be used. Additionally, I

would use a pseudonym for their name, school, and any other identifying information. The

questions used in the interview can be found in Appendix C. These questions will act more as

guidelines rather than strict required questions. As an incentive, all students participating in the

study (whether or not they were interviewed) will be entered into a raffle to win one of four $25

Target gift cards. Each of the three teachers assisting with the study will also receive a $25

Target gift card.

Research Design

All students will take the same self-report survey. The items in the survey are staggered

based on construct. For example, the first item measures interest, the second item measures

attainment value, the third item measures perceived task difficulty, and the remaining items

follow that same pattern. Items are staggered in this way in an effort to separate similar

questions.

Classes will be randomly assigned to take one of four versions of the math measure: Test

A, Test B, Test C, or Test D. Test A and Test B have the same prompts in the same order. On

Test A, the odd-numbered questions will be multiple choice questions, and the even-numbered

questions will be technology-enhanced questions (either fill-in-the-blank or selecting multiple

correct answers). On Test B, the even numbered questions will be multiple choice questions, and

the odd numbered questions will be technology-enhanced questions. For example, on both tests,

Question #1 asks students to solve a radical equation. On Test A, students will answer that

question in multiple-choice format, while on Test B, students will answer that same question in

fill-in-the-blank format. Test questions are staggered this way so that data will be collected for

every prompt in both a multiple choice and technology-enhanced format. On both Test A and

Test B, there will be twenty straightforward mathematics problems (ten multiple choice and ten

technology-enhanced) and ten real-world application problems (five multiple choice and five

technology-enhanced). Creating the assessments in this way yields four categories:

straightforward multiple choice, straightforward technology-enhanced, word problem multiple

choice, and word problem technology-enhanced. Both tests have the same number of questions

in each category. These tests were created in this way because the data collected will provide a

comparison of the same prompts with different item formats, as well as a comparison between

straightforward questions and word problems. The items in Test A and Test B can be found at

the end of this document in Appendix B.

Test C will have the same exact questions and formats as Test A, but the questions will

be in reverse order. Test D will have the same exact questions and formats as Test B, but the

questions will be in reverse order. Test C and Test D were created to account for test fatigue.

Tyrrell and colleagues (1995) support that visual and mental fatigue can occur as students take

assessments, especially when the assessments are taken on the computer. As students work

through the thirty-question math measure, some may become less motivated or energized toward

the end of the test. Thus, it is important to test whether or not this occurs in the measure before

drawing conclusions about specific test questions or overall achievement.

As mentioned previously, the results of the mathematics measure will be available online

to the student’s teacher once the test is complete. Results will include the number of correct

answers and the responses to each multiple choice and technology-enhanced question. The

results will also be separated based on question type (i.e. whether the question was

straightforward or a word problem). All students will have access to a graphing calculator during

the mathematics measure.

After the students have completed the measures, the five students that were chosen

randomly from the sample to be interviewed will be contacted. The interviews will be scheduled

at a time and place of convenience for the participant. Communication will primarily be

conducted via e-mail to schedule the interview, but the interviews themselves will actually be

conducted in person. The interviews will be recorded using an iPhone 6.

Data Analysis

All data analysis will be conducted using the computer programs Microsoft Excel, SPSS,

jMetrik, Mplus or NVivo. A data analysis matrix organizer can be found at the end of this

document in Appendix D. The first research question asks the following: To what extent do

personal interest, attainment value, and perceived task difficulty predict algebra achievement on

a computer-based assessment with different item formats and question types? Several types of

analysis will be used to address this question. First, descriptive statistics will be calculated for

each of the four versions of the mathematics measure. These descriptive statistics, which include

mean and standard deviation, will compare students’ overall achievement on each test. Second,

correlation coefficients will first be calculated to look at the relationships among personal

interest, attainment value, and perceived task difficulty. Ideally, the correlation coefficients will

be less than .80 to avoid multicollinearity (Shieh & Fouladi, 2003). Next, a confirmatory factor

analysis will be run using the survey items in order to confirm that three constructs are being

measured and that the model is an acceptable fit. If the model is not an acceptable fit, certain

items may be removed, and further analysis will be run with the adjusted number of items.

Finally, two separate multiple regression analyses will be run. The first regression will test how

well the three motivational constructs and question types predict achievement on multiple choice

questions. The second regression will test how well the three motivational constructs and

question types predict achievement on technology-enhanced items. In both regression models,

question type will be coded as 0 for straightforward problems and 1 for real-world problems.

The second research question asks: What are students’ perceptions on a computer-based

standardized math test, in terms of relevance, value, and importance? In order to answer this

question, student interviews will be conducted, transcribed, and coded. During the interview, the

researcher will take notes based on the interviewee’s responses. The purpose of these notes will

be to record initial reactions, mark unanticipated themes, and note when certain responses

correspond to those of other participants. The coding of the transcripts, which will be done by

hand using Microsoft Word, will be approached with some initial codes that the researcher will

look for (e.g. item format, pressure, value, importance, relevance), but will mostly be organized

accordingly once the researcher looks for patterns among the responses. The NVivo program will

allow the researcher to create “nodes,” which act as codes, as well as “sub-nodes,” which are

used if there are several common responses regarding a larger theme. This will allow the coding

to be organized in an outline format. Once the coding is complete, the researcher will consolidate

and combine nodes on NVivo in order to create a more organized system of codes. From this

method of organization, the researcher will then look for themes, connections, and conclusions

that can be found in the data.

Limitations, Further Research, and Educational Implications

There are several limitations in regard to this study. One such limitation is the validity of

a self-report survey. It is important to take into account the validity of each measure when

collecting data. In particular, the measure of interest should be carefully examined. Tracey

(2012) claims that single interest scales contain two types of error: systematic error and general

factor variance. Furthermore, these scales do not take into account bias that can influence

students’ interest. For example, interest is often correlated with students’ mathematics scores,

which can cause a problem with validity (Tracey, 2012). For that reason, it is important to

analyze the results of the interest survey to determine whether any of these problems can

compromise the research questions.

Another limitation of this study is the fact that students are taught by several different

teachers. Although the teachers whose students are participating in the study collaborate

throughout the year and use common lessons and assessments, teaching styles and effectiveness

may differ, which may result in a significant difference in achievement, motivation, or both.

Future research should control for this difference and use teachers who may have similar

teaching styles.

Third, the present study is specific in its content and demographics and, therefore, cannot

be generalized across all subjects, grade levels, or skill levels. In addition to a lack of

generalizability to other content areas, this study may only be applicable to only on-level

Algebra II students. As mentioned, Algebra II is more complex than Algebra I and, thus, requires

more skills to achieve mastery. Similarly, achievement in Geometry requires a different

mathematical skill set. Furthermore, this study was conducted with participants in only high

school, and the results cannot be generalized to elementary or middle school students. Further

research is needed to determine whether the results of this study will be similar across these

different demographics and subjects.

Next, the participant selection of this study is that of a convenient sample. The students

of this study attend the school at which the researcher teaches. Some of the participants are even

the researcher’s students. This can create bias and a potential of interest in the research because

of a personal connection to the researcher. Additionally, some of the responses to the self-report

survey may be a result of socially desirable responding; that is, students’ answers may be what

they think the teacher and researcher want to hear rather than an accurate representation of their

motivational beliefs. To prevent this, the study should be replicated using students outside of the

researcher’s school.

Finally, another limitation of this study is the researcher bias. As discussed previously,

my experience as a math teacher provided a foundation for this study, but also may have

impacted the interview and interpretation of responses. Because I have beliefs about the test

myself, it is possible that some of my personal thoughts influenced the results. Of course, I have

been making an effort to minimize this bias, but it should be noted that it cannot be fully

eliminated from the study.

Further research could add to the findings of this study in several ways. First, including

additional motivational constructs would give an added element to the study. For example,

measuring students’ task value, self-efficacy, and outcome expectations could provide a wealth

of additional information about students’ motivation as it relates to achievement on computer-

based assessments. Second, future research can also replicate this study in other subjects or

demographics. For instance, it would be worthwhile to determine whether the same results would

occur with geometry students or in a different school setting (e.g. rural or urban). Third, while

interviewing students provides valuable information regarding the assessments, interviewing the

teachers as well would allow the researcher to triangulate data and find common responses

among all those involved with the tests. Interviewing teachers would provide a different set of

factors regarding the exams, such as pressures put on by administration and autonomy within the

classroom.

The present study contains many practical implications for educational psychology and

secondary mathematics education. Overall, this study may bring to light certain aspects of

computer-based testing that have not previously been analyzed. With more counties and states

changing their standardized testing systems from paper-and-pencil to technology-enhanced, it is

important to investigate whether or not these assessments are testing students accurately and

fairly. The present study will not only determine the differences between item format and

question type, but it will also determine how personal interest, task value, and perceived

difficulty will impact achievement. Furthermore, the data of this study may also provide valuable

information to teachers on how to best prepare their students for these tests. It is the hope of the

researcher that testing companies will take into consideration the findings of this study and make

the necessary adjustments to the tests so that all students have a fair opportunity to succeed.

References

Anderman, E.M. & Wolters, C.A. (2006). Goals, values, and affects: Influences on student

motivation. In P.A. Alexander & P.H. Winne (Eds.), Handbook of educational

psychology (2nd ed., pp. 369-389). Mahwah, NJ: Erlbaum.

Atkinson, J. W. (1957). Motivational determinants of risk-taking behaviors. Psychological

Review, 64, 359-372.

Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory.

Englewood Cliffs, NJ: Prentice Hall.

Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman.

Bandura, A. & Schunk, D. H. (1981). Cultivating competence, self-efficacy, and intrinsic interest

through proximal self-motivation. Journal of Personality and Social Psychology, 41,

586-598. doi: 10.1037/0022-3514.41.3.586

Belenky, D. M., & Nokes-Malach, T. J. (2013). Mastery-approach goals and knowledge transfer:

An investigation into the effects of task structure and framing instruction. Learning and

Individual Difference, 25, 21-34. doi: 10.1016/j.lindif.2013.02.004

Blanton, M.L. & Kaput, J.J. (2005). Characterizing a classroom practice that promotes algebraic

reasoning. Journal for Research in Mathematics Education, 36(5), 412-446. Retrieved

from http://www.jstor.org/stable/30034944

Bong, M. (2004). Between- and within-domain relations of academic motivation among middle

and high school students: Self-efficacy, task-value, and achievement goals. Journal of

Educational Psychology, 93(1), 23-34. doi: 10.I037//0022-0663.93.1.23

Day, S. B. & Goldstone, R. L. (2012). The import of knowledge export: Connecting findings and

theories of transfer of learning. Educational Psychologist, 47(3), 153-176. doi:

10.1080/00461520.2012.696438

Dewey, J. (1913). Interest and effort in education. Cambridge, MA: Riverside Press.

Doerr, H.M. & Zangor, R. (2000). Creating meaning for and with the graphing calculator.

Educational Studies in mathematics, 41, 143-163.

Durik, A. M., Vida, M., & Eccles, J. S. (2006). Task values and ability beliefs as predictors of

high school literacy choices: A developmental analysis. Journal of Educational

Psychology, 98, 382-393. doi: 10.1037/0022-0663.98.2.382

Eccles, J. S. & Wigfield, A. (1992). The development of achievement task values: A theoretical

analysis. Developmental Review, 12, 265-310. doi: 10.1016/0273-2297(92)90011-P

Eccles, J. S. & Wigfield, A. (1995). In the mind of the actor: The structure of adolescents'

achievement task values and expectancy-related beliefs. Personality and Social

Psychology Bulletin, 21(3), 215-225. doi: 10.1177/0146167295213003

Fuchs, L. S., Fuchs, D., Stuebing, K., Fletcher, J. M., Hamlett, C. L., & Lambert, W. (2008).

Problem solving and computational skill: Are they shared or distinct aspects of

mathematical cognition? Journal of Educational Psychology, 100(1), 30-47. doi:

10.1037/0022-0663.100.1.30

Fulmer, S. M. & Tulis, M. (2013). Changes in interest and affect during a difficult reading task:

Relationships with perceived difficulty and reading fluency. Learning and Instruction,

27, 11-20. doi: 10.1016/j.learninstruc.2013.02.001

Gaspard, H., Dicke, A.-L., Flunger, B., Schreier, B., Häfner, I., Trautwein, U., & Nagengast, B.

(2014, October 27). More value through greater differentiation: Gender differences in

value beliefs about math. Journal of Educational Psychology. Advance online

publication. http://dx.doi.org/10.1037/edu0000003

Graham, S. & Weiner, B. (2012). Motivation: Past, present, and future. In Harris, K. R., Graham,

S., Urdan, T., McCormick, C. B., Sinatra, G. M., & Sweller, J. (Eds.), APA educational

psychology handbook, Vol 1: Theories, constructs, and critical issues. Washington, DC:

American Psychological Association.

Jodoin, M. G. (2003). Measurement efficiency of innovative item formats in computer-based

testing. Journal of Educational Measurement, 40(1), 1-15. doi: 10.1111/j.1745-

3984.2003.tb01093.

Kaput, J.J. (1989). Linking representations in the symbol systems of algebra. In S. Wagner & C.

Kieran (Eds.), Research issues in the learning and teaching of algebra (Vol. 4 of

Research agenda for mathematics education, pp. 167-194). Reston, VA: National

Council of Teachers of Mathematics.

Kaput, J. J. (1998). Transforming algebra from an engine of inequity to an engine of

mathematical power by “algebrafying” the K-12 curriculum. In the National Council of

Teachers of Mathematics & the Mathematical Sciences Education Board (Eds.), The

nature and role of algebra in the K-14 curriculum: Proceedings of a national symposium

(pp. 25-26). Washington, DC: National Research Council, National Academy Press.

Karabenick, S. A. & Youssef, Z. I. (1968). Performance as a function of achievement motive

level and perceived difficulty. Journal of Personality and Social Psychology, 10(4), 414-

419. doi: 10.1037/h0026735

Katz, I. R., Bennett, R. E., & Berger, A. E. (2000). Effects of response format on difficulty of

SAT-mathematics items: It’s not the strategy. Journal of Educational Measurement,

37(1), 39-57. doi: 10.1111/j.1745-3984.2000.tb01075.x

Kieran, C. (1996). The changing face of school algebra. In C. Alsina, J. Alvared, B. Hodgson, C.

Laborde, & A. Perez (Eds.), Eighth International Congress on Mathematical Education:

Selected lectures (pp. 271-290). Seville, Spain: S.A.E.M. Thales.

Kieran, C. (2007). Learning and teaching algebra at the middle school through college levels:

Building meaning for symbols and their manipulation. In Lester, F.K. (Ed.), Second

Handbook of Research on Mathematics Teaching and Learning (pp. 707-762). Charlotte,

NC: Information Age Publishing.

Kramarski, B., Weiss, I., & Sharon, S. (2013). Generic versus context-specific prompts for

supporting self-regulation in mathematical problem solving among students with low or

high prior knowledge. Journal of Cognitive Education and Psychology, 12(2), 197-214.

doi: 10.1891/1945-8959.12.2.197

Lee, L. (1997). Algebraic understanding: The search for a model in the mathematics education

community. Unpublished doctoral dissertation, Universite du Quebec a Montreal.

Lent, R.W., Brown, S.D., & Hackett, G. (1994). Towards a unifying social cognitive theory of

career and academic interest, choice, and performance. Journal of Vocational Behavior,

45, 79-122. doi: 10.1006/jvbe.1994.1027

Lopez, F.G., Lent, R.W., Brown, S.D., & Gore, P.A. (1997). Role of social-cognitive

expectations in high school students’ mathematical-related interest and performance.

Journal of Counseling Psychology, 44(1), 44-52. doi: 10.1037/0022-0167.44.1.44

Marsh, H. W., Trautwein, U., Ludtke, O., Koller, O., & Baumert, J. (2005). Academic self-

concept, interest, grades, and standardized test scores: Reciprocal effects models causal

ordering. Child Development, 76(2), 397-416. doi: 10.1111/j.1467-

8624.2005.00853.x

Martinez, M. E. & Bennett, R. E. (1992). A review of automatically scorable constructed-

response item types for large-scale assessment. Applied Measure in Education, 5(2), 151-

169. doi: 10.1207/s15324818ame0502_5

Mazzarella, M. (2015). Algebra II Review. Unpublished mathematics assessment, George Mason

University, Fairfax, VA.

Midgley, C., Feldlaufer, H., & Eccles, J. S. (1989). Change in teacher efficacy and student self-

and task-related beliefs in mathematics during the transition to junior high school.

Journal of Educational Psychology, 81(2), 247-258. doi: 10.1037/0022-0663.81.2.247

Mitchell, M. (1993). Situational interest: Its multifaceted structure in the secondary school

mathematics classroom. Journal of Educational Psychology, 85(3), 424-436. doi:

10.1037/0022-0663.85.3.424

O’Callaghan, B.R. (1998). Computer-intensive algebra and students’ conceptual knowledge of

function. Journal of Research in Mathematics Education, 29, 21-40.

Ozuru, Y., Best, R., Bell, C., Witherspoon, A. & McNamara, D. S. (2010). Influence of question

format and text availability on the assessment of expository text comprehension.

Cognition and Instruction, 25(4), 399-438. doi: 10.1080/07370000701632371

Ozyurek, R. (2005). Informative sources of math-related self-efficacy expectations and their

relationship with math-related self-efficacy, interest, and preference. International

Journal of Psychology, 40(3), 145-156. doi: 10.1080/00207590444000249

Pajares, F. & Miller, M.D. (1994). Role of self-efficacy and self-concept beliefs in mathematics

problem solving: A path analysis. Journal of Educational Psychology, 86(2), 193-203.

doi: 10.1037/0022-0663.86.2.193

Parsons, J. E., Adler, T., & Meece, J. L. (1984). Sex differences in achievement: A test of

alternate theories. Journal of Personality and Social Psychology, 46(1), 26-43. doi:

10.1037/0022-3514.46.1.26

Pomplun, M. & Omar, M. H. (1997). Multiple-mark items: An alternative objective item format?

Educational and Psychological Measurement, 57(6), 949-962. doi:

10.1177/0013164497057006005

Radford, L. (2004) Syntax and meaning In M.J. Hoines & A.B. Fuglestad (Eds.), Proceedings of

the 28th Conference of the International Group for the Psychology of Mathematics

Education (Vol. 1, pp. 161-166). Bergen, Norway.

Renninger, K. A., Ewen, L., & Lasher, A. K. (2002). Individual interest as context in expository

text and mathematical word problems. Learning and Instruction, 12, 467-491. doi:

10.1016/S0959-4752(01)00012-3

Scherer, R & Siddiq, F. (2015). The Big-Fish-Little-Pond-Effect revisited: Do different types of

assessments matter? Computers & Education, 80, 198-210. doi:

10.1016/j.compedu.2014.09.003

Schunk, D. H. (1995). Self-efficacy and education and instruction. In J. E. Maddux (Ed.), Self-

efficacy, adaptation, and adjustment: Theory, research, and application (pp. 281-303).

New York, NY: Plenum Press.

Schunk, D. S. (2012). Social cognitive theory. In Harris, K. R., Graham, S., Urdan, T.,

McCormick, C. B., Sinatra, G. M., & Sweller, J. (Eds.), APA educational psychology

handbook, Vol 1: Theories, constructs, and critical issues. Washington, DC: American

Psychological Association.

Schunk, D.H. & Zimmerman, B.J. (2006). Competence and control beliefs: Distinguishing the

means and ends. In P.A. Alexander & P.H. Winne (Eds.), Handbook of educational

psychology (2nd ed., pp. 349-367). Mahwah, NJ: Erlbaum.

Senko, C. & Harackiewicz, J. M. (2005). Achievement goals, task performance, and interest:

Why perceived goal difficulty matters. Personality and Social Psychology Bulletin,

31(12), 1739-1753. doi: 10.1177/0146167205281128

Sfard, A. & Linchevski, L. (1994). The gains and the pitfalls of reification- The case of algebra.

Educational Studies in Mathematics, 26, 191-228.

Shieh, Y. & Fouladi, R. T. (2003). The effect of multicollinearity on multilevel modeling

parameter estimates and standard errors. Educational and Psychological Measurement,

63(6), 951-985. doi: 10.1177/0013164403258402

Tracey, T. J. G. (2012). Problems with single interest scales: Implications of the general factor.

Journal of Vocational Behavior, 81, 378-384. doi: 10.1016/j.jvb.2012.10.001

Trautwein, U., Ludtke, O., Marsh, H. W., Koller, O., & Baumert, J. (2006). Tracking, grading,

and student motivation: Using group composition and status to predict self-

concept and interest in ninth-grade mathematics. Journal of Educational Psychology, 98(4),

788-806. doi: 10.1037/0022-0663.98.4.788

Tyrrell, R., Holland, K., Dennis, D., & Wilkins, A. J. (1995). Coloured overlays, visual

discomfort, visual search and classroom reading. Journal of Research in Reading, 181,

10-23.

Virginia Department of Education (2012). Standard of learning (SOL) and testing. Retrieved

from Virginia Department of Education website: http://www.doe.virginia.gov.

Wheeler, L. & Suls, J. (2005). Social comparisons and self-evaluations of competence. In A.J.

Elliot & C.S. Dweck (Eds.), Handbook of competence and motivation (pp. 566-578).

New York, NY: Guilford Press.

Wigfield, A. & Eccles, J.S. (2000). Expectancy-value theory of motivation. Contemporary

Educational Psychology, 25, 68-81. doi:10.1006/ceps.1999.1015

Wigfield, A. & Eccles, J.S. (2002). The development of competence beliefs, expectancies for

success, and achievement values from childhood through adolescence. In A. Wigfield &

J.S. Eccles (Eds.), Development of achievement motivation (pp. 91-120). San Diego, CA:

Academic Press. doi:10.1016/B978-012750053-9/50006-1

Wigfield, A., Tonks, S., & Eccles, J. S. (2004). Expectancy value theory in cross-cultural

perspective. In D. M. McInerney & S. Van Etten (Eds.), Big theories revisited (pp. 165-

198). Greenwich, CT: Information Age.

Zimmerman, B. J. (1986). Development of self-regulated learning: Which are the key

subprocesses? Contemporary Educational Psychology, 11, 307-313. doi: 10.1016/0361-

476X(86)90027-5

Appendix A: Demographics and Motivational Constructs Measure

Student ID _______________________________________ DO NOT PUT YOUR NAME ON THIS SURVEY

1. Circle your gender: Male Female

2. Circle your grade: 9 10 11 12

3. Write your age: ___________________

For items 4 – 13, circle the appropriate response based on the statement.

4. Mathematics is enjoyable to me.

Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree

5. The amount of effort it takes to do well in high school mathematics courses is worthwhile to me.

Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree

6. In general, mathematics is hard for me.

Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree

7. I have always enjoyed studying mathematics in school.

Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree

8. I feel that, to me, being good at solving problems which involve mathematics is important.

Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree

9. Compared to most other students, mathematics is hard for me.

Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree

10. Compared to other subjects, I feel relaxed studying math.

Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree

11. It is important for me to get good grades in math.

Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree

12. Compared to other subjects, mathematics is hard for me.

Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree

13. Compared to other subjects, mathematics is exciting to me.

Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree

Appendix B: Mathematics Measures

Test A

Question 1 :What is the solution set for this equation?

A:

B:

C:

D:

Question 2 :Type your answer into the box. You must give your answer in integer form.

The following sequence is given in recursive form.

What is the value of the fourth term of this sequence?

Question 3 :Which of the following situations involves a permutation?

A: Determining how many different groups of 3 employees can be chosen from 9 employees.

B: Determining how many different ways 7 runners can be assigned lanes on a track for a race.

C: Determining how many different ways to choose 10 students to attend a field trip from a group of 25 students.

D: Determining how many different ways 4 cashiers can be chosen to work from a group of 7 cashiers.

Question 4 :Click on a box to choose each y-coordinate you want to select. You must select all correct answers.

What are the y-coordinates for the solution to this system of equations?

A: y = -9

B: y = -3

C: y = -2

D: y = 1

E: y = 2

F: y = 6

G: y = 8

H: y = 9

Question 5 :The number of combinations of 7 objects taken 2 at a time is

A: 3B: 7

C: 21D: 42

Question 6 :Type your answer into the box. You must enter your answer in integer form.

The shoe sizes of a large population are normally distributed with a mean of 8.9 inches and a standard deviation of 0.705 inches. What percentage of the population has a shoe size greater than 9.8 inches? ROUND TO THE NEAREST INTEGER.

Question 7 :

Factor: 

A: (2x+3) (3x-7)

B: (2x-3) (3x+7)

C: (3x+2) (2x-7)

D: (3x-2) (2x+7)

Question 8 :Select a box for each correct part of the expression. You must select each correct expression.

Select each part of the simplified expression   .

A:

B:

C:

D:

E:

F:

Question 9 :The area of a triangle varies jointly with the product of the base and the height. A triangle has a base of 12 feet, a height of 3 feet, and an area of 18 square feet. What is the base of a triangle with a height of 4 feet and an area of 36 square feet?

A: 0.5 feet

B: 9 feet

C: 12 feet

D: 18 feet

Question 10 :Type the answer into the box.The number of permutations of 9 objects taken 3 times is

Question 11 :

Which is a solution of   ?

A: x = -5

B:x

= C: x = -

1

D: x = 1

Question 12 :Click on the box to select the value. You must select each correct value.

Two baseballs were thrown on a field as the same time. One ball follows the

path of the function  , and the other ball follows the path of

the function  , where x is the time in seconds, and f(x) and g(x) are the heights in feet. At what two times, in seconds, are the two balls the same height?

A: 0.7 seconds

B: 1.0 seconds

C: 4.0 seconds

D: 5.14 seconds

E: 5.45 seconds

F: 7.13 seconds

Question 13 :What is the sum of this infinite series?

72 - 36 + 18 - 9 + ...

A: -144

B: -48C: 48

D: 144

Question 14 :Type your answer into the box. You must enter your answer in integer form.

Let   and  , what is  ?

Question 15 :A new rollercoaster at an amusement park follows the path of the

function  , where x is the time, in seconds, after the rollercoaster begins, and f(x) is the height of the rollercoaster, in yards. Between which two times, in seconds, is the rollercoaster increasing in height?

A:

B:

C:

D:

Question 16 :Select each expression that is equivalent. You must select all correct expressions.

Identify each expression that is equivalent to 1.

A:

B:

C:

D:

E:

F:

G:

H:

Question 17 :

Which of the following describes the end behavior of   as x approaches infinity?

A: y approaches negative infinity

B: y approaches -2

C: y approaches 3

D: y approaches infinity

Question 18 :Type your answer into the box. You must enter your answer in integer form.

The heights of Galapagos penguins are normally distributed with a mean of 49 cm and a standard deviation of 1.82 cm. If a scientist measures the heights of 300 penguins, how many penguins are expected to be between 48.4 cm and 50.1 cm tall? ROUND YOUR ANSWER TO THE NEAREST INTEGER.

Question 19 :Which is the equation of an asymptote of the graph of the following equation?

A: x = -3

B: y = 3

C: x = 6

D: y = 6

Question 20 :Click on the box to select an interval. You must select each correct interval.

Indicate each intervals where the graph   is only increasing.

A:

B:

C:

D:

E:

F:

G:

H:

Question 21 :A mathematics class consists of 10 girls and 8 boys. The teacher

wants to choose 2 girls and 2 boys to go on a trip. How many different groups could the teacher choose?

A: 73

B: 146

C: 1260

D: 5040

Question 22 :Type your answer into the box. You must enter your answer in integer form.

If y varies directly with the square root of x, what is the constant of variation if y = 36 when x = 9?

Question 23 :

What are the zeros of the function   ?

A: x = -16 and x = 0

B: x = -16, x = -8, and x = 2

C: x = -8 and x = 2

D: x = -2 and x = 8

Question 24 :Click on the box to select the correct equation. You must select each correct equation.

A baseball was thrown by a player, and hit the ground after exactly 5 seconds. If x represents the time in seconds and y represents the height of the ball, which of the following functions could represent the path of the

ball?

A:

B:

C:

D:

E:

F:

Question 25 :A normally distributed set of numbers has a mean of 75 and a standard deviation of 7.97. What percentage of values lies between 70 and 85?

A: 11%B: 37%C: 63%D: 89%

Question 26 :Click on the correct box to select each value. You must select each correct value.

The domain of the function   is all real numbers except -

A: -7B: -4C: -3D: 0E: 3

F: 4G: 7

Question 27 :In a school, 12 students are running for 4 class officers- a president, a vice president, a secretary, and a treasurer. If each position is to be held by one person and no person can hold more than one position, in how many ways can those positions be filled?

A: 48

B: 495

C: 11880

D: 20736

Question 28 :Click on each box to select each function. You must select each correct function.

Which of the following functions are in the same family as the

function   ?

A:

B:

C:

D:

E:

F:

Question 29 :The steps to simplify an expression are shown below:

Step 1: 4(x+3) - 3x + 1Step 2: 4x + 12 - 3x + 1Step 3: 4x - 3x + 12 + 1Step 4: x + 13

Which of the following properties justifies getting from Step 2 to Step 3?

A: Associative Property

B: Commutative Property

C: Distributive Property

D: Transitive Property

Question 30 :Type your answer into the box. ROUND YOUR ANSWER TO THE NEAREST TENTH.

A baseball player throws a ball from one end of the field to the other. A fan measures the path of the ball and determines that it follows the

function  , where x is the time in seconds and f(x) is the height in feet. What is the maximum height of the ball, in feet? ROUND YOUR ANSWER TO THE NEAREST TENTH.

Test B

Question 1 :Type your answer into the box. Enter your answer as a whole number.

What is the solution set for this equation?

Question 2 :The following sequence is given in recursive form.

What is the value of the fourth term of this sequence?

A: 29B: 33C: 61

D: 125

Question 3 :Click on the box to choose the situation. You must select all correct situations.

Which of the following situations involve a permutation?

A: Determining how many different ways to choose 3 employees from a group of 9 employees.

B: Determining how many different ways 7 runners can be assigned lanes on a track for a race.

C: Determining how many different seating charts can be made placing 6 people around a table.

D: Determining how many 5-letter passwords can be made using the

word "graph."

E: Determining how many different groups of 10 students can be chosen to go on a field trip from a group of 25 students.

F: Determining how many different ways 4 cashiers can be chosen to work from a group of 6 cashiers.

Question 4 :What are the y-coordinates for the solutions to this system of equations?

A: y = 1 and y = 9

B: y = -3 and y = -9

C: y = -2 and y = 6

D: y = 2 and y = 8

Question 5 :Type the answer into the box.

The number of combinations of 7 objects taken 2 at a time is

Question 6 :The shoe sizes of a large population are normally distributed with a mean of 8.9 inches and a standard deviation of 0.705 inches. What percentage of the population has a shoe size greater than 9.8 inches? Round to the nearest integer.

A: 5%B: 10%C: 20%

D: 34%

Question 7 :Click on a box for each factor you want to select. You must select all correct factors.

Select all of the factors of: 

A: (2x+3)

B: (2x-3)

C: (3x+2)

D: (3x-2)

E: (2x+7)

F: (2x-7)

G: (3x+7)

H: (3x-7)

Question 8 :

Which of the following expressions is equivalent to   ?

A:

B:

C:

D:

Question 9 :Type your answer into the box. You must enter your answer in integer form.

The area of a triangle varies jointly with the product of the base and the height. A triangle has a base of 12 feet, a height of 3 feet, and an area of 18 square feet. What is the base of a triangle with a height, in feet, of 4 feet and an area of 36 square feet?

Question 10 :The number of permutations of 9 objects taken 3 times is

A: 27B: 84

C: 504

D: 729

Question 11 :Click on each solution to the equation. You must select each correct solution.

Select all the solutions of   .

A: x = -5

B:x

= 

C: x = -1

D: x = 1

E:x

= 

F: x = 5

Question 12 :Two baseballs were thrown on a field as the same time. One ball

follows the path of the function  , and the other ball

follows the path of the function  , where x is the time in seconds, and f(x) and g(x) are the heights in feet. At what two times, in seconds, are the two balls the same height?

A: 0.70 seconds and 5.45 seconds

B: 7.13 seconds and 5.14 seconds

C: 0.70 seconds and 7.13 seconds

D: 5.14 seconds and 5.45 seconds

Question 13 :Type your answer into the box.

What is the sum of this infinite series?

72 - 36 + 18 - 9 + ...

Question 14 :

Let   and  . What is   ?

A: -29B: 35C: 75

D: 152

Question 15 :Click on the box to select the interval. You must select each correct interval.

A new rollercoaster at an amusement park follows the path of the

function  , where x is the time, in seconds, after the rollercoaster begins, and f(x) is the height of the rollercoaster, in yards. Between which two times, in seconds, is the rollercoaster increasing in height?

A:

B:

C:

D:

E:

F:

Question 16 :Which expression is equivalent to 1?

A:

B:

C:

D:

Question 17 :Type your answer into the box. You must give your answer in integer form.

What value does y approach in the function   as x approaches infinity?

Question 18 :The heights of Galapagos penguins are normally distributed with a mean of 49 cm and a standard deviation of 1.82 cm. If a scientist measures the heights of 300 penguins, how many penguins are expected to be between 48.4 cm and 50.1 cm tall? Round your answer to the nearest integer.

A: 84

B: 107

C: 168

D: 204

Question 19 :Click on each box to choose each asymptote. You must select all correct asymptotes.

Which are the equations of the asymptotes of the graph of the following function?

A: x = -3

B: y = -3

C: x = 3

D: y = 3

E: x = 6

F: y = 6

Question 20 :

Throughout which of the following intervals is   only increasing?

A:

B:

C:

D:

Question 21 :Type your answer into the box. You must enter your answer in integer form.

A mathematics class consists of 10 girls and 8 boys. The teacher wants to choose 2 girls and 2 boys to go on a trip. How many different groups could the teacher choose?

Question 22 :If y varies directly as the square root of x, what is the constant of variation if y = 36 and x = 9?

A: 1.5

B: 2C: 4D: 12

Question 23 :Click on the box to select the zeros. You must select each correct zero.

Which of the following are zeros of the function   ?

A: x = -16

B: x = -8

C: x = -2

D: x = 0

E: x = 2

F: x = 8

Question 24 :A baseball was thrown by a player, and hit the ground after exactly 5 seconds. If x represents the time in seconds and y represents the height of the ball, which of the following functions could represent the path of the ball?

A:

B:

C:

D:

Question 25 :Type your answer into the box. You must enter your answer in

integer form.

A normally distributed set of numbers has a mean of 75 and a standard deviation of 7.97. What percentage of values lies between 70 and 85? ROUND TO THE NEAREST INTEGER.

Question 26 :

The domain of the function   is all real numbers except -

A: -7, -4, 4

B: -7, 4

C: -4, 7

D: 4

Question 27 :Type your answer into the box. You must enter your answer in integer form.

In a school, 12 students are running for 4 class officers- a president, a vice president, a secretary, and a treasurer. If each position is to be held by one person and no person can hold more than one position, in how many ways can those positions be filled?

Question 28 :Which of the following is in the same family as the

function   ?

A:

B:

C:

D:

Question 29 :Click on the box to select a property. You must select each correct property.

The steps to simplify an expression are shown below:

Step 1: 4(x+3) - 3x + 1Step 2: 4x + 12 - 3x + 1Step 3: 4x - 3x + 12 + 1Step 4: x + 13

Which of the following properties justify Step 2, Step 3, and Step 4?

A: Associative Property

B: Commutative Property

C: Distributive Property

D: Inverse Property

E: Substitution Property

F: Transitive Property

Question 30 :A baseball player throws a ball from one end of the field to the other. A fan measures the path of the ball and determines that it

follows the function  , where x is the time in seconds and f(x) is the height in feet. What is the maximum height of the ball, in feet?

A: 2 feet

B: 5.4 feet

C: 6 feet

D: 9.2 feet

Appendix C: Interview Questions

I. Algebra 2 Coursea. Do you find the Algebra 2 course difficult?

i. If so, what aspects?1. Content, rigor, pace, teacher, etc.?

ii. If not, why not?1. Teacher, content, etc.?

b. How important is taking Algebra 2 for you?i. For your academic career?

ii. For your future goals?iii. Why?

1. Content?a. How relevant is the content?

2. Critical thinking?c. What value, in general, do you see in the Algebra 2 course?

i. Preparation1. For what?

ii. Mathematical skills1. Which?

iii. Critical thinking skillsII. Algebra 2 SOL

a. Did you find the Algebra 2 SOL difficult?i. If so, what aspects?

1. Content, length, item format, question types, etc.?ii. If not, why not?

1. Preparation, similar to classroom tests, content, etc.?b. How important is it for you to do well on this test?

i. If so, why?1. College goals, content, career goals, etc.?

ii. If not, why not?1. Irrelevance, content, etc.?

c. If you were to improve the test, how would you do it?i. Change content, length, difficulty, types of questions, etc.?

d. What value, in general, do you see in passing the Algebra 2 SOL?i. Test of mathematical skills?

ii. Critical thinking skills?iii. Comparison to other students?iv. College?