The Effects of Normative and Self-Referenced Feedback on

Students’ Motivation, Punctuality, and Essay Lengths in Online

Learning Environments

Tae Seob ShinEPET, CEPSE

shintae@msu.edu

Overview

• Theoretical Background• Research Purpose• Research Hypotheses & Questions• Research Design• Sample• Procedures• Measures• Analyses• Results• Discussion• Limitations• Significance

Theoretical Background

• Research on Online Learning:– Potential benefits of Web-based courses– Characteristics of online learning– Student engagement in online learning environments– Effective teaching strategies

(Davies & Graff, 2005; Gilbert et al., 2007, Lebaron & Miller, 2007; Tallent-Runnels et al., 2006)

• Research on Online Learners’ Motivation– Focuses on intrinsic motivation (Sins et al., 2007; Hoskins & van

Hooff, 2005)

– Little research on why and how students are engaged why and how students are engaged in an academic activityin an academic activity in an online learning environment (Wang & Reeves, 2007)

Theoretical Background

• Achievement Goal Orientation Theory– A goal orientation (achievement goal)goal orientation (achievement goal): a specific goal

that individuals strive to attain in achievement contexts (Urdan & Schoenfelder, 2006; Ames, 1992; Kaplan & Maehr, 2006)

• A Mastery vs. Performance Goal– A mastery goalmastery goal: a concern with developing

competence and skills– A performance goalperformance goal: a concern with demonstrating

competence to others by appearing capable or outperforming others

(Urdan & Schoenfelder, 2006; Ames, 1992; Kaplan & Maehr, 2006)

Theoretical Background

• Goal Structures– Goal structuresGoal structures: messages in the learning

environment that make certain goals salient– Examples: task, evaluation, recognition, & authority– Affect students’ perception of the class -> students’

goal orientations(Ames, 1992, Urdan & Schoenfelder, 2006; Urdan, 2004)

• Manipulation Strategies – Examples: task instructions, reasons for engaging in

an activity, and type of feedback – Students’ perception about classroom structure can

be manipulated by the instructor(Ames, 1984; Butler, 1987; Jagacinksi & Nicholls, 1987; Elliott & Dweck, 1988)

Theoretical Background

• Feedback and Achievement Goals– Different types of feedback -> different motivational

orientations– Self-evaluation & self-improvement (Butler, 1987; Butler, 1995)

• Feedback in Online Learning Environments– Communication tools embedded in learning platforms– Research on effective online teaching: importance of

providing specific, objective, and individual feedback(Lebaron & Miller, 2005; Mory, 2004; Tallent-Runnels et al., 2006)

Theoretical Background

• Normative Feedback– Prompt students to compare their performance with

their peers– Provides interpersonal norms

• Did I do better than other students in the class?

– Performance goals: social comparison

• Self-referenced Feedback– Prompt students to look back on their own

performance– Provides internal norms

• Have I improved?

– Mastery goals: self-improvement, self-development(Ames, 1992; Brophy, 2004; Kaplan & Maehr, 2006; Urdan & Schoenfelder, 2006)

Research Purpose

• To examine the effects of normative and self-referenced feedback on:

1. Students’ motivation– mastery goals– performance goals– interest in the course

2. Students’ academic performance– punctuality of assignments– essay lengths

Research Hypotheses

• H1: Students who receive self-referenced graphical feedback should report higher gains in mastery goal mastery goal orientationsorientations than those who receive normative graphical feedback

• H2: Students who receive normative graphical feedback should report higher gains in performance goal performance goal orientationsorientations than those who receive self-referenced graphical feedback

• H3: Students who receive self-referenced graphical feedback should report higher gains in interest in the interest in the coursecourse than those who receive normative graphical feedback

Exploratory Questions

• Q1: Do students become more punctual in submitting their assignment after they receive feedback about punctuality?

• Q2: Do students write longer essays after they receive

feedback about essay lengths?

Research Design

• Cross-Over design

R1 O1 T1 O2 T2 O3R2 O1 T2 O2 T1 O3

R: Random Assignment (by Groups)O: Observations (Pre, Post, & Post Post-test)T1: Normative FeedbackT2: Self-Referenced Feedback (Petersen, 1985)

• Random Assignment– Group NS NS (R1): normative feedback (T1) -> self-

referenced feedback (T2)– Group SN SN (R2): self-referenced feedback (T2) ->

normative feedback (T1)

Sample

• Sample: graduate students enrolled in an online Master's program during the summer semester of 2008

• Total N = 29

Test Sample Size

Pre-Test N = 28

Post-Test N = 25

Post Post-Test N = 23

Procedures

• Pretest– First weekFirst week of the semester– Survey

• Mastery goals

• Performance goals

• Interest

– Sample Size• Group NS: 14

• Group SN: 14

Procedures

• Treatment 1– First half of the courseFirst half of the course– Graphical feedback on how punctually students

submitted their work each week– Feedback about punctuality was given twice in week

3 and week 4 via email

• Group NSNS: Normative Feedback • Group SNSN: Self-referenced feedback

Norm

ative Feedback

Self-R

ef Feedback

Procedures

• Posttest– End of the first halfEnd of the first half of the semester– Survey

• Mastery goals

• Performance goals

• Interest

– Sample Size• Group NS: 12

• Group SN: 13

Procedures

• Treatment 2– Second half of the courseSecond half of the course– Graphical feedback on the length of their assignments

each week

– Feedback about the essay lengths was given twice in week 7 and week 8 via email

• Group NSNS: Self-referenced feedback • Group SNSN: Normative feedback

Norm

ative Feedback

Self-R

ef Feedback

Procedures

• Post-Posttest– End of the semesterEnd of the semester– Survey

• Mastery goals

• Performance goals

• Interest

– Sample Size• Group NS: 10

• Group SN: 12

Measures• Achievement Goal Orientations

– Mastery Goals: 5 items• my goal in this class is to learn as much as I can about this topic

– Performance Goals: 5 items• it is important for me to do well compared to others in this class

– Interest in the Course: 5 items• I enjoy this course very much

(Barron & Harackiewicz, 2003; Harackiewicz et al., 2000)

• Academic Performance– Punctuality of Assignment: an hourly difference between

the deadline and the actual submission time

– Essay Lengths: a difference between the recommended essay length and the actual essay length

Analyses

• H1, H2, & H3: – 2 (group NS vs. group SN) x 3 (pretest vs. posttest

vs. post-posttest) repeated measures ANOVA

• Q1 & Q2:– Q1: 2 (group NS vs. group SN) x 3 (week 2 vs. week

3 vs. week 4) repeated measures ANOVA– Q2: 2 (group NS vs. group SN) x 3 (week 7 vs. week

8 vs. week 9) repeated measures ANOVA

Results (H1: Mastery Goals)

• Finding– No significant

interaction effect between group and time (F < 1)

• Interpretation– H1 rejected

Figure 1. Mean mastery goals score by test occasion and groupNS: N->SSN: S->N time 1: pretest; 2: posttest; 3: post-posttest

Results (H2: Performance Goals)

• Finding– No significant

interaction effect between group and time (F(2, 40) = 2.07, p = .14)

• Interpretation– H2 rejected– The direction of

changes in performance goals was consistent with the hypothesis

Figure 2. Mean performance goals score by test occasion and groupNS: N->SSN: S->N time 1: pretest; 2: posttest; 3: post-posttest

Results (H3: Interest)

• Finding– No significant

interaction effect between group and time (F < 1)

• Interpretation– H3 rejected– The direction of

changes in interest was consistent with the hypothesis

Figure 3. Mean interest score by test occasion and groupNS: N->SSN: S->Ntime 1: pretest; 2: posttest; 3: post-posttest

Results (Q1: Punctuality)

• Finding– No significant interaction

effect between group and time (F(2, 54) = 2.30, p = .11)

– No significant main effects of group (F < 1) and time (F < 1)

Figure 4. Mean punctuality score by feedback type and group.NS: normative feedbackSN: self-referenced feedbacktime 1: week2 (pretest); 2: week 3; 3: week 4

Results (Q2: Essay Lengths)

• Finding– No significant interaction

effect between group and time (F < 1)

– No significant main effects of group (F < 1) and time (F < 1)

Figure 5. Mean essay lengths score by feedback type and groupNS: normative feedbackSN: self-referenced feedbacktime 1: week7 (pretest); 2: week 8; 3: week 9

Discussion

• The findings did not support the hypotheses. In other words, students’ motivation and academic performance were not affected by the types of graphical feedback they received.

• However, the direction of changes in students’ performance goals and interest in the course after feedback was consistent with the hypotheses.

Limitations

• Weak treatment– No explicit statement– Interpretation of graphs

• A cross-over design – Issue of equal treatment of subjects– Treatment– Complex analysis of effects

• Small sample size – Issue of statistical analysis

Significance

• Theoretical perspective– Experimental nature– Application of traditional motivation theory in online

learning environments

• Instructional perspective – Use of a semi-automated feedback mechanism

Thank you !!!