G. Sherrie Lewis, DBA, Virginia State University TCC Online Conference April 16, 2009

No Significant Difference: An Update on Research Comparing

Performance of Online & Traditional Management

StudentsG. Sherrie Lewis, DBA, Virginia State University

TCC Online ConferenceApril 16, 2009

The Study: At A Glance Introduction Purpose Significance Scope Methods Research Questions Selection Criteria Coding Process Results Recommendations Limitations Implications Opportunities

Q & A

TCC Online Conference, April 16, 2009

IntroductionThe largest high-school class in United

States’ history is scheduled to graduate in 2009 (Parker, 2003). As a result, an increasing number of students seek traditional and online management courses.

This study includes an examination of student outcomes used to describe management student performance in online and traditional classrooms.


Purpose

The purpose of this study is to provide a discipline-specific update on comparisons of online and traditional management students’ classroom performance.


SignificanceDue to differences in institutional demographics,

comparison studies often exist in conditions that are difficult to replicate. These variables prevent generalizations that can be transferred from schools or classrooms. Examples include:

Length of Term Retention Subject Student Demographics Instructor Experience Attendance Policies Researcher Disclosure


Significance (continued)As a result of this study, educators and

administrators will be able to :Consider use of distance education in the

management disciplinesIdentify students for whom distance education

is appropriateExamine the role that previous academic

achievement plays in future online student success

Discuss the appropriateness of distance education in various business disciplines.


Scope Databases were searched by using a set of

key words as delimiters. Other key words were used as a means of locating additional literature. The search resulted in examination of 50 titles. Of theses studies, 20 were usable. These were selected for comprehensive analysis.


Scope (continued)This meta-analysis included 20 studies

published between 1995 and 2005. Results consists of:

8 Final Exam Grades 5 Final Course Grades 7 Grade Point Averages


Methods Effect sizes, or the differences in educational

outcomes for traditional and online students, serve as dependent variables. The effect size or “standard difference” or “d” is the ratio of the difference between the mean to the standard deviation (Cooper, 1994; Yu, 2001).

.20 or less = Small Effect Size.50 to .79 = Medium Effect Size.80 or larger = Large Effect Size


Research Questions: Q1 Is there a difference in student performance

characteristics (GPAs) of students who choose online courses versus those in enrolled in traditional undergraduate management classes? It was predicted that there will be no significant difference in prior levels of achievement in students who choose online courses versus those in enrolled in traditional undergraduate management classes.


Research Questions: Q2Is there a difference in student

achievement outcomes (final exam grades) in online delivery of undergraduate business management courses versus business management courses in the traditional classroom? It was predicted that there will be no significant difference in student achievement outcomes in online delivery of undergraduate business management courses versus traditional undergraduate business management courses.


Research Questions: Q3Is there a difference in student achievement

outcomes (final course grades) in online delivery of undergraduate business management courses versus business management courses in the traditional classroom? It was predicted that there will be no significant difference in student achievement outcomes in online delivery of undergraduate business management courses versus traditional undergraduate business management courses.


Selection Criteria• Be publicly available or archived.• Include at least one achievement, satisfaction or

retention measure.• Be published no earlier than 1995 and no later

than December 2005.• Include outcome measures that were the same or

comparable. • The studies included in this analysis were limited

to management disciplines.• Each study examined distance education’s effect

on student performance.TCC Online Conference, April 16, 2009

Coding Process Studies that make up the effect sized used in this analysis

were coded based on an adaptation of categories from Williams’ (2004) comparison study on distance education in the allied health fields.

Publication Information Discipline Subject Characteristics Treatment Characteristics Research Design Characteristics Bias Assessment Measures


Results: Q1• The weighted average effect size for comparisons

of management and online students’ GPAs was 0.26. There was a small amount of difference between students’ GPAs in distance education and traditional management courses.

• The first null hypothesis cannot be rejected


Results: Q2• The weighted mean effect size for

comparisons of management student final exam performance was - 0.12. There was a significant difference between final exam performance of online management students versus their online counterparts. Management students enrolled in distance education courses did not perform as well as their online counterparts.

Therefore, the null hypothesis was rejected.TCC Online Conference, April 16, 2009

Results: Q3• The weighted mean effect size for final

course grades was 0.16. Therefore the first hypothesis cannot be rejected.

• There was no significant difference between final grades earned by students in distance education and traditional management courses.


Recommendations• An examination of the relationship between

student demographics and their effect on distance education outcomes.

• Investigation of the instructional methods used in distance and traditional and their affects on student satisfaction and outcomes.

• Use of other measures of student performance (i.e. performance in subsequent coursework).

• Use of shorter terms in comparison studies so that the number of variables in these learning environments are reduced.


LimitationsParticipants were not randomly selected and

do not reflect the populationFewer than half of the 50 studies met the

criteria for additional reviewNo bias controls


Implications• Additional examination of the relationship between

student demographics and their effect on distance education outcomes is necessary.

• Investigation of the instructional methods used in distance and traditional classrooms and their affects on student satisfaction and outcomes are recommended.

• Use of other measures of student performance (i.e. performance in subsequent coursework) may result in additional clues.

• Use of shorter terms in comparison studies may result in a reduction of the variables affecting student outcomes.


Implications (continued)• Continued study of management education in the traditional

classroom is needed.

• Additional investigation of similarities and differences between student groups, attitudes, motivation and the impact of previous educational experience on distance education in management disciplines in necessary.

• This includes student workload and an examination of outcomes for academic programs rather than individual management courses in the traditional and online environments.

• Exploration of course workload options for distance education faculty and the effects of distance education activities on the tenure and promotion process is required.


ReferencesCooper, H., & Hedges, L. Eds. (1994). The Handbook of Research

Synthesis, Sage Publications: Thousand Oaks, California.

Keegan, D. Editor: (1999) Foundations of distance education. Third Ed.

Routledge: New York.

Parker, A. (2003). Motivation & Incentives for Distance Faculty. Online Journal of

Distance Learning Administration 2(4) Retrieved April 10, 2006 from

http://www.westga.edu/~distance/ojdla/fall33/rockwell24.html

Yu, C. (2001). Meta-analysis and effect size. Retrieved August 20, 2006

from: http://seamonkey.ed.asu.edu/~alex/teaching/WBI/es.html

Williams, S. (2004). A meta-analysis of the effectiveness of the

effectiveness of distance education in allied health science programs

(Doctoral dissertation, Ohio University, 2004).


http://www.westga.edu/~distance/ojdla/fall33/rockwell24.html

Questions & Answers


Date post:	16-Mar-2016
Category:	Documents
Upload:	elias
View:	34 times
Download:	0 times

G. Sherrie Lewis, DBA, Virginia State University TCC Online Conference April 16, 2009

Documents