1 Evaluation of Collaborative Adult · training curriculum and instructional design methods for...

1

An Evaluation of Collaborative Adult

Training Performance Utilizing Social

Media Tools

By

Richard Jaenke

A Research Report Submitted in

Partial Fulfillment of the Requirements for the

Education Specialist Degree in

Career and Technical Education

Approved for the Completion of 6 Credits

Dr. Katherine Lui, Research Committee

The Graduate School

University of Wisconsin-Stout

July, 2011

2

The Graduate School

University of Wisconsin-Stout

Menomonie, WI

Author: Jaenke, Richard M.

Title: An Evaluation of Collaborative Adult Training Performance Utilizing

Social Media Tools

Graduate Degree/ Major: Education Specialist in Career and Technical Education

Research Adviser: Carol Mooney, Ph.D.

Month/Year: May, 2011

Number of Pages: 54

Style Manual Used: American Psychological Association, 6th

edition

Abstract

Customer service employees are an important part of many business models in today’s

economy. The customer service employee is often the primary point of contact for many

businesses and can determine the success or failure of a company as it positions itself among

domestic or international competition. This paper includes an analysis of Utility ABC, as it

relates to training methodology and tools used in corporate adult learning for customer service.

The purpose of this research was to determine the effectiveness and impacts to learning

and performance in a training environment when social learning tools and collaborative

communication devices are utilized in the training process. An additional goal of this study was

to better understand the learning retention and application outcomes based on the use of specific

social media and collaborative communication tools within a blended classroom and web-based

training module designed for customer service employees at a large electric and gas utility.

3

This research included the incorporation of four specific collaborative and social media

tools in a standard customer service training delivery. The tools included were an interactive

Blog, Discussion Forum, Glossary and a Wiki. Customer service performance data was gathered

regularly to determine the effectiveness of the standard training within Utility ABC. This paper

will summarize the performance metrics and compare the end results of training participants to

determine if there was a performance advantage for the inclusion of social media and

collaborative tools in the customer service employee training design.

4

The Graduate School

University of Wisconsin Stout

Menomonie, WI

Acknowledgments

Completing this research has been a personal goal for several years. I would like to thank

all of the people that have supported me through this process including my family and the many

faculty members who have provided support and guidance. Special thanks to my research

committee and especially Dr. Carol Mooney for helping me through this process.

5

Table of Contents

Page

Abstract ............................................................................................................................................2

Acknowledgment .............................................................................................................................4

Table of Contents .............................................................................................................................5

List of Tables ...................................................................................................................................7

Chapter I: Introduction ....................................................................................................................8

Statement of the Problem ...................................................................................................12

Purpose of the Study ..........................................................................................................12

Research Objectives ...........................................................................................................13

Definition of Terms............................................................................................................15

Assumption of the Study ....................................................................................................16

Limitations of the Study.....................................................................................................16

Methodology ......................................................................................................................17

Chapter II: Literature Review ........................................................................................................18

Introduction ........................................................................................................................18

Online Corporate Training .................................................................................................18

Constructivist Learning Theory .........................................................................................20

Adult Learning Concepts ...................................................................................................22

Post Training Blended Design and Assessment.................................................................23

Summary ............................................................................................................................24

Chapter III: Methodology ..............................................................................................................26

Introduction ........................................................................................................................26

6

Research Design.................................................................................................................26

Research Participants .........................................................................................................27

Instrumentation ..................................................................................................................28

Data Collection Procedures ................................................................................................30

Data Analysis .....................................................................................................................30

Limitations .........................................................................................................................31

Chapter IV: Results ........................................................................................................................32

Introduction ........................................................................................................................32

Assessment Performance ...................................................................................................33

Quality Monitoring ............................................................................................................36

Call Center Performance Metrics .......................................................................................39

Average Handle Time ........................................................................................................39

Occupancy Performance ...................................................................................................40

Chapter V: Discussion ...................................................................................................................42

Summary ............................................................................................................................42

Conclusions ........................................................................................................................45

Recommendations ..............................................................................................................46

References ......................................................................................................................... 49

7

List of Tables

Page

Table 1: Assessment Week 1 .........................................................................................................26






Table 7: Quality Monitoring Week 2 .............................................................................................29



Table 10: Quality Monitoring Week 5 ...........................................................................................30

Table 11: Quality Monitoring Week 6 ...........................................................................................31

Table 12: Average Handle Time ....................................................................................................32

Table 13: Occupancy .....................................................................................................................33

8

Chapter I: Introduction

The purpose of this research was to determine the effectiveness and impacts to

learning performance in a training environment when social learning tools and

collaborative communication devices are utilized in the training process. An additional

goal of this study was to better understand the learning retention and application

outcomes based on the use of specific social media and collaborative communication

tools within a blended classroom and web-based training module designed for customer

service employees at a large electric and gas utility.

Over the past two decades, several innovations have been added to

training curriculum and instructional design methods for adult learning. The

variety of tools used to provide job instruction to employees has evolved

significantly from the early use of on-line presentations such as simple Microsoft

PowerPoint and self-guided reading to technically heavy scenario-based delivery

models that require the training participant to interact with hardware devices,

software and people. The traditional presentation methods were effective in many

ways, but resembled self-study courses or classroom lectures and included little or

no interaction or collaboration components. New on-line classrooms often have

several advantages over the traditional delivery and static online presentation

formats of the past. As illustrated by Lehmann & Chambelin, the dynamic nature

of providing social media and collaborative communication channels can force

active learning and interaction (2009, p. 30).

The evolution of training delivery has recently moved into a new era with

the creation of social media and rapid collaborative communication devices.

9

Tools such as filesharing sites, blogs, wikis and discussion forums were

developed out of desire for electronic social interaction more than for the

deliberate use in a learning environment. As educators have done in the past with

technology-assisted learning such as audio, video recordings and the computer,

educators today have adapted the new communication tools for use in the training

classroom in order to reach specific participants.

There are several potential reasons that educators may include

communication devices and social media tools in their curriculum and training

designs. A desire to increase learning transfer and retention is often the priority

objective in most training designs, but other factors play an important role in the

utilization of the new technology and software as well. MacDonald, Gabriel and

Cousins (2000) addressed the large increase in adult learning in the workplace as

a means of improving efficiencies and reducing costs.

Cost is a major influence for the use of the online learning delivery, as is

the ability to participate from nearly anywhere. Students of all levels no longer

need to drive to a physical campus to participate in course delivery and campus-

based schools have been able to reduce overhead and facility requirements within

the campus footprint. Students are now able to complete degree requirements or

entire degree programs from any internet connection.

In the area of business education and job related training, the same holds

true. Worker participants no longer need to drive to the primary classroom site to

participate in training courses. Remote training creates efficiencies that would not

have been true a few years ago. The ability to conduct asynchronous and

10

synchronous deliveries has been widely expanded and accepted as a budget-

positive manner for conducting needed sustainment and new employee training.

Rogers (2002) determined that training which includes multi-media formats can

have beneficial impacts on the overall outcome of the training as well. While this

concept was one dimensional in the past, the missing component for such training

has been the interaction of student peers and instructors. The advent of programs

such Microsoft Netmeeting and MeetingPlace provided a means of verbal

synchronous communication, but these channels can be difficult to manage and

often turn into lecture based environments with little or no collaboration between

participants.

The initial format of computer-based delivery has become widely accepted

as a means of providing employee training and new forms or channels have been

constantly introduced as potential enhancements. The same corporations that

considered social media channels to be novelty only a few years ago are now

looking for ways to incorporate them into their learning models. A resurgence of

the use of constructivist learning theory has gained new attention with interaction

and collaboration considered as positive methods for learning discovery and

transfer. As corporations look to these new channels and tools, the question

concerning impact and return on investment is ever present.

With all of the choices for enhancing training designs via numerous media

channel possibilities, it is important that the curriculum designer remains focused

on the overall learning objective while making media and communication

choices. Rothwell & Kazanas (1998) proposed concepts for design and

11

construction that establishes the need for a primary delivery media or channel

selection to establish which format will drive the participants to successfully

reach learning objects while choosing secondary channels and communication

tools only as enhancements to the primary media. This point is even more

important to consider as the toolbox of options continue to change and grow

rapidly.

Many of the social media and communication devices used for

consideration in this project are less than twenty years old. Blogs and Wikis

evolved in the 1990’s and were originally developed as diaries, journals or simple

communication channels with a social focus. Threaded forums were also a

product of the 1990’s and were eventually adapted by educators and universities

more prevalently in the past 10 years. The growth of this form of communication

has exploded over the recent years and as of December 2007; blog search engine

Technorati was tracking more than 112,000,000 blogs in use. Since then, the use

and creation of blogs and other social media channels has continued to increase

exponentially. Training professionals and designers will need to better

understand the use and implications of incorporating one or many of the social

media formats in their learning designs to create efficacy while increasing the

collaborative functionality and constructivist theory.

This research project strives to answer key questions for the effectiveness

of the utilization of various, but specific social learning tools and communication

used for collaborative learning in a work related training environment. The

primary questions revolve around determining what best practices can be

http://en.wikipedia.org/wiki/Technorati

12

developed to provide collaboration and higher rates of transfer for retention

through job applications while keeping an eye on costs and return on investment.

While the importance of cost versus value is key to the business model and

decision making process, the primary focus of this research is to first establish the

performance aspect of tools and methods being considered.

Statement of Problem

The problem that is addressed in this study is the use of social media and

collaborative communication in an employee training environment for Utility ABC. The

current corporate conventional on-line training model will be adapted to include several

social media and collaborative communication tools to if there is a measurable

performance difference or outcome when these devices are incorporated into the current

curriculum through the multiple performance assessment methods that are common to the

customer service business model.

Purpose of the Study

The purpose of this study was to collect and analyze data related to social media

and collaborative communication tools as they relate to influencing service levels and

performance for a customer service training effort. The study will help to determine if

participants in the new hire training program show performance increased based on the

utilization of specific social media and collaborative communication tools via post-

training data collection. The information gathered will allow the training design team to

better understand the use of such tools and increase the potential for learning transfer and

performance. Performance data will also provide a foundation for business decisions

13

based on potential performance increases and potential cost decreases due to the

implementation of such tools.

Research Objectives

The scope of this project includes a study of approximately two hundred new

employees for Utility ABC. The participants consisted of a diverse group of individuals

including representation of employees from wide ranging age groups, ethnic backgrounds

and gender representation. All participants were hired based on the same baseline

educational and experience requirements used as a standard for Utility ABC. The

objectives for this study are as follows:

1. Determine the impacts and influence of social media and collaborative

communication tools on the standard customer service new employee training

for a large electric and gas utility.

2. Analyze standard post-training employee performance metrics to establish

impacts associated with the use of social media and collaborative learning

tools including the impacts to the following:

a. Assessment

b. Quality Assurance

c. Average Handle Time

d. Occupancy

Many of the tools utilized in this study have only been in existence for a few years

making it difficult to find direct comparative data or studies addressing the specific social

media and collaborative communication tools in question for this research project

implemented in a corporate training environment. There has been peripheral work done;

14

however, with the concepts of adult learning and other media formats that do directly

relate to the scope of this study. The use of various media formats and correlation to

learning effectiveness for adults and design techniques for performance has developed

polarizing opinions as it has been debated. One side of the argument would claim that

there is a direct impact to learning and retention that involve media components; the other

side of the argument would deny the actual impacts of the media itself as the component

responsible for increased performance.

While the incorporation of social media and collaborative communication tools

has been more widely accepted in the corporate training environment, the application

impacts for learning and retention are still not clear cut. Opinions vary concerning the

tools themselves. Clark (1983) established a theory concerning the use of media

available at the time and impacts for learning. Clark proposed that media are “mere

vehicles that deliver instruction but do not influence student achievement any more than

the truck that delivers our groceries causes changes in our nutrition” (1983, p. 22).

The addition of media channels has also raised the issue of the effectiveness of

traditional instructional design methods. According to Kozma (1993, p. 17).

“Traditional models of instructional design do not address the complex interrelationships

among media, method and situation. In general, they are not compatible with

constructivist, social models of learning, being as they are derived from behavioral

model.” The question of effectiveness has been debated for as long as media devices

have been used for education. This study will attempt to address the role of social media

and collaborative communication in the delivery in a static training environment with the

goal of determining impacts on the learning experience and performance outcomes.

15

Definitions of Terms

Blog: A web site that contains an online personal journal with reflections,

comments, and often hyperlinks provided by the writer; also: the contents of such a site.

(http://www.merriam-webster.com/dictionary/blog)

Constructivist Learning Theory: Is a theory of knowledge (epistemology) that

argues that humans generate knowledge and meaning from an interaction between their

experiences and their ideas. During infancy, it is an interaction between their experiences

and their reflexes or behavior-patterns.

(http://en.wikipedia.org/wiki/Constructivism_%28learning_theory%29)

Discussion Forums: Refers to communication via two or more networked

computers.

Filesharing: This is the practice of distributing or providing access to digitally

stored information, such as computer programs, multi-media documents, or electronic

books. It may be implemented through a variety of ways. (

http://en.wikipedia.org/wiki/File_sharing)

Glossaries: Refers to the traditional in-person educational setting where in

students and instructor meet at a specified physical location at a specified time.

Microblog: Refers to personal reflections done with severe space or size

constraints typically by posting brief messages about personal activities.

(http://www.merriam-webster.com/dictionary/microblog)

Occupancy: Occupancy is the percentage of time that call agents actually spend

handling incoming calls against the available or idle time, which is determined by

dividing workload hours by staff hours. It is usually measured in seconds.

(http://searchcrm.techtarget.com-/definition/occupancy)

http://www.merriam-webster.com/dictionary/blog

http://en.wikipedia.org/wiki/Epistemology

http://en.wikipedia.org/wiki/Constructivism_%28learning_theory%29

http://en.wikipedia.org/wiki/File_sharing

http://www.merriam-webster.com/dictionary/microblog

16

Social Media: Refers to forms of electronic communication (as Web sites for

social networking and microblogging) through which users create online communities to

share information, ideas, and other content. (http://www.merriam-

webster.com/dictionary/social%20media)

Wikis: A web site that allows visitors to make changes, contributions, or

corrections. (http://www.merriam-webster.com/dictionary/wiki?show=0&t=1290986320)

Assumptions of the Study

Assumptions for the study primarily revolve around the human subject variability.

The study will rely on the random nature of hires as compared to previous efforts. The

study will assume that new employees hired during the study time frame are equivalent in

experience and knowledge as compared to previous hiring efforts. Additional

assumptions are identified as follows:

1. The three service centers participating in the study will use screening and hiring

processes that are identical to those used to hire previous employees in this job

classification.

2. The course duration and all other delivery methods will remain constant in order

to maintain a valid sampling to correlate with historical performance data.

3. The sample size will also be significant in size to provide a large base of

employees in training and counteract any unwanted influence based on small

groups or individual instructor abilities.

Limitations of the Study

Limitations of the study include the possible lack of comfort and familiarity with

social media tools. Additional instruction could be required of the training participants to

http://www.merriam-webster.com/dictionary/social%20media

http://www.merriam-webster.com/dictionary/social%20media

http://www.merriam-webster.com/dictionary/wiki?show=0&t=1290986320

http://www.merriam-webster.com/dictionary/wiki?show=0&t=1290986320

17

ensure understanding of the tools and the functionality of each when any new tool or

system is added. Other potential limitations of the research include the following:

1. Various degree of participant experience and familiarity with the social media

and collaborative communication tools prior to the implementation.

2. Participant bias based on previous on-line learning experiences outside of the

structure of the corporate environment.

3. Participant experience and impact may be influenced by potential technical

difficulties with the tools or functions.

Methodology

The methodology used for this research project included the collection of

quantitative data related to new employee training applications and experiences. The

study also included verification of the application and deployment of several social media

tools in a real time delivery of customer service training including multiple state and

federal regulatory topics as well as computer based applications currently used in this job

classification. The current training curriculum was modified prior to application and

included discussion forums, blog access, file sharing and a glossary component.

The participant’s performance was compared to historical post-delivery

assessment scoring and performance and quality assurance scoring to better identify

overall performance implications through quantitative research. The construction of the

materials was approved in advance by the management group to ensure compliance with

corporate communication policy.

18

Chapter II: Literature Review

Introduction

The literature review in this chapter contains relevant concepts pertaining to the

use of social media and collaborative communication channels in a pedagogical training

environment. The primary focus of this review is to provide a base level understanding

of topics directly related the current theories, practices and research tied to the structures

of adult learning in a corporate training environment. The topics for discussion include

online corporate training, constructivist learning theory, adult learning theory and Post

Training Blended Design and Assessment.

Online Corporate Training

Online education and delivery has become a mainstay for many corporations and

training departments over the past decade. The initial growth in this industry is a result

of the corporate desire to deliver high quality learning modules in the most cost effective

manner. As the growth in online educational delivery and training has become more

common, peripheral tools and channels for use within the delivery platform have been

added at a fast pace. Many companies have successfully adapted the standard

presentation format for delivery online, but have struggled to understand the implications

for including multiple social media and collaborative communication channels within the

delivery model.

There are conflicting opinions and studies that relate to the use of such devices

and channels within a corporate training environment. In a study performed by the

American Society for Training and Development (2010), it was established that of the

nearly four thousand companies surveyed, most participants felt that their company was

19

not fully utilizing the available tools at a very high rate while the majority of respondents

reported that their company incorporated some form of social media or collaborative

communication in their training and job education efforts. Of the participants in the

American Society for Training and Development study, only 25.3% felt that social media

and collaborative communication was useful for learning and 14.4% felt that the end

result of using social media and collaborative communication helped workers to better

perform on the job (2010). One possible reason for the small percentage is that the

companies do not yet fully understand the capability of utilizing such tools for learning

and performance enhancement. Without a commitment to fully integrate social media

into their training plans, there is a gap in correlating a true relationship to achieving better

performance. With this gap also comes speculation concerning the learner style based on

generational differences and demographics.

Kumar (2009) argued that the effectiveness of social media channels and

collaborative communication tools in learning is tied directly to generational differences.

Learners born between 1977 and 1990 have much more familiarity with technology and

specifically social media. Learners in this category are sometimes labeled as the net

generation and have a preference for the inclusion of various forms of media in their

learning experience. Palloff & Pratt (1999) established a need for any current training

design to incorporate individual learner characteristics along with attention given to the

training environment selection including delivery content and media forms.

Most companies would agree concerning the importance of utilizing online

training tools for efficiency and cost effectiveness. The need for adult workers to quickly

grasp concepts and incorporate them into the daily job function is critical to the financial

20

health and longevity of any corporation. Medved (2010) explained that developing strong

training delivery programs is the key to achieving this objective. Training and

development programs found in most organizations can represent from 1.3% to 13% of

the companies payroll (Galvin, 2001). Because of this potentially large expenditure,

companies need to constantly evaluate the effectiveness of programs being funded.

Phillips explained “The large expenditures and the need to show value are two of the

primary drivers placing increased emphasis on the measurement and evaluation of

training and performance programs” (2020, p. 22). Divergent theories directly influence

the current state of uncertainty for most corporate decision makers. They are unsure of

impacts and therefore unwilling to risk resources on uncharted or unproven technologies

and methods. There is however resurgence in providing tools and methods based heavily

in the constructivist learning methodologies in many corporate environments.

Constructivist Learning Theory

Constructivist learning theory has several variations and interpretations reaching

back to research in adult learning and psychology including the work of Jean Piaget and

Lev Vygotsky. The basic premise of constructivist learning includes a characteristic

involving interconnectivity between learners and communities. Hein (1991), defines

constructivist theory as the idea that learners “build knowledge for themselves---each

learner individually (and socially) constructs meaning---as he or she learns.”

There are many other variations of the constructivist learning theory definitions

pertaining to the concepts, but for the purpose of this study, the researcher will focus on

the basic definition provided above. Within the structure of constructivist theory, the

learner needs to build upon individual realizations through connections made from the

21

learning process itself. Some of the basics of this type of learning application include the

need for active learning scenarios and participation. Lehmann & Chamberlin (2009)

described this concept for the adult learner as it pertains to Knowles theory as adult’s

requirement to be self-directed within the learning experience. The learner can achieve

this when they are allowed to have specific choices concerning the format of the learning

that they participate in.

Computer mediated communication and the constructivist learning model can

have many positive implications as well as the potential for some negative results.

Advantages include flexibility, interaction, quality, privacy and equality, while source

distraction, information overload, lack of cues and slow communication can be to the

learner’s disadvantage (Courtney, Vasa, Luo & Muggy, 1999). Careful construction of

training designs is a critical aspect of minimizing the distraction and noise, while

increasing the potential for experiential learning.

In a work environment for adult learners, incorporating constructivist learning for

on-the-job training includes recognition of experiences brought from the perspective of

the work community as well as the individual. The same characteristics should be

utilized to build on the entire community of knowledge rather than solely on the

individual in isolation. For a learner to apply the learning, they must interact and draw

out new information from the sharing process. Lehmann & Chamberlin described

constructivist learning as having the following five characteristics: The learning should

be “(a) situated in real life experiences and scenarios,

(b) activities are hands on, (c) dialogue between learners and facilitators is emphasized,

22

(d) learning is not prescribed – it is guided but allowed to be evolutionary, and (e)

assessment is authentic.” (2009, p. 19).

Applications of constructivist learning theory often incorporate actual job

experiences that relate to the concepts listed in the training objectives strengthening the

overall impacts of learning for both the individual and groups. Kliebard (2004)

established the importance for connections for learners as they encounter new concepts

and argued that there is a real relationship between all facets of the learner’s life

including home, school and work. Learning must include and maintain an interest and

value to the participants in order for them to reach new conclusions and realizations.

Adult Learning Concepts

There are many theories tied to adults and the manner in which they best learn.

One the best known is the theory of “andragogy” proposed by Malcolm Knowles in 1984

and revised in 1990 through his research. The theory addresses basic concepts

concerning the application of the environment in which adults learn as well as

antecedents that lead to conceptual understanding. Brown (1997) argued that to best

engage and facilitate learning in young adults and specifically adults categorized as

Generation X learners, the instructor must make learning experiential. The participants in

the learning environment should work within a cooperative in order to build on the

experiences and knowledge of others. By working in an interactive and cooperative

environment, the learner will be able to engage with other learners and expand beyond

their personal experiences to gain a new perspective for exploring learning.

Cyr (1999) described the characteristics of adult learners based on Knowles work

as having distinctive preferences. The adult learner becomes more independent over time

23

and is self-directed. The adult learner builds on actual experiences which are a base or

platform for building additional learning opportunities. Knowles theory also incorporates

social aspects which can occur in the traditional learning environment or informal

learning environments such as in the workplace.

Post Training Blended Design and Assessment

Studies have also been developed in recent years in order to identify the results of

blended design and delivery methodology in a corporate environment as well as post

training assessment effectiveness. The objective for studies in this area is primarily to

decrease learning curve times and increase performance potential. There have also been

attempts to understand the impacts of performance assessments and the impact on the

learning outcome when there is involvement by the administrator. Torrence (2007)

argued that the post training assessment is directly related to the concept of

instrumentalism. Learning participants perform better when there is a heavy formative

influence in the overall design of the materials and assessments. If directions are clear

and precise, they can impact the learner’s ability to perform at a higher level and increase

overall participant learning outcomes. On the other hand, this type of assessment may

also have detrimental impacts to the long term ability of the learner to think critically.

Coaching in a collaborative environment may in fact create a dependency on the

administrator for the participant related to criterion-based assessments and produce

results that are skewed.

Ledinar and Javenpaa (2006) developed other theories concerning criterion based

assessments and the blended classroom. Collaborative methodology in a technology-

based training delivery model can be more effective in terms of post training assessment.

24

The learner may in fact thrive in an environment where assessments provide additional

learning opportunities as well as confirmation of learning transfer. In a similar study,

Marshall and Drummond (2006) describe a scenario in which learner performance can be

divergent as compared to the results that would be derived from post learning

assessments that were autonomous or required concept completion outside of a

collaborative state. The primary difference is that learners in the collaborative assessment

environment may only gain specific short term understanding based on feedback instead

of building knowledge based in problem solving associated with critical thinking.

Summary

There has been a significant body of work constructed to support the various

training delivery models and technological impacts for adult learning methods in the past

two decades. There are differing opinions and opposing findings when it comes to

identifying the ideal model and there clearly is not one perfect answer when it comes to

design. There are financial advantages to building on-line designs that can be delivered

remotely and without time restraints. The virtual classroom opens opportunities for

training that can be delivered in a just in time training environment and cost effective

based on rapid construction and delivery.

The current movement towards increased utilization of on-line content with a

constructivist and collaborative blended approach shows promise and is ripe for

additional study and exploration. The debate concerning the impacts of collaborative

adult learning theory is similar in many ways to the questions concerning delivery tools

and methods. While there are benefits to conventional and constructivist curriculum

designs, there is not a clear cut method for constructing a one size fits all for business

25

environments. Research in this area will be important in future to better understand the

interaction of participants with technology and people in the learning process.

Assessment methodology is also open to validation and research as it relates to

post training performance impacts. While researchers and business managers may decide

that the collaborative nature of current online and blended deliveries can impact the

ability of the participant to conceptualize information and internalize the learning based

on shared experiences, there is much left to individual learning style and competency.

The counter point is that over utilization of formative assessment can in fact reduce the

participant’s ability to look outside of the specifics of the criterion based assessment for

building new understanding. Learning via coached assessment could be limited as

opposed to experiential learning that requires higher levels of critical thinking.

This study will not conclusively answer these questions, but it will attempt to

analyze specific impacts related to the topics of adult learning and designs for work

environments and corporate implementations. Understanding the components that impact

the design and effectiveness of the performance outcomes is an extremely important

component for creating high level learning opportunities and maximizing resources.

26

Chapter III: Methodology

Introduction

The goal of this research project was to better understand the use of social media

and collaborative communication for learning in a training environment. The study

included analysis of a social media enhanced training module to measure performance

differences when collaborative communication channels were added to a standard

delivery model. The objectives for this research were as follows:

1. Determining the impacts of social media and collaborative communication

tools on new employee customer service training for a large electric and gas

utility.

2. Analyzing standard post-training employee performance metrics to establish


tools.

Research Design

This research was based on a qualitative model utilizing a post-test only

experimental design approach. The participants would not be the ability to perform in an

assessment format prior to the implementation of new employee training since they

would have no knowledge concerning the systems and job requirements. The pilot

treatment for the social media and collaborative tools implementation was included in

random groups to determine if there was a performance advantage to including these

tools in future training deliveries. This research format was also ideal for avoiding

possible participant sensitization to the study treatment since the participants were

included in the piloting of the tools prior to the creation of this study. The data was

27

constructed in a manner reducing impact that prior knowledge of the study may have had

on the final performance data and the results.

The research design included three distinct components. The first component was

to identify the social media and collaborative communication tools utilized in the

business plan model for piloting within the current training curriculum. The tools were

chosen based on the availability, functionality and security parameters and restraints

within the test groups company. The second component included the determination of

the time parameters for data acquisition concerning the application of social media and

collaboration in the live training environment. The third component pertained to the

analysis of data collected during and after the training application delivery window. The

research was conducted after the implementation of the tools based on business decisions

to attempt a pilot program with social media and collaborative tools involving new

employees in training.

Research Participants

The participants for the study were randomly selected from a large group of new

employees within the test company. The participants were hired using the same standard

practices already in place by the test company. No additional information was gathered

at the time of hiring to define the level of employee experience with the learning tools

used for this research. The research plan included utilizing approximately 100 new

employees for participation over the course of two months. The participants were trained

in a classroom environment with extensive online coursework included in the standard

delivery model. The participants were monitored for performance which was compared

to historical and current performance of groups not included in the pilot.

28

Instrumentation

The research included four primary instruments for evaluation of the effectiveness

concerning social media and collaborative communication channels. The instruments

used were as follows:

1. Online testing and assessments

2. Quality assurance monitoring data

3. Occupancy performance data

4. Average handle time performance data

The online assessments were primarily used to determine conceptual

understanding and appropriate application and confirmation of the participant’s

proficiency with the training materials. The testing phase included multiple choice and

true and false selections completed at the end of each individual module during the new

hire training sessions. Historical data comparisons included new hire participants overall

scoring based on a two-year history. The data provided a baseline for comparison and

also included the historical performance of 165 employees who have participated in the

same training course excluding the social media and collaborative communication

components. The research compared the overall percentage of correct answers from this

historical sample base with the results derived from the new model including the social

media and collaborative communication tools.

The quality assurance monitoring data included a standard scoring sheet utilizing

a scale of one to one hundred percent, with one hundred percent as the high mark for

proficiency demonstrated through live customer based transactions and interactions. The

participants were monitored for accuracy and adherence to job specific transactional

29

policies during the training period. The quality assurance performance data was then

compared to historical employee performance during the training session and with the

same scale and criteria. The information gathered for comparison included the same 165

employees analyzed in the sample for online performance assessments. This data

comparison also included an overall percentage of transactional compliance to policy and

scoring with participants that were trained in the environment without the social media

and collaborative communication tools.

The third instrument included in the research is Occupancy Performance data.

The Occupancy data is an actual performance metric commonly used for call center

customer service employee. This metric describes the employee’s ability to be available

for in-coming calls. The correlation for this metric to higher performance revolves

around the employee’s strengths in completing call transactions while still on the phone

with the customer. Agents who are still in the learning phase of this job often require

additional time after the call to complete the system processes and notes before putting

themselves back into the queue for a new customer interaction. As employees learn and

retain more information, they are able to complete the call in a synchronous manner

increasing overall productivity.

The fourth research instrument to be measured in this study was average handle

time. The average handle time metric was a measure for the duration of the transaction

between the customer service employee and the customer and was calculated in seconds.

The importance of this metric was similar to the occupancy metric in that it measures the

performance capabilities of employees and their ability to quickly handle issues or handle

them in an appropriate amount of time.

30

As a business indicator, average handle time is used to determine staffing needs

and it has also been proven to be directly related to customer satisfaction with the

transaction or interaction. The average handle time metric in this research was used as a

baseline against archived data demonstrating the normal learning curve or ramp-up time

for a new employee to become proficient. The average handle time data was also

compared to employees who were trained at the same time, but not included in the social

media and collaborative tools classes.

Data Collection Procedures

The participants were selected randomly by class and location to represent a valid

sampling of the total population of employees across the multi-state footprint of Utility

ABC. The new employees participated in the same training modules and delivery

formats across all classes and were measured for performance using the same criteria as

all previous training participants. The new employees were divided into three groups.

Group A was provided with the training materials and presentations, but was also

provided with the social media and collaborative learning tools while Group B was only

provided with standard training materials and process. Group C is a compilation of

historical pre-pilot performance data to act as a baseline control for the study. Scoring

information was gathered from both the online assessments and monitoring data as the

participants moved through the training process. The occupancy and average handle time

performance data was gathered in a similar manner and included post-training collection.

Data Analysis

The data analysis captured for this study included multiple components. The

performance aspects reflected the current scoring format used by Utility ABC to

31

determine employee proficiency. The assessment was scaled from 0% to 100% with

100% as the high mark for performance. The quality assurance monitoring was measured

using the same scale. The occupancy was scaled in the same manner with the

participant’s ability to remain logged in and available at all times being represented by a

score of 100%. All three of these metrics will be compared to new employees who were

trained at the same time, but without the social media and collaborative tools as well as a

baseline historical group of employees for the past year who went through identical

training methods and materials, but without the social media and collaborative tools.

The average handle time metric was measure in seconds and was averaged to

account for various call types that require shorter or longer call handling procedures. The

average handle time was benchmarked against the average handle times of the historical

average for overall performance and also against the group of employees who were not

exposed to the social media and collaborative tools during pilot training.

Limitations

The limitations for the data collection method included accuracy in data collection

overall and human error potential. The scoring for performance indicators were based on

the same indicators used for all current employees. The study limitations include the

potential for skewed results based on individuals who come to the company with a higher

level of familiarity of the social media and collaborative communication tools from

outside sources. Human error was also possible in the scoring component for monitoring.

The monitoring scores are calculated manually and have a small subjective component

included with scoring measured for compliance to all known policies and procedures.

32

Chapter IV: Results

Introduction

Chapter IV will discuss the data collected for this study and the overall findings of

the research. The data reflects the performance metrics pulled from three separate groups

which are described below. All of the groups analyzed were involved in identical

training sessions including the duration of the sessions, delivery method and materials.

The training design included a six-week classroom delivery including simulation

programs, on-line modules and subject matter experts as well as lecture components and

access to a job specific knowledge base references. Group A, which is the focus of this

study, was also provided additional social media and collaborative tools within the

learning environment platform. The learning tools incorporated included a daily

discussion thread, a blog, a group glossary and a wiki space.

The data collected for this study included weekly performance metrics for

comparison as well as historical data. Group A included an average of 31 participants

over the six-week study period. The number of participants was reduced due to attrition

of participants from the company and the study as the weeks progressed.

Group B consisted of 43 participants on average through the 6 week course and

was also impacted by attrition during the class period creating a variable frequency or

participant count from the starting numbers to the end of the six-week class. The Group

A and Group B participants were provided with identical resources and tools for the class

period with the exception of the collaborative and social learning tools that were provided

to Group A. The data gathered from the performance metrics were broken into weekly

segments as would normally be gathered prior to this research.

33

Group C was composed of participant information from historical data. The

results represent data combined from the past two years of participant performance

results broken into the same weekly segments as Group A and Group B for comparison.

The performance data was compiled and the findings of the analysis are discussed in this

chapter.

Assessment Performance

The first performance metric for consideration in this study was participant

assessment results. The assessment results were combined to develop an average for

weekly performance. The assessments administered were identical across all groups and

were delivered in an on-line environment consisting of multiple choice questions. The

weekly assessments are aligned with specific customer service skills and are used to

determine the participant’s level of understanding for policies and procedures

conceptually as well as their ability to construct logical solutions to common customer

service requests utilizing the corporate knowledge base and customer resource systems.

In Table 1, assessment scoring for Group A in week 1 resulted in a .8%

improvement over the Group B assessment results. Group A testing results were equal to

the overall baseline scoring results of Group C. Group C represents average performance

scoring for the previous 165 participants in the new hire training across three separate

geographic locations for Company ABC during the past year.

Table 1 Assessment Week 1

Group Performance Frequency Per Group Percentage

Increase/Decrease

Group A 94% 34 0.00%

Group B 93.2 43 -0.8%

Group C 94% 165 Baseline

34

Table 2 demonstrates the scoring for week 2 of the training. The participant data indicated a

slight increase in performance for Group A. The Group A results was 3.7% higher than the

Group B control participants. Group A also demonstrated improvement over the historical

participant performance average and scored .3% higher than the Group C baseline results.



Increase/Decrease

Group A 94% 31 0.30%

Group B 90.30% 43 -3.40%

Group C 93.70% 165 Baseline

Table 3 results show the results for week 3 of the training program. In week 3, Group A

participants once again scored significantly higher than the Group B participants with an

improvement of 3.05% over the control group. The scoring for Group A was slightly higher

than the Group C baseline testing as well. The improvement over the Group C average was

.85% higher for week 3 overall.



Increase/Decrease

Group A 93.75% 30 0.85%

Group B 90.7 42 -2.20%

Group C 92.9 165 Baseline

Table 4 demonstrated data findings for week 4 of the training, The week 4 data provided similar

testing results for the group analysis. Group A scored 1% higher than control Group B. Group A

also scored 1.2% higher than the baseline scoring of Group C during week 4 testing indicating a

performance improvement over the historical baseline for week 4 results.

35


Group Performance Frequency Per Group Percentage Increase/Decrease

Group A 96% 30 1.20%

Group B 95 42 -0.20%


Week 5 testing results found in table 5 demonstrate significant increases for Group A

participants as well. Group A scored 2.5% higher than Group B. Scoring for week 5 has

historically been higher than the first 4 weeks of the training delivery. Group A scored 2.3%

higher than Group C.



Group A 97.50% 30 2.30%

Group B 95 41 -0.20%


Week 6 results were the most dramatic in terms of the difference between Group A and Group

B. Group A was 5% higher in overall scoring for the week. Group A was also 1.3% higher than

the baseline results for Group C as shown in Table 6.



Group A 95% 30 1.30%

Group B 90 41 -3.70%


The overall assessment results showed a slight increase for Group A performance over the

control and the baseline group. The overall average for the group was 95.03%. The Group B

scores for the same time period averaged 92.4%. The average score for Group C as a long term

baseline was 94.02%. The performance increase in terms of the assessments resulted in a 2.63%

36

improvement for Group A over Group B and a 1.01% increase over Group C. This demonstrates

indicates a slight advantage for the inclusion of social media and collaborative tools based on the

assessment metric for performance.

Quality Monitoring

The second performance metric for consideration in the study was Quality

Monitoring. The Quality Monitoring metric was based on a scale of 0% to 100%, with

100% as the highest possible score. Quality Monitoring data is gathered starting in week

two and completed in week six for all new hire training classes. The scoring for this

metric was similar to that of the training assessments in structure. The Quality

Monitoring data provides performance feedback to participants and measures the

customer service representative’s ability to apply the knowledge from the week in a live

environment with actual customers. The scores below were divided into three groups in

the same manner as the assessment scores.

Quality Monitoring scores are different in one respect. While customer service

representatives in training may take up to 30-40 calls during the testing week, only 4 calls

are randomly selected for monitoring. This can lead to variable results in any one

specific week, but is used as an acceptable sample for performance assessment by the

corporation and is a historical standard. Group A as the study group, Group B as the

control and Group C as the baseline for performance including all performance for

employees hired in the past 2 years.

As illustrated in Table 7, the week 2 Quality Monitoring results were mixed. The

Group A average scoring was only slightly better than Group B with a .2% increase in

37

performance. Group A was equal to the overall base line group with no increase over

Group C performance.

Table 7 Quality Monitoring Week 2


Group 1 88.3% 34 0.00%

Group 2 88.1 43 -0.20%

Group 3 88.3 165 Baseline

Group A scored slightly higher in week 3. The average Quality score was .1%

higher than control Group B. Group A results demonstrated an increase of 2.3% over

Group C. This is a significant increase over the historical performance average for week

3 performance. These results are found in Table 8.



Group 1 87.5% 31 1.10%

Group 2 87.60% 43 0.20%

Group 3 86.40% 165 Baseline

Table 9 indicates that Group A was significantly higher than both Group B and

Group C in week 4. The average score for Group A was 3.1% higher than Group B. The

Group A average performance was also 2.3% higher than the Group C historical

performance results.



Group 1 90.2% 30 2.30%

Group 2 87.10% 43 -0.80%


38

Table 10 illustrates the findings for week 5 monitoring results. The week 5 results also

showed an increase in performance for Group A over the other two groups. Group A average

performance was 1.1% higher than Group B. The Group A scoring was also higher than Group

C by .3% overall indicating an advantage for participants who were provided with the social

media and collaborative tools.



Group 1 91.5% 30 0.30%

Group 2 90.4 43 -1.20%


As demonstrated in Table 11, group A scored higher in week 6 in the area of

Quality Assurance monitoring. The Group A score was 5% higher than Group B. This

was the largest percentage of difference for all weeks measured. The Group A score was

also 3.2% higher than the Group C.



Group 1 92.6% 30 3.20%

Group 2 87.60% 43 0.20%


The tables above indicate that there was a scoring advantage for Quality Assurance

monitoring performance based on the Group A results. Group A participated in a

classroom design which was identical to the Group B and Group C design with the

exception of the social media and collaborative tools utilized in Group A. Quality

Assurance monitoring indicated that Group A performed at an overall average of 90.2%.

Group B averaged 88.16% over the five week period. Group C averaged 88.64%. The

39

Group A results indicated that it scored 2.04% higher than Group B. Group A also scored

1.56% higher than Group C. Overall, the Quality Monitoring data collection for this study

indicated that Group A performed slightly better than the other two groups.

Call Center Performance Metrics

The Call Center employees are measured via several additional methods as an

indicator of real performance proficiency as well. This study also looked at two of the

other comparative metrics that indicate the level of performance for customer service

representatives utilizing the same basic data structures. The additional metrics that will

be analyzed for this study include average handle time and occupancy.

Average Handle Time Performance

The first of the additional metrics for consideration was average handle time. This

is a measure in seconds for transactional phone time that is required to handle customer

requests. The ideal performance related to this metric is a call that is lower in average

handle time as compared to longer. The shorter time indicates employees who are better

able to handle questions and requests without constantly checking reference materials or

getting assistance from other employees. This is an important productivity measure and

is one of the determining factors for business costs due to the impact on staffing

requirements and labor dollars. For this study, the researcher compared the average

handle time for the first ninety days as an employee of the company.

Table 12 illustrates the difference in average handle time between the three groups.

Group A performed slightly better than Group B with a decrease in average handle time

of 2 seconds per call. Group A also outperformed the baseline group by 2 seconds per

call on average.

40

Table 12 Average Handle Time 0-40 Days


Group A 333 seconds 30 1 second

Group B 335 seconds 40 0 seconds

Group C 335 seconds 165 Baseline

Though the increase is not large overall, the findings do indicate a slight

performance advantage to the Group A performance for handling calls in a timely

manner. When translated into overall impact, it is important to understand that new

employees take approximately 2.4 million calls per year and a reduction of this quantity

would result in a reduction of labor by more than 1300 hours for the same time period.

This is directly related to cost of transaction and could be a driver for utilizing this

training method to push the standard learning curve in a more favorable direction.

Occupancy Performance

The second call center metric for consideration in this study is Occupancy.

Occupancy measures the ability of new employees to adhere to expectations for staying

logged into the phone system. Occupancy can be used to determine if the employee has

the skills to complete calls without logging out to access additional information, complete

transactional system requirements or double check understanding with specialists. High

performing call center service employees will maintain higher levels of Occupancy. This

metric is measure after the course work has been completed and is also an indicator speed

to competency overall. Occupancy is measured in percentage of overall time logged in.

The maximum performance potential is 100% Occupancy.

Table 13 illustrates the difference in Occupancy proficiency for Groups A, B and

C. There was a slight increase in Occupancy for Group A related to the performance of

41

Group B with an improvement of 0.7% overall. Group A performance was 0.5% better

than that of the Group C as well. While there was a slight improvement in Occupancy

with the group that incorporated the social media and collaborative tools, the significance

for this metric is minimal and the performance proficiency is nearly identical to the other

groups that did not have access to the tools.

Table 13 Occupancy 0-40 Days


Increase/Decrease

Group A 89.7% 30 0.5%

Group B 89.0% 40 -0.2%


Overall, the data indicates a relevant improvement in terms of the four

performance metrics analyzed in this study. There are several implications for utilizing

the social media and collaborative tools in the training designs as indicated above. These

implications range from staffing requirements to cost avoidance as well as implications

directly related to the customer experience and the quality of service. In these

implications will be further discussed in the following chapter.

42

Chapter V: Discussion

Summary

This study was constructed in order to identify performance and productivity

relationships concerning social media and collaborative tools for learning in a standard

training environment for new employees at Utility ABC. The study incorporated the use

of discussion threads, blogs, wiki’s and glossary tools in order to understand the

relationship between collaboration via these channels as an opportunity for constructivist

based experiences, and the learning and performance results related to their use.

Each participant in the study was provided identical resources with the exception of

the social media and collaborative learning tools which were enabled for the pilot group.

All of the training participants were provided with a twenty minute daily block of time to

check their individual email, complete time entry and read notices and news. The pilot

group was also provided with four primary social media and collaborative tools to use at

their discretion during the daily time block. The use of the collaborative tools was self-

guided, voluntary and not a requirement for any participant. The key performance

metrics were collected as per standard training practice and then combined for

comparison at a later date for this study. The primary goals and objectives set out for his

study included the following:

1. Determine the impacts and influence of social media and collaborative

communication tools on the standard customer service new employee training

for Utility ABC.

43

2. Analyze standard post-training employee performance metrics to establish


tools including the impacts to the following:

a. Assessment

b. Quality Assurance

c. Average Handle Time

d. Occupancy

The first research objective, Determine the impacts and influence of social media

and collaborative tools on the standard customer service new employee training for

Utility ABC, focused on the establishment of any relationship or correlation with the

utilization of social media and collaborative tools a training environment. This first

objective was achieved by this study and the data did demonstrate that there is a

relationship between the use of social media and collaborative tools was established in

conjunction with performance outcomes in the training environment. In all cases,

trainees that participated in the training using the social media provided exceeded

performance measures of the comparative group. It is clear from the collected data that

there was in fact an influential impact and relationship between the use of the

collaborative tools and performance.

The second objective, Analyze standard post-training employee performance

metrics to establish impacts associated with the use of social media and collaborative

tools, including the impacts to the following: assessment, Quality Assurance, Average

handle Time, Occupancy, concerning the specific performance metrics for assessment,

quality monitoring, average handle time and occupancy was also achieved. The data

44

supported that there was an increase in learning transfer and job related performance in

the experimental design structure. The data clearly indicates increases in performance for

those participants that utilized the social media and collaborative tools in training

environment.

The study structure was designed in such a way as to compare the performance

data between three specific groups within Utility ABC. Group A was provided with

social media and collaborative tools in a standard six week training course. Group B took

part in an identical six week training program, but without the incorporation of social

media and collaborative tools and Group C consisted of two years of historical

performance data for past participants that participated in the same six week training

program. The data gathered provided insight into specific performance increases and

also provided a foundation for additional research opportunities. Specifically, the

findings demonstrated that participants in the pilot group or Group A demonstrated an

increase in accuracy on the in-class assessments by a combined total of 2.63% overall.

The participants also achieved higher results for quality monitoring by 2.04% over the

historical scoring for the same performance metric. The pilot group performed slightly

better in terms of average handle time and required 2 seconds less to complete calls on

average and the group also demonstrated a small increase in occupancy over the two

control groups with an overall improvement of .7%.

The performance in each of these testing areas demonstrated an advantage to

utilizing the social media and collaborative tools in the training sessions. The combined

impacts of the scoring data provided important implications beyond the individual results.

The higher scores for quality monitoring were achieved while decreasing average handle

45

time and while increasing occupancy. These three metrics were tied directly to customer

experience and also to business costs or staffing requirements for the organization. In

other words, the calls were handled better and faster resulting in potential increases for

customer survey results in satisfaction while demonstrating the potential for reduced

staffing costs of up to 1300 hours per year based on the 2 second reduction during a

typical year of call volume.

The cost of constructing the tools was minimal since the development platform was

based on a Microsoft SharePoint site that was already in use by the training organization.

There was an additional labor cost for building the tools on an individual class basis, but

this was offset by the potential savings in staffed hours related to increases in

performance. There was also no increase in participant labor required since the

participant’s utilized time already set aside for learning in the sessions. Additional study

would need to be constructed to validate the savings potential, but from the initial

findings the estimated savings for this study would include the reduced staffing

requirement of 1300 hours on a yearly basis. The overall performance increases indicated

that new employees performed at least incrementally better in a constructivist

environment where they were allowed to utilize social media and collaborative tools for

learning.

Conclusions

This study included utilization of several sources for measuring performance data

tied to the delivery methods and channels included in a standard new employee training

session. The performance data indicated an advantage or correlation between the use of

social media and collaborative tools in the delivery model. The study also indicated that

46

there was a slight increase for performance levels during the training sessions as well as

immediately after the training was completed in comparison to historically gathered

information for participants in training that were not provided the social media and

collaborative tools.

The data also indicates that there is an increase in performance in all four

performance metrics on at least an incremental basis. The classroom learning metrics

clearly showed an overall increase in assessment scoring and the quality monitoring

performance also indicates a benefit to utilizing social media and collaborative tools in

the classroom. The average handle time showed an increase as did the occupancy metric;

therefore, the tools should continue to be utilized in the training environment to increase

performance potential of newly hired employees.

There is a financial incentive for utilizing social media and collaborative tools to

increase performance and decrease the overall learning curve associated with training

customer service employees at Utility ABC. There also appears to be an increase in the

participant’s ability to process information and demonstrate that knowledge through

written assessments, quality monitoring for practical application and based on standard

call center metrics around average handle time and occupancy. The research established

that based on the increases listed in the study for key performance metrics as well as

financial potential, Utility ABC should continue to utilize social media and collaborative

tools in the training model for new employees in the customer service job category.

Recommendations

Based on the results of this study, the data indicates that there was a relationship

between the use of social media and collaborative tools in the training program as an

47

enhancement to the current training materials and practices and increased performance

results of those who utilize those tools in a training environment. The researcher’s

recommendations based on this studies finding are as follows:

1. The assessment results for participants who took part in the enhanced version of

the training including the social media and collaborative tools performed at a

higher level than those who were not provided with the tools. The participants

tool advantage of opportunities to collaborate with peers in the online

environment and increase the collective knowledge of the group. They had equal

opportunity to collaborate face to face and the same amount of time as all other

participants. The primary difference was the channel for communicating between

peers, therefore the researcher recommends that this method for incorporating

social media and collaborative tools should continue in the training model.

2. Although the study did indicate increases in performance across all four of the

metrics commonly used by Utility ABC, the researcher recommends that the

study should be replicated to ensure long term validation of the applications prior

to wide based implementation.

3. Additional research is required to better understand the cost implications for

implementing the social media and collaborative tools implementation practices

in the corporate training environment. The researcher recommends that the cost

benefits for including the social media and collaborative tools should be

confirmed to determine current and potential savings due to performance and

staffing changes, while calculating any potential increases in design time, trainer

time or resources required for wide-based implementation of these tools.

48

4. Input from the researcher’s faculty committee recommended the dissemination of

this study and its findings to appropriate, selected professional groups.

49

References

Clark, R. E. (1983 Winter). Media will never influence learning? Educational

Technology Research and Development, 32(4), 7-19. Retrieved Oct 10, 2010,

from http://www.springerlink.com/content/1042-1629/

Cobb, T. (1997). Cognitive efficiency: Toward a revised theory of media. Educational

Technology Research and Development, 45(4), 7-19. Retrieved Oct 10, 2010,

from http://www.springerlink.com/content/1042-1629

Courtney, S., Vasa, S., Luo, J., & Muggy V. (1999). Characteristics of adults as learners

and implications for computer-based system for information and instruction. MN:

Minnesota Department of Economic Security, Americas Learning Exchange.

Cyr, A. V. (1999). Overview of theories and principles relating to characteristics of adult

learners: 1970s - 1999. Clearwater, Florida : Cyr Consultant Service.

Fosnot, C. T. (1983 Winter). Media and Technology in Education: A Constructivist

View. Educational Technology Research and Development, 32(4), 7-19.

Retrieved Oct 10, 2010, from

http://www.springerlink.com/content/d77816g671651n22/

Galvin, T. (2001). Industry Report 2001,

http://eric.ed.gov/ERICWebPortal/search/detailmini.

Hein, G. E. (2001). Industry Learning Theory, International Committee of Museum

Educators,

http://www.edtechpolicy.org/AAASGW/Session2/const_inquiry_paper.pdf

Kliebard, H. M. (2004). The Struggle for the American Curriculum, New York, NY:

RoutledgeFalmer.

http://www.springerlink.com/content/1042-1629/

http://www.springerlink.com/content/1042-1629

http://www.springerlink.com/content/d77816g671651n22/

http://eric.ed.gov/ERICWebPortal/search/detailmini

50

Kozma, R. B. (1994). Will Media Influence Learning? Reframing the Debate.

Educational Technology Research and Development, 42(2), 7-19. Retrieved Oct

1, 2010, from http://www.springerlink.com/content/1042-1629/

Lehmann, K., & Chamberlin, L. (2009). Making the Move to eLearning, Lanham, ML:

Rowman & Littlefield Education.

MacDonald, C. J, Gabriel, M. A. & Cousins, J. B. (2000). Factors influencing adult

learning in technology based firms, Journal of Management Development.

(pp.220-240).

Mager, R. F. (1997). Measuring Instructional Results, Atlanta, GA: The Center for

Effective Performance Inc.

Medved, M. B. (2010). When to Use Social Media for Learning. Learning Circuits,

ASTD. Retrieved Nov 1, 2010, from

http://www.astd.org/LC/2010/0510_medved.htm

Palloff, R. M. & Pratt, K. (1999). Building Strategies for the online classroom, (pp. 33-

45). San Fransisco, CA: Jossey-Bass Inc,. Publishers.

Rogers, P. L. (2002). Designing instruction for technology enhanced learning, (161-183.

Hersey, PA: Idea Group Publishing.

Rothwell, R. J., & Kazanas, H. C. (1998). Mastering the Instructional Design Process,

San- Francisco, CA: Jossey-Bass/Pfeiffeer.

Torrance, Harry. (2007). Assessment in Education: Principles, Policy & Practice,

Nov2007,

Vol.14 Issue 3, p281-294, 14p; DOI: 10.1080/09695940701591867

http://www.astd.org/LC/2010/0510_medved.htm

Date post:	31-Jul-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

1 Evaluation of Collaborative Adult · training curriculum and instructional design methods for...

Documents