Final IDS Duties Tasks - DECADE Consulting Instructo… · list of ranked tasks performed as part...

Learning Systems Institute: Copyright © 2005 Florida State University. All rights reserved

IDS Duties and Tasks in an Online Learning Environment

Prepared for the

Naval Education and Training Command

Contract Number: N00140-03-Q-2897

Prepared by:

The Learning Systems Institute

Florida State University

Suite C4600 University Center

Tallahassee, Florida 32306-2540

February 22, 2005


2

Acknowledgements The Learning Systems Institute would like to acknowledge the assistance of Betty Brown from Academic Computing and Network Services for providing instruction on data analysis. The project team is also grateful to all the administrators who forwarded our study participation request to their instructors. The team is most appreciative of all the instructors who afforded us their valuable time to complete the online survey and our focus group members.


3

Table of Contents Executive Summary ........................................................................................................................4 Overview .........................................................................................................................................6 Methodology....................................................................................................................................6 From Instructor Competencies to Instructor Duties and Task ................................................... 6 Research Design..........................................................................................................................7 Survey Development.................................................................................................................. 8 Sample ........................................................................................................................................9 Data Collection ........................................................................................................................10 Data Analysis ...........................................................................................................................10 Results ...........................................................................................................................................11 Demographics ..........................................................................................................................11 Tasks Performed by Instructors ............................................................................................... 12 Importance of Performed Tasks............................................................................................... 19 Time Spent Performing Tasks ..................................................................................................26 Conclusions ...................................................................................................................................30 References .....................................................................................................................................32 Appendix A: Online Instructor Duties and Task List ....................................................................34 Appendix B: Online Instructor Duty and Task Survey..................................................................37 Appendix C: List of Tasks in Performance Order .........................................................................71 Appendix D: List of Tasks by Importance Mean...........................................................................73 Appendix E: List of Tasks by Time Mean.....................................................................................75 Appendix F: Demographic Data ....................................................................................................77 Appendix G: Summary Data for Each Task ..................................................................................83


4

Executive Summary The Naval Education and Training Command has multiple initiatives underway to increase the efficiency and effectiveness of training. Instructors will play a key support role in implementing the Navy’s Revolution in Training by transitioning to instruction in the Integrated Learning Environment (ILE). This report focuses on duties and tasks that may define the Instructional Delivery Specialist’s job as an instructor in the online environment (See Facilitator Competencies in an Integrated Learning Environment, submitted to NETC by the Learning Systems Institute at Florida State University in October 14, 2003, for a comprehensive review of applicable theories and concepts contained in current literature). It is submitted in fulfillment of the task to revalidate IDS skill sets for the online component of the integrated learning environment.

Procedures

LSI conducted searches of recent publications to confirm whether additional instructor competencies, duties, or tasks had been published since the October 2003 literature review, and to verify appropriate methodology for task inventories. The project team generated an inventory of online instructor duties and tasks from the list of competencies. This inventory was then reviewed by a focus group of U.S. Navy instructors. The revised inventory was forwarded as an online survey to approximately 367 instructors currently teaching online at military commands and public or private institutions of higher learning. The research questions for this study were defined as:

1. What tasks are instructors currently performing to deliver their online course? 2. How do online instructors rate the importance of the tasks they perform? 3. How do online instructors rate the relative time spent performing these tasks?

Results

One hundred forty-eight (40 %) of the instructors completed the survey. The survey resulted in the identification of seventeen tasks that 90% of the sampled instructors indicated they performed as part of their job as an online instructor. Of these seventeen tasks, eight appeared across every sub-group sample (three military and one non-military). The two tasks rated highest based on their importance mean were Reviewing the course for accuracy and Assessing learners’ attainment of learning objectives, with the latter also having the highest time mean in the sample. No task in the overall sample was rated below a mean of 3.00 (Moderate) in relative importance by instructors who performed the tasks. In contrast, almost half of the tasks instructors indicated they performed rated below a mean of 3.00 (About the same) in relative time spent performing the task. Overall, based on importance mean, the instructors rated Provide rewards and incentives for student participation as the task with the lowest relative importance (3.08) and Use strategies to address undesirable interaction behavior as the task with the lowest relative time performing the task (2.07). The standard deviation for means ranged from 0.642 to 1.199 for importance ratings and from 0.843 to 1.187 for time spent ratings. Two tasks were suggested as additions to the inventory: Online program advising (suggested by one respondent; ratings: Very


5

Important; Less time spent) and Advocacy for online education within institutional context (suggested by one respondent; ratings: High importance, More time spent). Complete data from the survey are available in the appendices. Seventy-five percent of the sample indicated they had experience as an online student or trainee and had a post-graduate degree (85.2 %). Female instructors were the majority in the overall sample (55.4 %) and the non-military sample (67.7 %) while male instructors were the majority in the military sample (69.4 %). Instructors in the sample population indicated they had received on-the-job training (75.7 %) and/or training courses or workshops (69.6 %) that prepared them to be an online instructor. Most instructors in this study reported their standard courses had a length of three to four months (78.2 %). Slightly over half the instructors indicated they used only an asynchronous delivery method (54.8 %). Instructors also reported their courses were moderately interactive (44.5 %) or highly interactive (37.6 %), and that the typical taxonomic level of the learning outcomes for their online course was higher order skills (72.4 %). Most instructors also reported their typical class size as 11 to 20 students (43.5 %). Complete demographic data are available in Appendix F: Demographic Data.

Conclusions

It is anticipated that information from this study will be useful for developing online instructor training programs and for developing online instructor evaluations. All tasks in the inventory were performed by some portion of online instructors and all tasks received an overall importance mean of at least moderate importance (across all instructors). Only two tasks were proposed as additions to the inventory, suggesting that the task listing conforms to common best practices for online instructors. This study found a task performed by a large majority of instructors does not necessarily indicate the task’s importance level or the amount of time (relative to other instructor tasks) required to perform the task. There were some differences in which tasks instructors performed, and in how they rated the relative importance and time spent performing tasks as part of their jobs as online instructors. These differences appeared to be associated with whether instructors were teaching in a military or non-military environment. Several areas for further study have emerged from this research. One area that was not explored was the student perspective on the most important instructor tasks. This perspective could be valuable in focusing instructor training efforts and in evaluating online instructors. There were also some unexpected similarities that emerged when comparing novice instructors with experienced instructors regarding tasks performed and their importance ratings. In many cases, instructors with moderate levels of experience rated task importance lower than the novice and experienced instructors. These findings suggest the need for further data analysis and follow-up interviews. The sample sub-groupings chosen for analysis affected the comparison results presented in this document. Other approaches to analysis of these data, perhaps with different sub-groupings, could yield other interesting results.


6

Overview

The Naval Education and Training Command (NETC) has multiple initiatives underway to increase the efficiency and effectiveness of training. Instructors are seen as playing a key support role in implementing the Navy’s Revolution in Training as they transition to instruction in the Integrated Learning Environment (ILE). To ensure effective instructor performance in this environment, the Learning Systems Institute (LSI) at Florida State University previously developed a list of competencies for the Instructional Delivery Specialist (IDS) that were supported by a comprehensive review of applicable theories and concepts contained in current literature. As a preliminary validation effort, LSI identified and invited professional educators and experienced practitioners to review the list of competencies and to provide content validation (See Facilitator Competencies in an Integrated Learning Environment, submitted to NETC on October 14, 2003). This document has been prepared for the Naval Education Training Command (NETC) as part of its Integrated Learning Environment Framework Project with Florida State University. It is submitted in fulfillment of the task to revalidate IDS skill sets for the online component of the integrated learning environment. The main body of the document discusses the task background, methodology used in revalidation, and the results and conclusions reached through the revalidation process. The appendices present the list of online instructor duties and tasks and detailed data summaries from the online survey.

Methodology

To identify the skills needed by online instructors in the Navy learning environment, the project team first searched the literature for new studies related to online instructor competencies that had not been included in the earlier document submitted to NETC in October 2003. Several studies were identified as similar to our research task: (a) Competencies for Distance Education Professionals, conducted by Thach and Murphy, 1995; (b) Differential Skills and Competencies Required of Faculty Teaching Distance Education Courses, conducted by Schoenfeld-Tacher and Persichitte in 2000; and (c) Roles and Competencies for Distance Education Programs in Higher Education Institutions, conducted by Williams in 2003. The results of these studies were compared with our existing list of competencies to see if additional competencies should be added to the list. The project team determined that skills and competencies produced in these studies corresponded to existing project competencies and that no additions were needed. The competencies list was also reviewed against the 2003 Instructor Competencies, produced by the International Board of Standards for Training Performance and Instruction, and several modifications were made to the project competency list based on that review.

From Instructor Competencies to Instructor Duties and Tasks However, the project team determined that the competencies and roles identified throughout these studies remained understandably vague for NETC’s ultimate purposes of designing training for online instructors and evaluating online instructor performance. In the pursuit of defined instructor skills (rather than roles) LSI initiated an abbreviated job analysis study to produce a


7

list of ranked tasks performed as part of the job. It was reasoned that a job analysis would produce a better description of optimal online instructor performance in terms of duties (groupings of related tasks) and tasks (single behaviors in a job that perform a meaningful function of the job). The identification of the duties and tasks for any job is considered critical for effective training design, communication of expectations, and evaluation of performance. Job inventories (also referred to as a task inventory) usually are completed by incumbents as the basis for reporting some job-related information about the incumbent’ positions (McCormick, 1976). Because there were no current IDS personnel functioning in the Navy’s online environment, the primary challenge in this revalidation effort was to predict the tasks and duties for a job that was being developed, rather than for one that already existed.

The project team’s approach to identifying necessary skills for this particular function of the IDS’s job was to identify the important tasks and time-consuming tasks for online instructors both within the military training environment and external to the military context. In the absence of access to job incumbents, the LSI research team drafted an inventory of duties and tasks for online instructors based on the updated instructor competencies list. Cumulative results from the other contract deliverables were also reviewed for potential additions to the list of IDS duties and tasks. This approach provided a rapid prototype document for a small group of available military job incumbents to review the list and to provide feedback. The revised list of instructor tasks formed the basis of an online survey administered to a convenience sample of instructors currently teaching online in both military and non-military institutions. Online Instructor Duties and Tasks Duties are a major subdivision of work performed by one individual and therefore duties are actually clusters of tasks. The online instructor duties were generated by grouping related tasks, resulting in four duty areas for online instructors: (a) Planning and Preparing for Instruction, (b) Implementing and Managing Instruction, (c) Assessing Learners, and (d) Evaluating Courses. Unlike the description of duties, tasks are discrete performances and the lowest level of behavior in a job that describes the performance under consideration. As previously described, the tasks for the various online instructor duties were created from the previous competency list, feedback from prior survey results, review of other competency studies and competency definition efforts, and the inclusion of findings from other deliverables on the Navy-Learning Systems Institute 2003-2004 contracts. The task statements were written following the guidelines provided by Cox, J., Rayner, G., and Branson, R. (1998). The final task inventory presented to the study participants included 54 individual tasks for online instruction (see Appendix A: Online Instructor Duties and Tasks List).

Research Design The research questions for this study were defined as: 1. What tasks are instructors currently performing to deliver their online course? 2. How do online instructors rate the importance of the tasks they perform? 3. How do online instructors rate the relative time spent performing these tasks?


8

The method used to answer these questions was an online survey distributed to a convenience sample of online instructors. Particular efforts were made to identify and survey online instructors in a military setting. Rating scales used in the online survey were based on recommendations provided by McCormick (1976), which indicated that valuable task information relates to time spent on tasks, importance of the task to the position, and frequency of task performance. Relative time-spent scales were recommended as more successful than those based on absolute time or percentages of time (Bruskiewicz, K. Hanson, M., Kubisiak, U., Wright, C.; 1996). The importance of each task was measured on a five-point Likert scale with levels consisting of (a)Very High, (b)High, (c) Moderate, (d) Low, and (e)Very Low. The relative time spent on each task was also measured on a Likert scale consisting of five levels comparing time spent on the selected task to other tasks: (a) Much More, (b) More, (c) About the Same, (d) Less, and (e) Much Less.

Survey Development A focus group consisting of two faculty members from the Navy Postgraduate School and two faculty members from the Navy War College was used to assist in the development of the text (tasks listing) for the online survey. This group also pilot tested the online version of the survey once it was programmed and available on the Internet.

Tasks were submitted to a focus group of military online instructors for review and feedback. Each focus group member was requested to provide feedback regarding our assumptions, inclusion of demographic questions, task descriptions, and language use or interpretation. Each member of the focus group was provided with a list of assumptions for the IDS’s duties and tasks that were associated with the responsibilities of online instructors. The following assumptions were provided to focus group participants:

• A course is the basic unit of instruction. • Learner analysis has been conducted and learner needs have been addressed during the

course design phase. • The IDS is assigned to online courses that have been completely designed, developed,

reviewed for instructional adequacy, and pre-packaged for delivery (including scheduled asynchronous and synchronous activities).

• The IDS is authorized to modify course content to ensure accuracy. • The IDS may accommodate learner characteristics or equipment needs by adjusting

programmed instructional strategies (e.g., demonstration, role play, case study, discussion).

• The IDS may provide online learners with basic computer skills assistance (e.g., word processing, saving files to folder, creating e-mail attachments).

• The IDS is authorized to provide supplemental online learning resources, add or modify discussion questions, and to invite guest speakers.

• The IDS does not have the authority to make major course revisions (e.g., changes to any learning objective or terminal objective, to increase course length, or to make any change that requires additional resources).


9

• The IDS is not responsible for providing technical support to learners. Appropriate survey revisions were made based on feedback received from the focus group (e.g., one task was removed for redundancy and Task 38 was changed from Maintain focus of the learning activities to Use strategies to maintain the learners’ focus on the learning activities).

Sample

As discussed in Simon (2002) and Williams (2000), when using a survey or questionnaire there is an established practice of using a group of experts in the field to provide the data. This study used the following criteria to identify an individual as an expert: (a) must be employed or associated with a military command or academic institution of higher learning, and (b) must be currently teaching a course in the online environment.

The study was conducted using a convenience sample of 148 instructors and faculty members who were currently teaching an online course. A review of IP addresses for completed surveys revealed the sample included experts from 23 states, two Canadian provinces, the Netherlands, and Australia. The sample represents 12 online instructors from military commands and 37 online instructors contacted through a second round of emails who identified themselves as currently working in a military environment. An additional 96 online instructors identified themselves as not currently working in a military environment. Three instructors did not identify themselves as currently teaching in either environment (military or non-military); therefore their data are only used in overall statistical analysis.

Email contact was made with a total of 38 military commands that mentioned distance, online, eLearning, or distributed courses as being available through their websites. This initial contact resulted in a total of 12 completed surveys from five commands (the Navy Postgraduate School, Naval Medical Center San Diego, Army Medical Division Non-Commissioned Officer Academy Distance Learning Program, Canadian Forces Management Development School, and the Air Force Institute of Technology).

Since this initial contact did not result in a sufficient number of completed surveys to adequately conduct the study, emails were sent to member schools of the Navy College Program Distance Learning Partnership, the Community College of the Air Force program, the eArmyU Learning Network, and schools affiliated with the Servicemembers Opportunity Colleges Navy consortium not already contacted in any of the previous attempts. (It was discovered that some schools contacted through previous lists were also members of the Servicemembers Opportunity Colleges Navy consortium). Because this consortium consists of over 1800 schools, it was determined that the best approach would be to contact only those schools associated with the Navy. Contact was made with a total of 88 schools requesting their participation in the study. A personal email was also sent to 30 individuals who had published articles or conference papers related to distance learning in the military. Finally, an email requesting participation from online instructors was posted to the following listservs: ASSESS, DEOS, EDTECH, ITFORUM and WWWDEV. In sum, a total of 161 email requests were made to participate in the study. Demographic information about study participants is presented in the Results section of this document and in Appendix F.


10

Data Collection

Data collection was conducted using “protection of human subjects” guidelines approved by the Florida State University Institutional Review Board that informed participants that information obtained during the course of the study would remain confidential, and that participation was completely voluntary.

The study was conducted using SurveyConsole, a Web-based survey tool, over a five-week period between December 2004 and January 2005. The survey consisted of 54 questions related to tasks associated with the actual performance of teaching an online course. The survey also asked the participants to answer 19 demographic questions that could be used to analyze the data. These questions were also used to capture a demographic picture of current online instructors constituting the sample. LSI was unable to determine the response rate as more than half of the emails (88 of 161) were sent to administrative personnel in distance learning programs with a request to forward the email to instructors in their online programs. There were, however, 367 experts who viewed the first page of the survey, 259 experts who started the survey (clicked on the Continue button on the first page) and 148 experts who completed the survey (clicked on the Submit button on the last page). This represents a 70.57 viewed-to-started percentage, a 57.14 started-to-completed percentage, and a 40.33 viewed-to-completed percentage. These percentages fall within the framework of expected response rates for email and Web-based surveys (Tello, 2002).

Data Analysis The study was descriptive in nature and relied primarily on descriptive measures to analyze the data. Data from the completed surveys were processed using Statistical Package for Social Sciences (SPSS) version 12.0 for analysis and tabulation. A measure of central tendency (mean) and a measure of variability (standard deviation) were used to analyze the data. The analysis process for the collected data consisted of categorization and rank ordering using descriptive frequencies. Data were categorized into groups based on teaching environment. Task rankings were based on two factors: (a) the number of instructors who performed the task and (b) high and low mean scores of the task ratings. It was decided that data collected from incomplete surveys would not be used in the data analysis which reduced the final sample size from 259 to 148. Because the results of this study are to be used by Naval Education and Training, participants were divided into three subgroups for some aspects of the data analysis, based on whether or not they were currently teaching in a military setting. The first group, Military Group A, was composed of instructors who were contacted through military commands currently offering distance learning courses. The second group, Military Group B, was composed of instructors contacted through post-secondary schools who identified themselves as currently teaching in a military setting. For ease of contrasting instructors teaching in a military setting with instructors teaching in a non-military setting, military groups A and B were sometimes combined into one military group (Military Group C). Those instructors who were contacted through post-secondary schools and who indicted that they were not currently teaching in a military environment were identified as Non-Military instructors.


11

The point values for the importance scales and for the relative time spent scales were adjusted during data analysis so that a higher rating signified a more highly important task, and less important tasks were represented by lower values. Similarly, higher values on the relative time spent scale indicated more time spent on a task. This was done to make the results more intuitive to common understandings rank ordering.

Results

The study offered some clarification regarding which tasks online instructors were currently performing to deliver their courses, how they ranked the importance of each task, and the relative time spent performing these tasks. This section presents the results of this research. Analysis of the demographic data is presented first. The remainder of this section discusses results of the survey in the context of the research questions: 1. What tasks are instructors currently performing to deliver their online course? 2. How do online instructors rate the importance of the tasks they perform? 3. How do online instructors rate the relative time spent performing these tasks?

Demographics There were 148 participants who completed the task inventory presented within SurveyConsole; however, several individuals did not respond to one or more demographic questions. Consequently the number of responses to various demographic questions changes from one set of responses to another. The following results are percentages from the overall population surveyed. Most respondents were online instructors (94.6%), with prior experience as an online student and/or trainee (74.7%), and who reported a high (44.8%), very high (26.5%), or moderate (25.9%) level of expertise using instructional technology. However, the majority of these respondents stated that they did not instruct online courses in a military environment. Demographics for all participants are included in Appendix F. Of the population of respondents, the majority were female (55.4%), and over 50 years of age (56.2%). The majority of instructors had at least a Master’s degree (59.5%) and had at least 20 years of experience teaching in any environment -- classroom or distance (30.4%). Their preparation to be instructors occurred within many contexts: college or university settings (89.2 %), and through training courses (68.2%). Most had 6 to 10 years of experience instructing online (46.9%) and their preparation to perform as an online instructor was obtained through on-the-job training (75.7%) and training courses (69.6%). Demographics Related To Online Courses A high percentage of instructors responding to the task inventory instruct asynchronous courses only (54.8%) while 61 % of the same population reported instructing both asynchronous and synchronous courses. The majority taught 1 to 5 online courses (27.7%) and 20.3 % of the same population taught as many as 6 to 10 online courses. The most common range of different courses (i.e., distinct courses with different subjects, rather than iterations of the same course)


12

taught online was 1 to 2 courses (47.3 %). Most instructors reported typical course duration of 3 to 4 months (78.2 %). Instructors also reported that their courses were moderately interactive (44.5%) or highly interactive (37.6%) and that the typical taxonomic level of the learning outcomes for their online course was higher order skills (72.4%). Student Demographics The participating online instructors reported the typical age range of students within their online course was 26 to 33 years (34.9%) and the typical number of students per online course was 11 to 20 (43.5%). Perhaps unique to the online environment is the finding that 25.3 % of the instructors responded that they did not know the age range of their online students. Demographic Differences When comparing demographic information across the subpopulations (military and non-military) three major differences emerged: gender, years of experience teaching in any environment, and the number of online courses taught. The non-military respondents consisted of a majority of females (67.7%); however the military respondents consisted of a majority of males (69.4%). The second difference across instructor populations was teaching experience in any environment. Most non-military instructors (33.3%) reported over 20 years of teaching experience, whereas most military instructors reported 6 to 10 years experience (26.5%). The third difference was found in the total number of online courses taught. Most non-military instructors (34.4%) reported a total of 1 to 5 online courses taught, while most military instructors (42.9%) reported a total of more than 25 courses taught. The total number of courses taught reflects only courses taught completely online, but may represent multiple iterations of the same course.

Tasks Performed by Instructors This section addresses the first research question: What tasks are instructors currently performing to deliver their online course? Participating instructors were asked to indicate whether each task was part of their job as an online instructor by choosing either Yes or No on the survey screen. If instructors answered Yes, they were asked to rate the task in terms of importance and relative time spent performing the task. If instructors answered No, they proceeded to the next task.

Instructor Tasks Performed in All Environments Overall, the two most performed tasks were Task 1 (Review the course for accuracy) and Task 39 (Share supplemental learning resources with the learners) with 97% of all instructors indicating they performed the two tasks. The means for relative time spent performing the two tasks were relatively similar (3.01 vs. 2.88) but the difference in means for importance of each task showed Task 1 rated much higher (4.58) than Task 2 (3.51). This finding shows Reviewing a course for accuracy to be much more important to the sample group than Sharing supplemental learning resources with learners. The next two tasks performed by most instructors (96% of instructors performed them) were Task 42 (Assess learners’ attainment of learning objectives) and Task 8 (Maintain expertise in subject area). Both these tasks were rated as similar in importance (means of 4.55 and 4.38) and in relative time spent on the tasks (means of 4.22 and


13

3.45). Table 1 shows the 17 tasks that 90% of all instructors indicated were part of their jobs as online instructors. A complete list of tasks ranked in order of performance by the instructors in the sample can be found in Appendix C. Table 1 Tasks performed by 90% of all instructors and their mean ratings of relative importance and relative time spent

Overall (n=148) n Importance

Mean Time Mean

Task 1 – Review the course for accuracy 144 4.58 3.01 Task 39 – Share supplemental learning resources with the learners 144 3.51 2.88 Task 42 – Assess learners’ attainment of learning objectives 143 4.55 4.22 Task 8 – Maintain expertise in subject areas 143 4.38 3.45 Task 9 – Maintain expertise in online instructional techniques … 142 3.86 3.01 Task 17 – Maintain record-keeping related to course activities 141 4.40 2.89 Task 2 – Make changes as need to maintain course accuracy 141 4.31 3.33 Task 30 – Use strategies to create a friendly and open environment 140 3.97 3.34 Task 33 – Use questions that promote higher order thinking 138 4.26 3.59 Task 46 – Provide feedback on the accuracy of learners’ statements … 137 4.24 3.85 Task 47 – Provide feedback to learners on how to correct errors 137 4.09 3.61 Task 21 – Provide directions for assignments and other instructional activities 136 4.25 3.46 Task 22 – Provide assessment criteria 136 4.01 2.99 Task 20 – Communicate learning goals and objectives … 135 4.13 3.08 Task 7 – Identify critical instructional areas, personalized or relevant examples… 135 4.04 3.37 Task 32 – Use questions that promote discussion among learners 134 4.05 3.36 Task 49 – Follow-up individually with learners who fail to participate … 133 3.99 3.00

Instructor Tasks Performed in a Military Environment Like the overall group, 98% of instructors currently teaching in a military environment indicated that Task 1 (Review course for accuracy) and Task 39 (Share supplemental learning resources with learners) are part of their jobs as online instructors. The differences in means for importance (4.67 vs. 3.48) and time spent (2.96 vs. 2.79) for both tasks are consistent with that found in the overall group. Ninety-eight percent of instructors currently teaching in a military environment indicated Tasks 2, 8, and 9 were also part of their job as an online instructor, which is similar to the overall group of instructors. However, the number of tasks that were performed by 90% of instructors in this group is less (13) than that found in the overall group (17) or the non-military group (19). Table 2 shows the 13 tasks that 90% of all instructors currently teaching in a military environment indicated were part of their job as an online instructor. The four tasks that did not appear for this group but were performed by at least 90% of the overall sample were: Task 20 (Communicate learning goals and objectives for each instructional component), Task 21 (Provide directions for assignments and other instructional activities), Task 32 (Use questions that promote discussion among learners) and Task 49 (Follow-up individually with learners who fail to participate in course activities). These four tasks did, however, rate as one of the more performed tasks in the military sample; at least 86% of instructors in this group indicated the four


14

tasks were part of their job as an online instructor. Complete data for this group are available in Appendix C. Table 2 Tasks performed by 90% of instructors teaching in a military environment and their mean ratings of relative importance and relative time spent

Military Group C (n=49) n Importance

Mean Time Mean

Task 1 – Review the course for accuracy 48 4.67 2.96 Task 2 – Make changes as need to maintain course accuracy 48 4.38 2.79 Task 8 – Maintain expertise in subject areas 48 4.25 3.29 Task 9 – Maintain expertise in online instructional techniques … strategies 48 3.58 2.85 Task 39 – Share supplemental learning resources with the learners 48 3.48 2.79 Task 33 – Use questions that promote higher order thinking 47 4.21 3.66 Task 42 – Assess learners’ attainment of learning objectives 46 4.57 4.24 Task 47 – Provide feedback to learners on how to correct errors 46 4.26 3.96 Task 30 – Use strategies to create a friendly and open environment 46 3.89 3.59 Task 17 – Maintain record-keeping related to course activities 45 4.27 3.29 Task 46 – Provide feedback on the accuracy of learners’ statements … 45 4.24 3.96 Task 22 – Provide assessment criteria 45 3.91 3.00 Task 7 – Identify critical instructional areas, personalized or relevant examples… 44 3.93 3.32

Instructor Tasks Performed by Military Group A Members Military Group A was composed of instructors who were contacted through military commands currently offering distance learning courses. This group, although relatively small (12 participants), represents a sample that may most closely resemble Navy IDS job incumbents—because these instructors are currently teaching online courses at and under the direction of a military command, and were contacted through military channels—when compared to the other groups. However, these instructors may also be teaching college-level and postgraduate courses, which may be different than the bulk of training delivered by the Navy IDS online. The top three tasks as indicated by all instructors in this group were Task 1 (Review the course for accuracy), Task 2 (Make changes as need to maintain course accuracy), and Task 17 (Maintain record-keeping related to course activities). This group had the highest importance mean for Task 1 (4.75) as compared to the importance mean for the other groups [Overall (4.58), Military Group B (4.64), Military Group C (4.67), and Non-Military (4.53)]. The data also showed the mean time spent performing this task compared to other tasks for this group was lower than that reported in the other groups (2.67), compared with a mean of 3.01 for all instructors, a mean of 2.96 for Military Group C, and a mean of 3.06 for both Military Group B and Non-Military. Five tasks were performed by 90% of Military Group A instructors that were not performed by 90% of instructors in any other group. These tasks are:


15

Task 11 (Use appropriate technology to support instructional administration) Task 16 (Ensure all course activities comply with organizational rules…) Task 23 (Use strategies to gain and maintain learner attention) Task 53 (Solicit input from learners for course improvements) Task 54 (Use evaluation data to recommend course improvements)

Table 3 shows the 13 tasks 90% of the Military Group A sample indicated were part of their job as an online instructor. Complete data for this group are available in Appendix C. Table 3 List of tasks performed by 90% of Military Group A instructors and their mean ratings of relative importance and relative time spent

Military Group A (n=12) n Importance

Mean Time Mean

Task 1 – Review the course for accuracy 12 4.75 2.67 Task 2 – Make changes as needed to maintain course accuracy 12 4.42 2.42 Task 17 – Maintain record-keeping related to course activities 12 3.92 3.00 Task 47 – Provide feedback to learners on how to correct errors 11 4.36 3.91 Task 8 – Maintain expertise in subject areas 11 4.36 2.91 Task 7 – Identify critical instructional areas, personalized or relevant examples… 11 3.73 2.91 Task 11 – Use appropriate technology to support instructional administration 11 3.55 2.82 Task 23 – Use strategies to gain and maintain learner attention 11 3.55 2.73 Task 53 – Solicit input from learners for course improvement 11 3.36 2.64 Task 9 – Maintain expertise in online instructional techniques … 11 3.36 2.45 Task 16 – Ensure all course activities comply with organizational rules … 11 3.27 2.09 Task 54 – Use evaluation data to recommend course revisions 11 3.18 2.64 Task 39 – Share supplemental learning resources with learners 11 3.09 2.18

Instructor Tasks Performed By Military Group B Members Military Group B was composed of instructors who were contacted through institutions of higher learning and who indicated they were currently teaching in a military environment. All members of this group indicated Tasks 8, 9, 33, and 39 were part of their jobs as online instructors. Of the four tasks which all members of this group noted were part of their job, Task 33 (Use questions that promote higher order thinking) received the highest importance mean (4.27) and the highest time mean (3.76). This task was not performed by the Military Group A instructors. Additionally, 97% of the Military Group B members responded that Tasks 1, 2, 22, 30, and 42 were part of their jobs as online instructors. Tasks 1 and 2 were performed by 90% of all instructors and by 90% of Group A Military instructors. However, tasks 22 (Provide assessment criteria), 30 (Use strategies to create a friendly and open environment), and 42 (Assess learners’ attainment of learning objectives), performed by Military Group B, were not performed by at least 90% of the Military Group A instructors. The explanation of these differences must be sought through follow-on study.


16

Table 4 shows the 18 tasks 90% of Military Group B members indicated was part of their job as an online instructors. Complete data for this group are available in Appendix C.

Table 4 List of tasks performed by 90% of Military Group B instructors and their mean ratings of relative importance and relative time spent

Military Group B (n=37) n Importance

Mean Time Mean

Task 33 – Use questions that promote higher order thinking 37 4.27 3.76 Task 8 – Maintain expertise in subject areas 37 4.22 3.41 Task 9 – Maintain expertise in online instructional techniques … 37 3.65 2.97 Task 39 – Share supplemental learning resources with the learners 37 3.59 2.97 Task 1 – Review the course for accuracy 36 4.64 3.06 Task 42 – Assess learners’ attainment of learning objectives 36 4.58 4.31 Task 2 – Make changes as need to maintain course … 36 4.36 2.92 Task 30 – Use strategies to create a friendly and open environment 36 4.08 3.86 Task 22 – Provide assessment criteria 36 3.94 3.06 Task 46 – Provide feedback on the accuracy of learners’ statements … 35 4.29 4.00 Task 47 – Provide feedback to learners on how to correct errors 35 4.23 3.97 Task 32 – Use questions that promote discussion among learners 34 4.26 3.74 Task 49 – Follow-up individually with learners who fail to participate … 34 4.03 3.29 Task 43 – Provide opportunities for learners to self-assess their performance 34 3.65 3.03 Task 17 – Maintain record-keeping related to course activities 33 4.39 3.39 Task 21 – Provide directions for assignments and other instructional activities 33 4.12 3.39 Task 7 – Identify critical instructional areas, personalized or relevant examples… 33 4.00 3.45 Task 36 – Require learners to comply with communication guidelines 33 3.52 3.00

Instructor Tasks Performed by Non-Military Group Members The Non-Military group was composed of instructors who were contacted through institutions of higher learning who indicated they were not currently teaching in a military environment. The majority of this group of instructors (94 of the 96 instructors) indicated that Task 42 (Assess learners’ attainment of learning objectives) was the most performed task in their job as an online instructor. There are some differences in the tasks performed by 90% of the Non-Military instructors when compared to the tasks performed by 90% of the military instructors. The additional tasks performed by most Non-Military instructors included:

• Task 20 – Communicate learning goals and objectives … • Task 21 – Provide directions for assignments and other instructional activities • Task 31 – Use strategies to solicit sharing of personal knowledge … • Task 32 – Use questions that promote discussion among learners • Task 35 – Model appropriate communication behavior


17

Task 21 (Provide directions for assignments and other instructional activities), however, was performed by the 90% of the larger sub-group of military instructors, Military Group B. These differences in tasks performed by 90% of instructors in the two groups (military and Non-Military) may indicate different views of what the online instructor role requires. Of the 19 tasks performed by 90% of the instructors in this group, Task 42 tied for the highest importance mean (4.53) with Task 1, but had the highest time mean (4.19) when compared to the other eighteen tasks. Table 5 is a list of the 19 tasks that 90% of Non-Military group members indicated were part of their job as an online instructor. Complete data for this group are available in Appendix C. Table 5 List of tasks performed by 90% of Non-Military instructors and their mean ratings of relative importance and relative time spent

Non-Military (n=96) n Importance

Mean Time Mean

Task 42 – Assess learners’ attainment of learning objectives 94 4.53 4.19 Task 1 – Review the course for accuracy 93 4.53 3.06 Task 17 – Maintain record-keeping related to course activities 93 4.30 3.34 Task 39 – Share supplemental learning resources with the learners 93 3.51 2.94 Task 8 – Maintain expertise in subject areas 92 4.46 3.55 Task 9 – Maintain expertise in online instructional techniques … 91 4.01 3.08 Task 30 – Use strategies to create a friendly and open environment 91 4.00 3.23 Task 2 – Make changes as need to maintain course accuracy 90 4.39 2.91 Task 21 – Provide directions for assignments and other instructional activities 90 4.30 3.51 Task 20 – Communicate learning goals and objectives … 90 4.21 3.07 Task 46 – Provide feedback on the accuracy of learners’ statements … 89 4.21 3.79 Task 31 – Use strategies to solicit sharing of personal knowledge … 89 3.64 3.01 Task 33 – Use questions that promote higher order thinking 88 4.26 3.53 Task 7 – Identify critical instructional areas, personalized or relevant examples… 88 4.08 3.39 Task 22 – Provide assessment criteria 88 4.05 2.99 Task 32 – Use questions that promote discussion among learners 88 4.00 3.26 Task 47 – Provide feedback to learners on how to correct errors 88 3.98 3.40 Task 35 – Model appropriate communication behavior 87 4.13 3.24 Task 49 – Follow-up individually with learners who fail to participate … 87 3.94 2.86

In summary, the tasks performed and the number of tasks at least 90% of the sampled instructors perform as part of their job as an online instructor changed based on the organizational environment. The eight tasks that appeared in every group were:

• Task 1 – Review the course for accuracy • Task 2 – Make changes as need to maintain course … • Task 7 – Identify critical instructional areas, personalized or relevant examples… • Task 8 – Maintain expertise in subject area • Task 9 – Maintain expertise in online instructional techniques … • Task 17 – Maintain record-keeping related to course activities


18

• Task 39 – Share supplemental learning resources with the learners • Task 47 – Provide feedback to learners on how to correct errors

Tasks Least Performed by Instructors When looking at the two least performed tasks, Task 5 (Verify the existence and practicality of a back-up plan for personnel or resources required for course delivery) and Task 15 (Diagnose technical problems related to the course) rated as the two least performed tasks for the overall group and each sub-group, except Military Group A. Overall, only 27% of instructors indicated they performed Task 5 and only 34% performed Task 15. For Military Group A, the least performed was Task 28 (Guide learners’ ongoing self-evaluation of their personal learning strategies). Only one instructor indicated they performed this task. Task 15 was also rated as one of the two least performed tasks for Military Group A instructors, along with three other tasks with only 3 (25%) instructors indicating they performed the task. A speculation as to why Task 5 did not appear in Military Group A might be the emphasis in the military on including back-up plans in all their endeavors. This task was, however, rated in the bottom third of tasks performed by Military Group A instructors. Another speculation is the technology has become more reliable, rendering back-up plans less important. Instructors in Military Group A and all military instructors rated Task 25 (Provide rewards and incentives for student participation) as the second least performed task. For instructors in Military Group B, Non-Military and overall, Task 25 rated as the third least performed task. The two least performed tasks by group are shown in Table 6. Table 6 List of tasks least performed by instructors and their mean ratings of importance


Mean Std.

Deviation Task 15 - Diagnose technical problems related to the course 51 3.43 1.044 Task 5 - Verify the existence and practicality of a back-up plan … 40 3.38 1.170


Mean Std.

Deviation Task 48 - For group projects, provide feedback on the performance … 3 4.00 0.000 Task 34 - Provide collaboration guidelines 3 3.00 2.000 Task 15 - Diagnose technical problems related to the course 3 3.00 1.000 Task 25 - Provide rewards and incentives for student participation 3 2.67 1.155 Task 28 - Guide learners' … self-evaluation of their personal learning … 1 3.00 0.000


Mean Std.



Mean Std.

Deviation Task 25 - Provide rewards and incentives for student participation 18 2.94 0.938


19

Task 5 - Verify the existence and practicality of a back-up plan … 16 3.56 0.892 Task 15 - Diagnose technical problems related to the course 16 3.50 1.095

Non-Military Group (n=96) n Importance

Mean Std.


Importance of Performed Tasks This section addresses the second research question: How do online instructors rate the importance of the tasks they perform? If instructors answered, Yes to the question of whether they perform the task as part of their job as an online instructor, they were asked to rate the task in terms of relative importance using a five-point scale ranging from Very Low to Very High. If an instructor did not consider a task to be part of their job as an online instructor, he did not have the opportunity to rate the task for importance nor for relative time spent on the task. The following results are related to performed tasks that were given a mean importance rating of at least 4.00 (high) or above for the overall group and each independent group of instructors. For the overall group, 17 tasks received an importance rating of High or Very High. Task 1 (Review course for accuracy) was rated as most important with a mean of 4.58 followed by Task 42 (Assess learners’ attainment of learning objectives) with a mean of 4.55. Task 1 was rated as the most important for Group A military instructors (4.75), Group B military instructors (4.64) and Military Group C (4.67) followed by Task 42. Non-military instructors rated Task 1 and Task 42 as equally important (4.53). Six tasks appeared in the top ten for mean importance across all groups:

• Task 1 (Review the course for accuracy) • Task 2 (Make changes as needed to maintain course accuracy) • Task 8 (Maintain expertise in subject area) • Task 33 (Use questions that promote higher order thinking) • Task 42 (Assess learners’ attainment of learning objectives) • Task 46 (Provide feedback on the accuracy of learners’ statements or performances)

Two tasks appeared in the top ten for every group except Military Group A. These were Task 12 (Maintain online security and privacy) and Task 17 (Maintain record-keeping related to course activities). Task 21 (Provide directions for assignments and other instructional activities) appeared in the top ten for each group except Military Group B. Less than half of the instructors in every group and overall indicated they performed Task 12 (Maintain online security and privacy). This may be a reflection that the responsibility for online security and privacy is a task performed by someone else in the organization other than the instructor. But when performed by the instructor, this task was rated High or above in importance. The mean rating was 4.24 for all instructors, 4.42 for all military instructors, 4.00 for Group A military instructors, 4.50 for Group B military instructors, and 4.13 for non-military


20

instructors. Tasks rated High or Very High in importance appear in Table 7 below. Appendix D contains complete data on tasks listed by importance means. Table 7 Means and standard deviations for tasks rated as “high” or “very high” in importance


Mean Std.

Deviation Task 1 – Review the course for accuracy 144 4.58 0.642 Task 42 – Assess learners’ attainment of learning objectives 143 4.55 0.699 Task 2 – Make changes as need to maintain course accuracy 141 4.40 0.755 Task 8 – Maintain expertise in subject areas 143 4.38 0.787 Task 17 – Maintain record-keeping related to course activities 141 4.31 0.836 Task 33 – Use questions that promote higher order thinking 138 4.26 0.758 Task 21 – Provide directions for assignments and other … 136 4.25 0.748 Task 46 – Provide feedback on the accuracy of learners’ statements ... 137 4.24 0.853 Task 12 – Maintain online security and privacy 72 4.24 1.041 Task 20 – Communicate learning goals and objectives … 135 4.13 0.871 Task 35 – Model appropriate communication behavior 122 4.13 0.802 Task 47 – Provide feedback to learners on how to correct errors 137 4.09 0.874 Task 32 – Use questions that promote discussion … 134 4.05 0.826 Task 7 – Identify critical instructional areas … 135 4.04 0.827 Task 6 – Prepare for subject areas where learners may have questions … 130 4.02 0.876 Task 22 – Provide assessment criteria 136 4.01 0.865


Mean Std.

Deviation Task 1 – Review the course for accuracy 12 4.75 0.452 Task 42 – Assess learners’ attainment of learning objectives 10 4.50 0.707 Task 2 – Make changes as need to maintain course accuracy 12 4.42 0.515 Task 47 – Provide feedback to learners on how to correct errors 11 4.36 0.924 Task 8 – Maintain expertise in subject areas 11 4.36 0.924 Task 51 – Provide instructor input for course improvement 9 4.22 0.667 Task 21 – Provide directions for assignments and other instructional … 10 4.20 0.919 Task 49 – Follow-up individually with learners who fail to participate 9 4.11 0.782 Task 46 – Provide feedback on the accuracy of learners’ statements … 10 4.10 0.738 Task 33 – Use questions that promote higher order thinking 10 4.00 0.817 Task 18 – Maintain scheduled hours for student-initiated contact 5 4.00 1.414 Task 3 – Verify contact information for student support … 5 4.00 0.707 Task 12 – Maintain online security and privacy 4 4.00 1.414 Task 48 – For group projects, provide feedback on the performance … 3 4.00 0.000


Mean Std.

Deviation Task 1 – Review the course for accuracy 36 4.64 0.683 Task 42 – Assess learners’ attainment of learning objectives 36 4.58 0.770 Task 12 – Maintain online security and privacy 22 4.50 0.802 Task 17 – Maintain record-keeping related to course activities 33 4.39 0.827 Task 2 – Make changes as need to maintain course accuracy 36 4.36 0.867 Task 46 – Provide feedback on the accuracy 35 4.29 0.825 Task 33 – Use questions that promote higher order thinking 37 4.27 0.769 Task 32 – Use questions that promote discussion … 34 4.26 0.751


21

Task 47 – Provide feedback to learners on how to correct errors 35 4.23 0.843 Task 8 – Maintain expertise in subject areas 37 4.22 0.854 Task 35 – Model appropriate communication behavior 28 4.18 0.772 Task 21 – Provide directions for assignments and activities 33 4.12 0.650 Task 30 – Use strategies to create a friendly and open environment 36 4.08 0.841 Task 20 – Communicate learning goals and objectives for each … 32 4.06 0.716 Task 49 – Follow-up individually with learners who fail to participate … 34 4.03 0.870 Task 7 – Identify critical instructional areas, personalize or relevant … 33 4.00 0.791 Task 6 – Prepare for subject areas where learners may have questions 32 4.00 0.950


Mean Std.

Deviation Task 1 – Review the course for accuracy 48 4.67 0.630 Task 42 – Assess learners’ attainment of learning objectives 46 4.57 0.750 Task 12 – Maintain online security and privacy 26 4.42 0.902 Task 2 – Make changes as need to maintain course accuracy 48 4.38 0.789 Task 17 – Maintain record-keeping related to course activities 45 4.27 0.863 Task 47 – Provide feedback to learners on how to correct errors 46 4.26 0.855 Task 8 – Maintain expertise in subject areas 48 4.25 0.863 Task 46 – Provide feedback on the accuracy of learners’ statements … 45 4.24 0.802 Task 33 – Use questions that promote higher order thinking 47 4.21 0.778 Task 21 – Provide directions for assignments and activities 43 4.14 0.710 Task 32 – Use questions that promote discussion … 43 4.12 0.823 Task 35 – Model appropriate communication behavior 32 4.06 0.801 Task 49 – Follow-up individually with learners who fail to participate … 43 4.05 0.844


Mean Std.

Deviation Task 42 – Assess learners’ attainment of learning objectives 94 4.53 0.683 Task 1 – Review the course for accuracy 93 4.53 0.653 Task 8 – Maintain expertise in subject areas 92 4.46 0.747 Task 2 – Make changes as need to maintain course accuracy 90 4.39 0.745 Task 17 – Maintain record-keeping related to course activities 93 4.30 0.831 Task 21 – Provide directions for assignments and other instructional … 90 4.30 0.771 Task 33 – Use questions that promote higher order thinking 88 4.26 0.750 Task 46 – Provide feedback on the accuracy of learners’ statements … 89 4.21 0.885 Task 20 – Communicate learning goals and objectives for each … 90 4.21 0.855 Task 12 – Maintain online security and privacy 46 4.13 1.108 Task 6 – Prepare for subject areas where learners may have questions 86 4.13 0.794 Task 35 – Model appropriate communication behavior 87 4.13 0.804 Task 7 – Identify critical instructional areas, personalize or relevant … 88 4.08 0.834 Task 22 – Provide assessment criteria 88 4.05 0.870 Task 23 – Use strategies to gain and maintain learner attention 85 4.02 0.886 Task 9 – Maintain expertise in online instructional techniques … 91 4.01 0.876 Task 30 – Use strategies to create a friendly and open environment 91 4.00 0.894 Task 32 – Use questions that promote discussion … 88 4.00 0.831

Additional tasks that had fairly high importance means (between 3.8 and 4.0), based on ratings by all instructors are listed below with the number of instructors who performed the task and the accompanying importance means and standard deviations. Several of these tasks relate to the


22

facilitative aspect of the instructor role, focusing on monitoring discussions, proactive follow-up with learners, creating a learner-friendly environment, and maintaining learner attention and focus. Two tasks generally relate to ensuring that skills learned will be used on the job: establishing relevance of the instruction, and promoting transfer. Two tasks concern the instructor’s personal level of expertise and the need to document data for any evaluations of their teaching. Less than half of the instructors in this study performed the final task in this list, ensuring that any additional resources required for the instruction were available to learners. But, for those instructors who saw this task as part of their job, many instructors rated it fairly high in importance. Table 8 List of tasks with importance means between 3.8 and 4.0


Mean Std.

Deviation Task 49 - Follow-up individually with learners who fail to participate … 133 3.99 0.857 Task 30 - Use strategies to create a friendly and open environment 140 3.97 0.881 Task 23 - Use strategies to gain and maintain learner attention 128 3.96 0.891 Task 44 - Monitor group discussions 126 3.95 0.920 Task 50 - Document all evaluation data 119 3.91 0.863 Task 52 - Evaluate your performance as an online instructor 92 3.88 0.970 Task 24 - Establish the relevance of the instruction to learning goals … 126 3.87 0.933 Task 9 - Maintain expertise in online instructional techniques … 142 3.86 0.912 Task 41 - Use strategies to promote the transfer of learning to the job … 108 3.81 0.855 Task 38 - Use strategies to maintain the learners' focus … 121 3.81 0.820 Task 4 - Verify that any required equipment, tools, … are available 70 3.80 1.199

Least Important Performed Tasks No task the sample performed as part of their job as an online instructor received a mean importance rating below 2.00 (Low). Task 25 (Provide rewards and incentives for student participation) was ranked as the least important task overall (mean of 3.08), and by instructors in Military Group B (mean of 3.00), Military Group C (mean of 2.94), and Non-Military instructors (mean of 3.10). Military Group A instructors rated Task 37 (Use strategies to address undesirable interaction behavior) as the least important task they performed (mean of 2.60). This may not be an indication of the importance of the task but more a reflection of the military environment where respect for authority and peers is an expected behavior. Similar to other groups in the sample, Task 25 also had a low importance mean (2.67) in Military Group A. The least important task based on mean for each group is shown in Table 9 below.


23

Table 9

List of tasks with lowest mean importance ratings

Overall (n=148) n

Importance Mean

Std. Deviation

Task 25 - Provide rewards and incentives for student participation 62 3.08 1.013


Mean Std.

Deviation Task 37 - Use strategies to address undesirable interaction … 5 2.60 0.894


Mean Std.



Mean Std.



Mean Std.


Importance Ranking of Tasks Performed by 90% of Instructors Task 1 (Review the course for accuracy) was ranked as the top task based on the number of instructors who indicated they performed the task (144) and also in mean importance (4.58). Task 1 was the most performed task and had the highest importance mean (above 4.50) for all but the non-military group, where it was second to Task 42 (Assess learners’ attainment of learning objectives) but had a similar importance mean of 4.53. The data also suggested that the likelihood of performing a particular task was not necessarily a predictor of its perceived importance. This is evident in a number of instances. For example, in the overall sample, Task 1 and Task 39 (Share supplemental learning resources with the learners) were both performed by the same number of instructors (144) but the importance means were quite different. Task 1 appeared as the most important (mean of 4.58) while Task 39 represented the least important (mean of 3.51) task performed by at least 90% of the instructors. A similar finding was noted in the Non-Military sub-group where 90 % of instructors (93) indicated they performed Task 1 and Task 39 but their importance means placed Task 1 at the top with a mean of 4.53 while Task 39 was rated as the least important with a mean of 3.51. Another interesting finding was that Task 46 (Provide feedback on the accuracy of learners statements or performance) was performed by 90% of instructors overall and in all sub-groups with the exception of Military Group A. Ninety percent of Military Group A instructors did, however, report a similar task as among those they perform: Task 47 (Provide feedback to learners on how to correct errors). The following table shows the list of tasks 90% of instructors indicated they performed, ranked by importance means.


24

Table 10

Tasks performed by 90% of instructors ranked by mean rating of relative importance

Overall (n=148) n Min Max Mean

Std. Dev.

Task 1 - Review the course for accuracy 144 2 5 4.58 0.642Task 42 - Assess learners' attainment of learning objectives 143 2 5 4.55 0.699Task 2 - Make changes as needed to maintain course accuracy 141 1 5 4.40 0.755Task 8 - Maintain expertise in subject area 143 2 5 4.38 0.787Task 17 - Maintain record-keeping related to course activities 141 2 5 4.31 0.836Task 33 - Use questions that promote higher order thinking 138 2 5 4.26 0.758Task 21 - Provide directions for assignments 136 2 5 4.25 0.748Task 46 - Provide feedback on the accuracy of learners' statements or … 137 1 5 4.24 0.853Task 20 - Communicate learning goals and objectives for each … 135 1 5 4.13 0.871Task 47 - Provide feedback to learners on how to correct errors 137 2 5 4.09 0.874Task 32 - Use questions that promote discussion among learners 134 2 5 4.05 0.826Task 7 - Identify critical instructional areas, personalized or relevant … 135 2 5 4.04 0.827Task 22 - Provide assessment criteria 136 2 5 4.01 0.865Task 49 - Follow-up individually with learners who fail to participate in … 133 2 5 3.99 0.857Task 30 - Use strategies to create a friendly and open environment 140 1 5 3.97 0.881Task 9 - Maintain expertise in online instructional techniques … 142 1 5 3.86 0.912Task 39 - Share supplemental learning resources with the learners 144 1 5 3.51 0.893

Military Group A (n=12) n Min Max MeanStd. Dev

Task 1 - Review the course for accuracy 12 4 5 4.75 0.452 Task 2 - Make changes as needed to maintain course accuracy 12 4 5 4.42 0.515 Task 47 - Provide feedback to learners on how to correct errors 11 3 5 4.36 0.924 Task 8 - Maintain expertise in subject area 11 2 5 4.36 0.924 Task 17 - Maintain record-keeping related to course activities 12 3 5 3.92 0.900 Task 7 - Identify critical instructional areas, personalized or relevant … 11 2 5 3.73 0.905 Task 11 - Use appropriate technology to support instructional administration 11 2 5 3.55 0.820 Task 23 - Use strategies to gain and maintain learner attention 11 1 5 3.55 1.128 Task 53 - Solicit input from learners for course improvement 11 2 4 3.36 0.674 Task 9 - Maintain expertise in online instructional techniques … 11 2 4 3.36 0.809 Task 16 - Ensure all course activities comply with organizational rules … 11 1 4 3.27 0.905 Task 54 - Use evaluation data to recommend course revisions 11 1 4 3.18 0.873 Task 39 - Share supplemental learning resources with the learners 11 2 4 3.09 0.539

Military Group B (n=37) n Min Max MeanStd. Dev

Task 1 - Review the course for accuracy 36 2 5 4.64 0.683 Task 42 - Assess learners' attainment of learning objectives 36 2 5 4.58 0.770 Task 17 - Maintain record-keeping related to course activities 33 2 5 4.39 0.827 Task 2 - Make changes as needed to maintain course accuracy 36 1 5 4.36 0.867 Task 46 - Provide feedback on the accuracy of learners' statements or … 35 2 5 4.29 0.825


25

Task 33 - Use questions that promote higher order thinking 37 2 5 4.27 0.769 Task 32 - Use questions that promote discussion among learners 34 3 5 4.26 0.751 Task 47 - Provide feedback to learners on how to correct errors 35 2 5 4.23 0.843 Task 8 - Maintain expertise in subject area 37 2 5 4.22 0.854 Task 21 - Provide directions for assignments and other instructional … 33 3 5 4.12 0.650 Task 30 - Use strategies to create a friendly and open environment 36 2 5 4.08 0.841 Task 49 - Follow-up individually with learners who fail to participate … 34 3 5 4.03 0.870 Task 7 - Identify critical instructional areas, personalized or relevant … 33 2 5 4.00 0.791 Task 22 - Provide assessment criteria 36 2 5 3.94 0.791 Task 9 - Maintain expertise in online instructional techniques … 37 1 5 3.65 0.978 Task 43 - Provide opportunities for learners to self-assess their performance 34 2 5 3.65 0.849 Task 39 - Share supplemental learning resources with the learners 37 2 5 3.59 0.956 Task 36 - Require learners to comply with communication guidelines 33 2 5 3.52 0.940

Military Group C (n=49) n Min Max MeanStd Dev

Task 1 - Review the course for accuracy 48 2 5 4.67 0.630Task 42 - Assess learners' attainment of learning objectives 46 2 5 4.57 0.750Task 2 - Make changes as needed to maintain course accuracy 48 1 5 4.38 0.789Task 17 - Maintain record-keeping related to course activities 45 2 5 4.27 0.863Task 47 - Provide feedback to learners on how to correct errors 46 2 5 4.26 0.855Task 8 - Maintain expertise in subject area 48 2 5 4.25 0.863Task 46 - Provide feedback on the accuracy of learners’ statements … 45 2 5 4.24 0.802Task 33 - Use questions that promote higher order thinking 47 2 5 4.21 0.779Task 7 - Identify critical instructional areas, personalized or relevant … 44 2 5 3.93 0.818Task 22 - Provide assessment criteria 45 2 5 3.91 0.848Task 30 - Use strategies to create a friendly and open environment 46 2 5 3.89 0.875Task 9 - Maintain expertise in online instructional techniques … 48 1 5 3.58 0.942Task 39 - Share supplemental learning resources with the learners 48 2 5 3.48 0.899

Non-Military (n=96) n Min Max MeanStd. Dev

Task 42 - Assess learners' attainment of learning objectives 94 2 5 4.53 0.683Task 1 - Review the course for accuracy 93 3 5 4.53 0.653Task 8 - Maintain expertise in subject area 92 2 5 4.46 0.747Task 2 - Make changes as needed to maintain course accuracy 90 2 5 4.39 0.745Task 17 - Maintain record-keeping related to course activities 93 2 5 4.30 0.831Task 21 - Provide directions for assignments and other . . . activities 90 2 5 4.30 0.771Task 33 - Use questions that promote higher order thinking 88 2 5 4.26 0.750Task 46 - Provide feedback on the accuracy of learners' … 89 1 5 4.21 0.885Task 20 - Communicate learning goals and objectives for each … 90 2 5 4.21 0.855Task 35 - Model appropriate communication behavior 87 2 5 4.13 0.804Task 7 - Identify critical instructional areas, personalized or relevant … 88 2 5 4.08 0.834Task 22 - Provide assessment criteria 88 2 5 4.05 0.870Task 9 - Maintain expertise in online instructional techniques … 91 2 5 4.01 0.876Task 30 - Use strategies to create a friendly and open environment 91 1 5 4.00 0.894Task 32 - Use questions that promote discussion among learners 88 2 5 4.00 0.831Task 47 - Provide feedback to learners on how to correct errors 88 2 5 3.98 0.871Task 49 - Follow-up individually with learners who fail to participate … 87 2 5 3.94 0.868


26

Task 31 - Use strategies to solicit sharing of personal knowledge … 89 2 5 3.64 0.815Task 39 - Share supplemental learning resources with the learners 93 1 5 3.51 0.892

Time Spent Performing Tasks This section addresses the third research question: How do online instructors rate the relative time spent performing these tasks? If instructors answered, Yes to the question of whether they performed the task as part of their job as online instructors, they were asked to rate the task in terms of relative time spent performing the task using a five-point scale ranging from Much Less to Much More. Only Task 42 (Assess learners’ attainment of learning objectives) rated a mean of 4.00 (more) or above on the five-point scale of relative time spent performing the task across all groups. Military Group B instructors also rated Task 46 (Provide feedback on the accuracy of learners’ statements or performance) at a mean of 4.00. Overall, Task 42 (Assess learners’ attainment of learning objectives) rated as the most time-consuming task (mean of 4.22). More than half of the performed tasks had a mean of 3.00 (about the same) or below. The performed tasks that had the highest mean for time spent appears in Table 11 below. Data for all tasks performed by time mean are available in Appendix E. Table 11 Task rated “more” or “much more” on relative time spent scale

Overall (n=148) n Time Mean Std.

Deviation Task 42 – Assess learners’ attainment of learning objectives 143 4.22 0.865

Military Group A (n=12) Task 42 – Assess learners’ attainment of learning objectives 10 4.00 1.054

Military Group B (n=37) Task 42 – Assess learners’ attainment of learning objectives 36 4.31 0.786 Task 46 – Provide feedback on the accuracy of learners’ … 35 4.00 1.057

Military Group C (n=49) Task 42 – Assess learners’ attainment of learning objectives 46 4.24 0.848

Non-Military (n=96)

Task 42 – Assess learners’ attainment of learning objectives 94 4.19 0.883 It was found Task 42 (Assess learners’ attainment of learning objectives) appeared as the top task in the amount of time spent with a time mean of at least 4.19 across each group except Military Group A. This could be somewhat misleading as Task 42 had the highest time mean for time spent among the tasks performed by Military Group A with a mean of 4.00. However, less


27

than 90% of the instructors in Military Group A indicated they performed this task. Table 12 below shows the tasks 90% of instructors in each group indicated they perform based on time mean. Table 12 Tasks performed by 90% of instructors and ranked based on mean ratings of relative time spent

Overall (n=148) n Time Mean

Std. Deviation

Task 42 – Assess learners’ attainment of learning objectives 143 4.22 0.865 Task 46 – Provide feedback on the accuracy of learners’ … 137 3.85 0.946 Task 47 – Provide feedback to learners on how to correct errors 137 3.61 0.964 Task 33 – Use questions that promote higher order thinking 138 3.59 0.925 Task 21 – Provide directions for assignments and other … 136 3.46 0.934 Task 8 – Maintain expertise in subject areas 143 3.45 1.026 Task 7 – Identify critical instructional areas, personalized or relevant … 135 3.37 0.887 Task 32 – Use questions that promote discussion among learners 134 3.36 0.984 Task 30 – Use strategies to create a friendly and open environment 140 3.34 0.957 Task 17 – Maintain record-keeping related to course activities 141 3.33 1.005 Task 20 – Communicate learning goals and objectives … 135 3.08 0.962 Task 1 – Review the course for accuracy 144 3.01 1.031 Task 9 – Maintain expertise in online instructional techniques … 142 3.01 1.062 Task 49 – Follow-up individually with learners who fail to participate … 133 3.00 1.187 Task 22 – Provide assessment criteria 136 2.99 0.951 Task 2 – Make changes as need to maintain course accuracy 141 2.89 0.934 Task 39 - Share supplemental learning resources 144 2.88 0.960

Military Group A (n=12) n Time Mean

Std. Deviation

Task 47 – Provide feedback to learners on how to correct errors 11 3.91 0.831 Task 17 – Maintain record-keeping related to course activities 12 3.00 0.954 Task 8 – Maintain expertise in subject areas 11 2.91 0.944 Task 7 – Identify critical instructional areas … 11 2.91 0.701 Task 11 – Use appropriate technology to support instructional admin … 11 2.82 0.874 Task 23 – Use strategies to gain/maintain learner attention 11 2.73 0.905 Task 1 – Review the course for accuracy 12 2.67 0.651 Task 54 – Use evaluation data to recommend course improvements 11 2.64 0.809 Task 53 – Solicit input from learners for course improvements 11 2.64 0.809 Task 9 – Maintain expertise in online instructional techniques 11 2.45 0.820 Task 2 – Make changes as need to maintain course accuracy 12 2.42 0.515 Task 39 - Share supplemental learning resources 11 2.18 0.603 Task 16 – Ensure all course activities comply with organizational rules … 11 2.09 0.831

Military Group B (n=37) n Time Mean

Std. Deviation

Task 42 – Assess learners’ attainment of learning objectives 36 4.31 0.786 Task 46 – Provide feedback on the accuracy of learners’ statements or … 35 4.00 1.057


28

Task 47 – Provide feedback to learners on how to correct errors 35 3.97 0.985 Task 30 – Use strategies to create a friendly and open environment 36 3.86 0.961 Task 33 – Use questions that promote higher order thinking 37 3.76 0.983 Task 32 – Use questions that promote discussion among learners 34 3.74 0.963 Task 7 – Identify critical instructional areas, personalized or relevant … 33 3.45 0.869 Task 8 – Maintain expertise in subject areas 37 3.41 1.092 Task 17 – Maintain record-keeping related to course activities 33 3.39 1.029 Task 21 – Provide directions for assignments and other … activities 33 3.39 0.864 Task 49 – Follow-up individually with learners who fail to participate in … 34 3.29 1.060 Task 1 – Review the course for accuracy 36 3.06 1.094 Task 22 – Provide assessment criteria 36 3.06 0.893 Task 43 - Provide opportunities for learners to self-assess their performance 34 3.03 1.029 Task 36 - Require learners to comply with communication guidelines 33 3.00 1.000 Task 39 - Share supplemental learning resources with the learners 37 2.97 1.041 Task 9 – Maintain expertise in online instructional techniques … 37 2.97 1.142 Task 2 – Make changes as need to maintain course accuracy 36 2.92 1.079

Military Group C (n=49) n Time Mean

Std. Deviation

Task 42 – Assess learners’ attainment of learning objectives 46 4.24 0.848 Task 47 – Provide feedback to learners on how to correct errors 46 3.96 0.942 Task 46 – Provide feedback on the accuracy of learners’ statements or … 45 3.96 1.022 Task 33 – Use questions that promote higher order thinking 47 3.66 0.984 Task 30 – Use strategies to create a friendly and open environment 46 3.59 1.087 Task 7 – Identify critical instructional areas, personalized or relevant … 44 3.32 0.857 Task 8 – Maintain expertise in subject areas 48 3.29 1.071 Task 17 – Maintain record-keeping related to course … 45 3.29 1.014 Task 22 – Provide assessment criteria 45 3.00 0.954 Task 39 - Share supplemental learning resources 48 2.79 1.010 Task 2 – Make changes as need to maintain course accuracy 48 2.79 0.988

Non-Military (n=96) n Time Mean

Std. Deviation

Task 42 – Assess learners’ attainment of learning objectives 94 4.19 0.883 Task 46 – Provide feedback on the accuracy of learners’ statements or … 89 3.79 0.910 Task 8 – Maintain expertise in subject areas 92 3.55 0.999 Task 33 – Use questions that promote higher order thinking 88 3.53 0.896 Task 21 – Provide directions for assignments and other … activities 90 3.51 0.951 Task 47 – Provide feedback to learners on how to correct errors 88 3.40 0.917 Task 7 – Identify critical instructional areas, personalized or relevant … 88 3.39 0.903 Task 17 – Maintain record-keeping related to course activities 93 3.34 1.016 Task 32 – Use questions that promote discussion among learners 88 3.26 0.977 Task 35 – Model appropriate communication behavior 87 3.24 0.976 Task 30 – Use strategies to create a friendly and open environment 91 3.23 0.870 Task 9 – Maintain expertise in online instructional techniques … 91 3.08 1.035 Task 20 – Communicate learning goals and objectives for each … 90 3.07 0.909 Task 1 – Review the course for accuracy 93 3.06 1.030 Task 31 - Use strategies to solicit sharing of personal knowledge … 89 3.01 0.935


29

Task 22 – Provide assessment criteria 88 2.99 0.941 Task 39 - Share supplemental learning resources with the learners 93 2.94 0.930 Task 2 – Make changes as need to maintain course accuracy 90 2.91 0.882 Task 49 – Follow-up individually with learners who fail to participate … 87 2.86 1.222

Least Time Consuming Tasks

Overall, Task 37 (Use strategies to address undesirable interaction behavior) rated as the least time consuming task (mean of 2.07). This finding is somewhat understandable, as research has indicated the average online student is older in age than other students in higher education. This usually translates to more respect and acceptance of statements made by peers and instructors. It is especially true in the military where respect for authority and peers is tantamount to expected behavior. Therefore, it is not surprising that less than half of the Military Group A instructors indicated they performed this task and of those who did, the mean was 1.40 which is well below 2.00 (Less). Task 37 also rated the lowest mean (2.02) for time spent compared to other tasks in the Non-Military Group. Task 5 (Verify the existence and practicality of a back-up plan…) was the task Military Group B and Military Group C indicated took the least amount of time compared to other tasks. The task with the lowest time mean for each group is shown in Table 13 below. Table 13 List of least time consuming tasks based on relative time spent

Overall (n=148) n Time Mean

Std. Deviation

Task 37 - Use strategies to address undesirable interaction behavior 122 2.07 1.046

Military Group A (n=12) n Time Mean

Std. Deviation


Military Group B (n=37) n Time Mean

Std. Deviation

Task 5 - Verify the existence and practicality of a back-up plan … 10 2.30 1.160

Military Group C (n=49) n Time Mean

Std. Deviation

Task 5 - Verify the existence and practicality of a back-up plan … 16 2.19 1.047

Non-Military (n=96) n Time Mean

Std. Deviation



30

Conclusions

The following conclusions represent a synthesis of findings or commonalities found through the data analysis. This study was performed using a non-randomized convenience sample; therefore the results of this research might not generalize to a larger population. 1. It is anticipated that information from this study will be useful for developing online instructor training programs and for developing online instructor evaluations. 2. All tasks in the inventory were performed by some portion of online instructors and all tasks received an overall importance mean of at least moderate importance (across all instructors). Only two tasks were proposed as additions to the inventory, suggesting that the task listing conforms to common best practices for online instructors. 3. In this study, there were some differences in which tasks instructors performed as part of their jobs as online instructors and in how they rated the relative importance and time spent performing these tasks. These differences appeared to be associated with whether the instructors were teaching in a military or non-military environment. 4. Instructors in this study reported their courses were moderately interactive (44.5%) or highly interactive (37.6%) and that the typical taxonomic level of the learning outcomes for their online course was higher order skills (72.4%). This could vary significantly from the situation in beginning Navy online courses where the outcomes might be focused on comprehension outcomes versus higher order skills such as synthesis and evaluation. Differences in the amount of instructor interaction with students would also likely influence importance rankings for instructor tasks and relative time spent on those tasks. 5. The survey resulted in the identification of seventeen tasks as being performed by 90% of all instructors who completed the survey. • Task 1 – Review the course for accuracy (n=144) • Task 39 – Share supplemental learning resources with the learners (n=144) • Task 42 – Assess learners’ attainment of learning objectives (n=143) • Task 8 – Maintain expertise in subject areas (n=143) • Task 9 – Maintain expertise in online instructional techniques … (n=142) • Task 17 – Maintain record-keeping related to course activities (n=141) • Task 2 – Make changes as need to maintain course accuracy (n=141) • Task 30 – Use strategies to create a friendly and open environment (n=140) • Task 33 – Use questions that promote higher order thinking (n=138) • Task 46 – Provide feedback on the accuracy of learners’ statements … (n=137) • Task 47 – Provide feedback to learners on how to correct errors (n=137) • Task 21 – Provide directions for assignments and other instructional activities (n=136) • Task 22 – Provide assessment criteria (n=136) • Task 20 – Communicate learning goals and objectives … (n=135)


31

• Task 7 – Identify critical instructional areas, personalized or relevant examples… (n=135) • Task 32 – Use questions that promote discussion among learners (n=134) • Task 49 – Follow-up individually with learners who fail to participate … (n=133) 6. Ninety percent of instructors overall and across every sub-group indicated eight tasks were part of their job as an online instructor. • Task 1 – Review the course for accuracy • Task 2 – Make changes as need to maintain course accuracy • Task 7 – Identify critical instructional areas, personalized or relevant examples… • Task 8 – Maintain expertise in subject areas • Task 9 – Maintain expertise in online instructional techniques … • Task 17 – Maintain record-keeping related to course activities • Task 39 – Share supplemental learning resources with the learners • Task 47 – Provide feedback to learners on how to correct errors 7. A task performed by a large majority of instructors does not necessarily indicate the task’s importance level or the amount of time (relative to other instructor tasks) required to perform the task. 8. Suggestions for future studies: allowing participants to explain why they do not consider a task to be part of their job could clarify whether the task is performed by another individual in the organization, is not required for online instruction within the organization, or whether the instructor simply chooses not to perform the task.


32

References

Anderson, N., Ones, D., Sinangil, H., & Viswesvaran, C. (Eds.). (2001). Handbook of industrial, work and organizational psychology (Vol. 1). London: Sage Publications.

Bruskiewicz, K., Hanson, M., Kubisiak, U., Wright, C., Coovert, M., & Levine, E. (1996).

Florida Schoolyear 2000 job and task analysis: Guidance counselors, media specialists, and technical support personnel (No. 277). Tampa, FL: University of South Florida (USF), Psychology Personnel Decisions Research Institutes, Inc.

Cox, J., Rayner, G., & Branson, R. (1998). Interservice procedures for instructional systems

development, Phase I: Analyze. Tallahassee, FL: Florida State University, Center for Performance Technology.

International Board of Standards for Training Performance and Instruction. (2003). 2003

Instructor competencies. Retrieved June 2, 2003, from http://www.ibstpi.org/instructor_competencies.htm

Learning Systems Institute. (2003). Facilitator Competencies in an Integrated Learning

Environment. Tallahassee, FL: Florida State University McCormick, E. (1976). Job and task analysis. In Dunnette, M. (Ed.), Handbook Of Industrial

and Organizational Psychology (pp. 651-696). Chicago: Rand McNally. Rothwell, W., & Kazanas, H. (1992). Mastering the instructional design process: A systematic

approach. San Francisco, CA: Jossey-Bass Inc. Schoenfeld-Tacher, R., & Persichitte, K. A. (2000). Differential skills and competencies

required of faculty teaching distance education courses. International Journal of Educational Technology, 2 (1). Retrieved February 24, 2004, from http://www.ao.uiuc.edu/ijet/v2n1/index.html

Simon, J. (2002). Perceptions of web-based instruction experts regarding the roles, outputs, and

competencies needed in the field of web-based instruction: A model for training and development. Unpublished dissertation, Drake University, Des Moines, IA. Retrieved June 4, 2004, from ProQuest Digital Dissertations database.

Tello, S. (2002). An analysis of the relationship between instructional interaction and student

persistence in online education. Unpublished dissertation, University of Massachusetts Lowell, Lowell, Massachusetts. Retrieved September 13, 2004, from Proquest database.

Thach, L., & Murphy, K. (1995). Competencies for distance education professionals.

Educational Technology Research and Development, 43 (1), 57 - 79. Retrieved May 4, 2004, from Wilson Web database.


33

Williams, P. E. (2000). Defining distance education roles and competencies for higher education institutions: A computer-mediated Delphi study. Unpublished dissertation, Texas A&M University, College Station. Retrieved August 4, 2004, from ProQuest Digital Dissertations database.

Williams, P. E. (2003). Roles and competencies for distance education programs in higher

education institutions. The American Journal of Distance Education, 17 (1), 45-57

Learning Systems Institute: Copyright © 2005 Florida State University. All rights reserved 34

Appendix A: Online Instructor Duties and Tasks List Duty 1: Planning and Preparing for Instruction Task 1 - Review the course for accuracy (e.g., content accuracy, schedule/procedure updates, equipment updates, course syllabus accuracy and functionality). Task 2 - Make changes as needed to maintain course accuracy (e.g., content updates, schedule/procedure updates). Task 3 - Verify that contact information for student support is accurate (e.g., help desk, mentors, records access). Task 4 - Verify that any required equipment, tools, and instructional materials that are not part of the instructional website are available (e.g., manuals, textbooks). Task 5 - Verify the existence and practicality of a back-up plan for personnel or resources required for course delivery. Task 6 - Prepare for subject areas where learners may have questions or difficulty mastering the course content. Task 7 - Identify critical instructional areas, personalized or relevant examples, and additional materials. Task 8 - Maintain expertise in subject area. Task 9 - Maintain expertise in online instructional techniques/teaching strategies. Task 10 - Maintain expertise in course-related technologies. Duty 2: Implementing and Managing Instruction Task 11 - Use appropriate technology to support instructional administration (e.g., for communication, accessing and storing information). Task 12 - Maintain online security and privacy (e.g., passwords, email addresses, grades). Task 13 - Provide guidance to learners for obtaining necessary instructional materials, equipment, tools, and other required materials that are not part of the instructional website. Task 14 - Provide guidance to learners on using course-related technologies (e.g., create a technical problem discussion area for learners to assist each other with technical problems). Task 15 - Diagnose technical problems related to the course (e.g., log-in access, uploading/attaching files). Task 16 - Ensure all course activities comply with organizational rules, regulations, and policies (e.g., copyright restrictions, ethical standards, and security guidelines). Task 17 - Maintain record-keeping related to course activities (e.g., student attendance/participation, grades). Task 18 - Maintain scheduled hours for student-initiated contact.


Task 19 - Provide guidelines for communication protocol (e.g., netiquette guidelines, examples of appropriate communication, how and when to contact instructor, time frame for instructor feedback). Task 20 - Communicate learning goals and objectives for each instructional component. Task 21 - Provide directions for assignments and other instructional activities (e.g., high quality examples of similar assignments/activities/discussions, uploading/downloading assignments or activities). Task 22 - Provide assessment criteria (e.g., minimum requirements for course website login, discussion postings, late submissions policies). Task 23 - Use strategies to gain and maintain learner attention (e.g., invite virtual guests, arrange student debates, create discussions, and provide examples that relate to instruction and to application settings). Task 24 - Establish the relevance of the instruction to learning goals and/or job performance. Task 25 - Provide rewards and incentives for student participation (e.g., team competitions, recognitions). Task 26 - Guide learners to set personal learning goals. Task 27 - Guide learners to create action plans to achieve learning goals (e.g., identify learning resources and/or learning strategies). Task 28 - Guide learners' ongoing self-evaluation of their personal learning strategies. Task 29 - Use strategies to increase interaction among learners (e.g., group projects, group problem solving, learner presentations, peer critiques, icebreakers). Task 30 - Use strategies to create a friendly and open environment (e.g., initiate contact with individual learners, use personal stories and/or relevant personal examples during instruction). Task 31 - Use strategies to solicit sharing of personal knowledge and experience from all learners. Task 32 - Use questions that promote discussion among learners (e.g., open-ended and divergent questions). Task 33 - Use questions that promote higher order thinking (e.g., questions that require learners to compare, analyze, evaluate, or prioritize relevant information or concepts). Task 34 - Provide collaboration guidelines (e.g., assigned roles within a group, guidelines for resolving conflicts, guidelines for peer feedback). Task 35 - Model appropriate communication behavior (e.g., language, acknowledge diverse viewpoints, active listening). Task 36 - Require learners to comply with communication guidelines. Task 37 - Use strategies to address undesirable interaction behavior (e.g., intervene with students engaging in disruptive conversations, give specific suggestions for improvement). Task 38 - Use strategies to maintain the learners' focus on the learning activities. Task 39 - Share supplemental learning resources with the learners (e.g., links to relevant websites). Task 40 - Provide summation for learning activities and assignments (e.g., threaded discussions).


Task 41 - Use strategies to promote the transfer of learning to the job environment (e.g., realistic examples and practice activities, discussions on how to promote transfer or barriers to transfer). Duty 3: Assessing Learners Task 42 - Assess learners' attainment of learning objectives (e.g., exams, papers, projects, portfolios). Task 43 - Provide opportunities for learners to self-assess their performance. Task 44 - Monitor group discussions (e.g., quality and frequency of participation). Task 45 - Monitor learner attitudes (e.g., reactions to instructional activities). Task 46 - Provide feedback on the accuracy of learners’ statements or performances. Task 47 - Provide feedback to learners on how to correct errors. Task 48 - For group projects, provide feedback on the performance of each group as a whole, and on individual contributions toward the group’s goals. Task 49 - Follow-up individually with learners who fail to participate in course activities. Duty 4: Evaluating Course Task 50 - Document all evaluation data (e.g., instructional activities undertaken to support personal evaluation as an instructor). Task 51 - Provide instructor input for course improvement (e.g., instructional materials, learning activities, instructional technologies, support systems, equipment). Task 52 - Evaluate your performance as an online instructor. Task 53 - Solicit input from learners for course improvement (e.g., instructional materials, learning activities, instructional technologies, support systems, equipment). Task 54 - Use evaluation data to recommend course revisions (e.g., to accommodate learners needs, problems with the instructional design, course content, delivery systems, media, equipment).


37

Appendix B: Online Instructor Duty and Task Survey Thank you for completing this survey. The survey contains 54 task statements derived from literature, previous interviews, and surveys of online instructors. Design and development tasks usually associated with online courses are purposely not included in this survey. For each task, please indicate whether the task is part of your job as an online instructor, then click on the CONTINUE button. If the task is not part of your job, you will continue to the next task. If you perform the task, you will be asked to rate the task in terms of importance and comparative time spent performing the task. Please rate each task based on your personal experience within your particular online environment. We welcome and greatly appreciate any comments that might clarify your responses. For this survey, the word online is used to describe instruction delivered via the Internet. Please start the survey by clicking on the CONTINUE button below. --------------------------------------------------------------------------------------------------------------------- Task 1 - Review the course for accuracy (e.g., content accuracy, schedule/procedure updates, equipment updates, course syllabus accuracy and functionality).

Is this task part of your job as an online instructor? *

Yes No

Task - Review the course for accuracy (e.g., content accuracy, schedule/procedure updates, equipment updates, course syllabus accuracy and functionality).

Very high High Moderate Low Very low The importance of performing this task in your job as an online instructor is: *

Much more More About the same

Less Much less

Compared to other tasks you do in your job as an online instructor, the time you spend on this task is: *

Comments:


38

Task 2 - Make changes as needed to maintain course accuracy (e.g., content updates, schedule/procedure updates).


Yes No Task - Make changes as needed to maintain course accuracy (e.g., content updates, schedule/procedure updates).



Less Much less


Comments: Task 3 - Verify that contact information for student support is accurate (e.g., help desk, mentors, records access).


Yes No

Task - Verify that contact information for student support is accurate (e.g., help desk, mentors, records access).



Less Much less


Comments:


39

Task 4 - Verify that any required equipment, tools, and instructional materials that are not part of the instructional website are available (e.g., manuals, textbooks).


Yes No

Task - Verify that any required equipment, tools, and instructional materials that are not part of the instructional website are available (e.g., manuals, textbooks).



Less Much less


Comments: Task 5 - Verify the existence and practicality of a back-up plan for personnel or resources required for course delivery.


Yes No

Task - Verify the existence and practicality of a back-up plan for personnel or resources required for course delivery.



Less Much less


Comments:


40

Task 6 - Prepare for subject areas where learners may have questions or difficulty mastering the course content.


Yes No

Task - Prepare for subject areas where learners may have questions or difficulty mastering the course content.



Less Much less


Comments: Task 7 - Identify critical instructional areas, personalized or relevant examples, and additional materials.


Yes No

Task - Identify critical instructional points, personalized or relevant examples, and additional materials.



Less Much less


Comments:


41

Task 8 - Maintain expertise in subject area.


Yes No

Task - Maintain expertise in subject area. Very high High Moderate Low Very low The importance of performing this task in your job as an online instructor is: *


Less Much less


Comments: Task 9 - Maintain expertise in online instructional techniques/teaching strategies.


Yes No

Task - Maintain expertise in online instructional techniques/teaching strategies.



Less Much less


Comments:


42

Task 10 - Maintain expertise in course-related technologies.


Yes No

Task - Maintain expertise in course-related technologies.



Less Much less


Comments: Task 11 - Use appropriate technology to support instructional administration (e.g., for communication, accessing and storing information).


Yes No

Task - Use appropriate technology to support instructional administration (e.g., for communication, accessing and storing information).



Less Much less


Comments:


43

Task 12 - Maintain online security and privacy (e.g., passwords, email addresses, grades).


Yes No

Task - Maintain online security and privacy (e.g., passwords, email addresses, grades).



Less Much less


Comments: Task 13 - Provide guidance to learners for obtaining necessary instructional materials, equipment, tools, and other required materials that are not part of the instructional website.


Yes No

Task - Provide guidance to learners for obtaining necessary instructional materials, equipment, tools, and other required materials that are not part of the instructional website.



Less Much less


Comments:


44

Task 14 - Provide guidance to learners on using course-related technologies (e.g., create a technical problem discussion area for learners to assist each other with technical problems).


Yes No

Task - Provide guidance to learners for using course-related technologies (e.g., create a technical problem discussion area for learners to assist each other with technical problems).



Less Much less


Comments: Task 15 - Diagnose technical problems related to the course (e.g., log-in access, uploading/attaching files).


Yes No

Task - Diagnose technical problems related to the course (e.g., log-in access, uploading/attaching files).



Less Much less


Comments:


45

Task 16 - Ensure all course activities comply with organizational rules, regulations, and policies (e.g., copyright restrictions, ethical standards, and security guidelines).


Yes No

Task - Ensure all course activities comply with organizational rules, regulations, and policies (e.g., copyright restrictions, ethical standards, and security guidelines).



Less Much less


Comments: Task 17 - Maintain record-keeping related to course activities (e.g., student attendance/participation, grades).


Yes No

Task - Maintain record-keeping related to course activities (e.g., student attendance/participation, grades).



Less Much less


Comments:


46

Task 18 - Maintain scheduled hours for student-initiated contact.


Yes No

Task - Maintain scheduled hours for student-initiated contact.



Less Much less


Comments: Task 19 - Provide guidelines for communication protocol (e.g., netiquette guidelines, examples of appropriate communication, how and when to contact instructor, time frame for instructor feedback).


Yes No

Task - Provide guidelines for communication protocol (e.g., netiquette guidelines, examples of appropriate communication, how and when to contact instructor, time frame for instructor feedback).



Less Much less


Comments:


47

Task 20 - Communicate learning goals and objectives for each instructional component.


Yes No

Task - Communicate learning goals and objectives for each instructional component.



Less Much less


Comments: Task 21 - Provide directions for assignments and other instructional activities (e.g., high quality examples of similar assignments/activities/discussions, uploading/downloading assignments or activities).


Yes No

Task - Provide directions for assignments and other instructional activities (e.g., high quality examples of similar assignments/activities/discussions, uploading/downloading assignments or activities).



Less Much less


Comments:


48

Task 22 - Provide assessment criteria (e.g., minimum requirements for course website login, discussion postings, and late-submissions policies).


Yes No

Task - Provide assessment criteria (e.g., minimum requirements for course website login, criteria for discussion postings, and late-submissions policies).



Less Much less


Comments: Task 23 - Use strategies to gain and maintain learner attention (e.g., invite virtual guests, arrange student debates, create discussions, provide examples that relate to instruction and to application settings).


Yes No

Task - Use strategies to gain and maintain learner attention (e.g., invite virtual guests, arrange student debates, create discussions, provide examples that relate to instruction and to application settings).



Less Much less


Comments:


49

Task 24 - Establish the relevance of the instruction to learning goals and/or job performance.


Yes No

Task - Establish the relevance of the instruction to learning goals and/or job performance.



Less Much less


Comments: Task 25 - Provide rewards and incentives for student participation (e.g., team competitions, recognitions).


Yes No

Task - Provide rewards and incentives for student participation (e.g., team competitions, recognitions).



Less Much less


Comments:


50

Task 26 - Guide learners to set personal learning goals.


Yes No

Task - Guide learners to set personal learning goals.



Less Much less


Comments: Task 27 - Guide learners to create action plans to achieve learning goals (e.g., identify learning resources and/or learning strategies).


Yes No

Task - Guide learners to create action plans to achieve learning goals (e.g., identify learning resources and/or learning strategies).



Less Much less


Comments:


51

Task 28 - Guide learners' ongoing self-evaluation of their personal learning strategies.


Yes No

Task - Guide learners' ongoing self-evaluation of their personal learning strategies.



Less Much less


Comments: Task 29 - Use strategies to increase interaction among learners (e.g., group projects, group problem solving, learner presentations, peer critiques, icebreakers).


Yes No

Task - Use strategies to increase interaction among learners (e.g., group projects, group problem solving, learner presentations, peer critiques, icebreakers).



Less Much less


Comments:


52

Task 30 - Use strategies to create a friendly and open environment (e.g., initiate contact with individual learners, use personal stories and/or relevant personal examples during instruction).


Yes No

Task - Use strategies to create a friendly and open environment (e.g., initiate contact with individual learners, use personal stories and/or relevant personal examples during instruction).



Less Much less


Comments: Task 31 - Use strategies to solicit sharing of personal knowledge and experience from all learners.


Yes No

Task - Use strategies to solicit sharing of personal knowledge and experience from all learners.



Less Much less


Comments:


53

Task 32 - Use questions that promote discussion among learners (e.g., open-ended and divergent questions).


Yes No

Task - Use questions that promote discussion among learners (e.g., open-ended and divergent questions).



Less Much less


Comments: Task 33 - Use questions that promote higher order thinking (e.g., questions that require learners to compare, analyze, evaluate, or prioritize relevant information or concepts).


Yes No

Task - Use questions that promote higher order thinking (e.g., questions that require learners to compare, analyze, evaluate, or prioritize relevant information or concepts).



Less Much less


Comments:


54

Task 34 - Provide collaboration guidelines (e.g., assigned roles within a group, guidelines for resolving conflicts, guidelines for peer feedback).


Yes No

Task - Provide collaboration guidelines (e.g., assigned roles within a group, guidelines for resolving conflicts, guidelines for peer feedback).



Less Much less


Comments: Task 35 - Model appropriate communication behavior (e.g., language, acknowledge diverse viewpoints, active listening).


Yes No

Task - Model appropriate communication behavior (e.g., language, acknowledge diverse viewpoints, active listening).



Less Much less


Comments:


55

Task 36 - Require learners to comply with communication guidelines.


Yes No

Task - Require learners to comply with communication guidelines.



Less Much less


Comments: Task 37 - Use strategies to address undesirable interaction behavior (e.g., intervene with students engaging in disruptive conversations, give specific suggestions for improvement).


Yes No

Task - Use strategies to address undesirable interaction behavior (e.g., intervene with students engaging in disruptive conversations; give specific suggestions for improvement).



Less Much less


Comments:


56

Task 38 - Use strategies to maintain the learners' focus on the learning activities.


Yes No

Task - Use strategies to maintain the learners' focus on the learning activities.



Less Much less


Comments: Task 39 - Share supplemental learning resources with the learners (e.g., links to relevant websites).


Yes No

Task - Share supplemental learning resources with the learners (e.g., links to relevant websites).



Less Much less


Comments:


57

Task 40 - Provide summation for learning activities and assignments (e.g., threaded discussions).


Yes No

Task - Provide summation for learning activities and assignments (e.g., threaded discussions).



Less Much less


Comments: Task 41 - Use strategies to promote the transfer of learning to the job environment (e.g., realistic examples and practice activities, discussions on how to promote transfer or barriers to transfer).


Yes No

Task - Use strategies to promote transfer of learning to the job environment (e.g., realistic examples and practice activities, discussions on how to promote transfer or barriers to transfer).



Less Much less


Comments:


58

Task 42 - Assess learners' attainment of learning objectives (e.g., exams, papers, projects, portfolios).


Yes No

Task - Assess learners' attainment of learning objectives (e.g., exams, papers, projects, portfolios).



Less Much less


Comments: Task 43 - Provide opportunities for learners to self-assess their performance.


Yes No

Task - Provide opportunities for learners to self-assess their performance.



Less Much less


Comments:


59

Task 44 - Monitor group discussions (e.g., quality and frequency of participation).


Yes No

Task - Monitor group discussions (e.g., monitor quality and frequency of participation).



Less Much less


Comments: Task 45 - Monitor learner attitudes (e.g., reactions to instructional activities).


Yes No

Task - Monitor learner attitudes (e.g., reactions to instructional activities).



Less Much less


Comments:


60

Task 46 - Provide feedback on the accuracy of learners’ statements or performances.


Yes No

Task - Provide feedback on the accuracy of learners’ statements or performances.



Less Much less


Comments: Task 47 - Provide feedback to learners on how to correct errors.


Yes No

Task - Provide feedback to learners on how to correct errors.



Less Much less


Comments:


61

Task 48 - For group projects, provide feedback on the performance of each group as a whole, and on individual contributions toward the group’s goals.


Yes No

Task - For group projects, provide feedback on the performance of each group as a whole, and on individual contributions toward the group’s goals.



Less Much less


Comments: Task 49 - Follow-up individually with learners who fail to participate in course activities.


Yes No

Task - Follow-up individually with learners who fail to participate in course activities.



Less Much less


Comments:


62

Task 50 - Document all evaluation data (e.g., instructional activities undertaken to support personal evaluation as an instructor).


Yes No

Task - Document all evaluation data (e.g., instructional activities undertaken to support personal evaluation as an instructor).



Less Much less


Comments: Task 51 - Provide instructor input for course improvement (e.g., instructional materials, learning activities, instructional technologies, support systems, equipment).


Yes No

Task - Provide instructor input for course improvement (e.g., instructional materials, learning activities, instructional technologies, support systems, equipment).



Less Much less


Comments:


63

Task 52 - Evaluate your performance as an online instructor.


Yes No

Task - Evaluate your performance as an online instructor.



Less Much less


Comments: Task 53 - Solicit input from learners for course improvement (e.g., instructional materials, learning activities, instructional technologies, support systems, equipment).


Yes No

Task - Solicit input from learners for course improvement (e.g., instructional materials, learning activities, instructional technologies, support systems, equipment).



Less Much less


Comments:


64

Task 54 - Use evaluation data to recommend course revisions (e.g., to accommodate learners needs, problems with the instructional design, course content, delivery systems, media, equipment).


Yes No

Task - Use evaluation data to recommend course revisions (e.g., to accommodate learners needs, problems with the instructional design, course content, delivery systems, media, equipment).



Less Much less


Comments: --------------------------------------------------------------------------------------------------------------------- We welcome and greatly appreciate your input on any task(s) not related to course design or development you think should have been included in the survey. For each addition, please provide a description of the task, your importance rating, and comparative time estimate using the scale provided in the survey. Comments: ---------------------------------------------------------------------------------------------------------------------


65

These final questions are for demographic purposes and will only be used for data analysis. 1. What is your role in the delivery of online courses? (Please check all that apply).

Administrator

Instructor

Instructional designer and/or developer

Subject matter expert

Technology expert

Other 2. Do you have experience as an online student or trainee?

Yes

No 3. Please rate your level of expertise in using technology for instruction.

Very low

Low

Moderate

High

Very high 4. Are you currently teaching online courses in a military setting? (Please do not include blended or hybrid courses)

Yes

No 5. Which of the following is a delivery mode for your online courses?

Asynchronous only

Synchronous only

Both (asynchronous and synchronous)


66

6. What type of education or training have you completed that prepared you to be an instructor? (Please check all that apply).

College or university courses

Training courses or workshops

Instructional certificate programs

On-the-job training or mentoring

None 7. What type of education or training have you completed that prepared you to be an online course instructor? (Please check all that apply)

College or university courses

Training courses or workshops

Instructional certificate programs

On-the-job training or mentoring

None 8. Please select the range that best represents your years of teaching experience in any environment (classroom or distance).

Less than a year

1 to 5

6 to 10

11 to 15

15 to 20

Over 20

None


67

9. Please select the range that best represents your years of teaching experience in an online environment. (Please do not include blended or hybrid courses)

Less than a year

1 to 3

4 to 6

7 to 9

Over 10

None 10. Please select the range that best represents the number of online courses you have taught. (Please do not include blended or hybrid courses).

1 to 5

6 to 10

11 to 15

16 to 20

21 to 25

More than 25

None 11. Please select the range that best represents the number of different online courses you have taught (Please do not count iterations of the same course).

1 to 2

3 to 4

5 to 6

More than 6


68

12. Please select the category that best represents the level of interaction among students and between you and your students during your typical online course. [Adapted from Bedore and Goins (2001) Online instructional strategies: How many students fit in an online course?]

Highly Interactive. Collaborative learning environment where content is applied by using a diversity of student inputs, discussion, and other learner-centered activities (e.g., peer critiques, consensus-building). The level of discussion is high and assessments may be open-ended requiring personalized instructor feedback.

Moderately Interactive. Instructional activities include small group projects and a moderate level of discussion and instructor feedback. Some independent study and assignments may be used. Student groups may discuss issues and report back to the larger group; discussion questions may be assigned to control the number of postings.

Minimally Interactive. The majority of activities include presentation of content and students working independently with limited contact with instructor. Feedback may be pre-programmed. Interaction between instructor and students and among students is minimal, with very little or no online discussion.

Other 13. Please select the category that best represents the taxonomic level of the learning outcomes for your typical online course.

Higher order learning goals (i.e., evaluation, synthesis, analysis, application).

Lower order learning goals (i.e., recall of information, comprehension, basic concepts).

Other 14. Please select the range that best represents the length of your typical online course.

Less than a month

1 to 2 months

3 to 4 months

5 to 6 months

More than 6 months 15. Please select the range that best represents the number of students in your typical online course.

1 to 10

11 to 20

21 to 30

More than 30


69

16. Please select the range that best represents the age of the majority of your online students.

18 to 25

26 to 33

34 to 41

42 to 49

Over 50

Don't know/can't answer 17. What is your gender?

Male

Female 18. Please select the range that best represents your age.

18 to 25

26 to 33

34 to 41

42 to 49

Over 50 19. What is the highest level of education you have completed?

High school

Some college courses

Associate Degree or Technical Certificate

Bachelor's Degree

Master's Degree

Ph.D.

Other


70

We welcome and greatly appreciate any comments you may have regarding the survey. Comments: Please enter your email address if you would like a copy of the final report. ------------------------------------------------------- End of Survey


71

Appendix C: List of Tasks in Performance Order Overall (n=148) n

Imp. Mean

Time Mean

Military A (n=12) n

Imp. Mean

Time Mean

Military B (n=37) n

Imp. Mean

Time Mean

Military C (n=49) n

Imp. Mean

Time Mean

Non-Military (n=96) n

Imp. Mean

Time Mean

Task 1 144 4.58 3.01 Task 1 12 4.75 2.67 Task 33 37 4.27 3.76 Task 1 48 4.67 2.96 Task 42 94 4.53 4.19 Task 39 144 3.51 2.88 Task 2 12 4.42 2.42 Task 8 37 4.22 3.41 Task 2 48 4.38 2.79 Task 1 93 4.53 3.06 Task 42 143 4.55 4.22 Task 17 12 3.92 3.00 Task 9 37 3.65 2.97 Task 8 48 4.25 3.29 Task 17 93 4.30 3.34 Task 8 143 4.38 3.45 Task 47 11 4.36 3.91 Task 39 37 3.59 2.97 Task 9 48 3.58 2.85 Task 39 93 3.51 2.94 Task 9 142 3.86 3.01 Task 8 11 4.36 2.91 Task 1 36 4.64 3.06 Task 39 48 3.48 2.79 Task 8 92 4.46 3.55 Task 17 141 4.40 2.89 Task 7 11 3.73 2.91 Task 42 36 4.58 4.31 Task 33 47 4.21 3.66 Task 9 91 4.01 3.08 Task 2 141 4.31 3.33 Task 11 11 3.55 2.82 Task 2 36 4.36 2.92 Task 42 46 4.57 4.24 Task 30 91 4.00 3.23 Task 30 140 3.97 3.34 Task 23 11 3.55 2.73 Task 30 36 4.08 3.86 Task 47 46 4.26 3.96 Task 2 90 4.39 2.91 Task 33 138 4.26 3.59 Task 53 11 3.36 2.64 Task 22 36 3.94 3.06 Task 30 46 3.89 3.59 Task 21 90 4.30 3.51 Task 46 137 4.24 3.85 Task 9 11 3.36 2.45 Task 46 35 4.29 4.00 Task 17 45 4.27 3.29 Task 20 90 4.21 3.07 Task 47 137 4.09 3.61 Task 16 11 3.27 2.09 Task 47 35 4.23 3.97 Task 46 45 4.24 3.96 Task 46 89 4.21 3.79 Task 21 136 4.25 3.46 Task 54 11 3.18 2.64 Task 32 34 4.26 3.74 Task 22 45 3.91 3.00 Task 31 89 3.64 3.01 Task 22 136 4.01 2.99 Task 39 11 3.09 2.18 Task 49 34 4.03 3.29 Task 7 44 3.93 3.32 Task 33 88 4.26 3.53 Task 20 135 4.13 3.08 Task 42 10 4.50 4.00 Task 43 34 3.65 3.03 Task 21 43 4.14 3.35 Task 7 88 4.08 3.39 Task 7 135 4.04 3.37 Task 21 10 4.20 3.20 Task 17 33 4.39 3.39 Task 32 43 4.12 3.53 Task 22 88 4.05 2.99 Task 32 134 4.05 3.36 Task 46 10 4.10 3.80 Task 21 33 4.12 3.39 Task 49 43 4.05 3.28 Task 32 88 4.00 3.26 Task 49 133 3.99 3.00 Task 33 10 4.00 3.30 Task 7 33 4.00 3.45 Task 20 42 3.93 3.14 Task 47 88 3.98 3.40 Task 31 132 3.64 3.02 Task 44 10 3.80 3.40 Task 36 33 3.52 3.00 Task 6 41 3.80 3.00 Task 35 87 4.13 3.24 Task 6 130 4.02 3.16 Task 20 10 3.50 2.70 Task 20 32 4.06 3.28 Task 24 41 3.68 3.00 Task 49 87 3.94 2.86 Task 23 128 3.96 3.16 Task 10 10 3.40 2.50 Task 6 32 4.00 3.09 Task 43 41 3.56 2.93 Task 6 86 4.13 3.24 Task 10 128 3.61 2.70 Task 30 10 3.20 2.60 Task 38 32 3.84 3.31 Task 16 41 3.46 2.49 Task 23 85 4.02 3.21 Task 44 126 3.95 3.67 Task 51 9 4.22 3.67 Task 31 32 3.72 3.25 Task 44 40 3.90 3.78 Task 51 85 3.65 2.72 Task 24 126 3.87 3.04 Task 49 9 4.11 3.22 Task 24 32 3.72 3.09 Task 23 40 3.78 3.08 Task 37 85 3.58 2.02 Task 51 126 3.71 2.83 Task 50 9 3.89 3.33 Task 44 30 3.93 3.90 Task 10 40 3.65 2.80 Task 10 85 3.56 2.62 Task 53 125 3.67 2.54 Task 22 9 3.78 2.78 Task 10 30 3.73 2.90 Task 31 40 3.60 3.05 Task 36 84 3.60 2.65 Task 36 125 3.58 2.70 Task 40 9 3.67 3.11 Task 16 30 3.53 2.63 Task 38 39 3.79 3.18 Task 44 83 3.94 3.60 Task 35 122 4.13 3.34 Task 32 9 3.56 2.78 Task 50 29 3.93 3.52 Task 53 39 3.72 2.69 Task 53 83 3.61 2.48 Task 37 122 3.53 2.07 Task 24 9 3.56 2.67 Task 23 29 3.86 3.21 Task 50 38 3.92 3.47 Task 24 82 3.94 3.07 Task 38 121 3.81 2.93 Task 6 9 3.11 2.67 Task 41 29 3.76 3.10 Task 51 38 3.84 3.11 Task 29 81 3.81 3.25


72

Overall (n=148) n Mean

Std. Dev

Military A (n=12) n Mean

Std. Dev

Military B (n=37) n Mean

Std. Dev

Military C (n=49) n Mean

Std. Dev

Non-Military (n=96) n Mean

Std. Dev

Task 40 121 3.72 3.28 Task 19 9 3.11 2.44 Task 51 29 3.72 2.93 Task 11 38 3.71 3.08 Task 40 81 3.69 3.20 Task 29 120 3.77 3.18 Task 31 8 3.13 2.25 Task 29 29 3.59 3.07 Task 36 38 3.50 2.87 Task 38 79 3.81 2.80 Task 43 120 3.64 2.87 Task 45 8 3.00 2.75 Task 19 29 3.55 2.83 Task 19 38 3.45 2.74 Task 50 78 3.87 3.21 Task 50 119 3.91 3.29 Task 14 8 2.63 1.50 Task 37 29 3.45 2.34 Task 40 37 3.76 3.46 Task 19 78 3.64 2.83 Task 19 119 3.59 2.80 Task 38 7 3.57 2.57 Task 45 29 3.24 2.79 Task 45 37 3.19 2.78 Task 43 76 3.64 2.82 Task 54 114 3.76 2.78 Task 29 7 3.57 2.86 Task 35 28 4.18 3.86 Task 54 36 3.72 3.06 Task 54 75 3.77 2.68 Task 16 114 3.59 2.43 Task 43 7 3.14 2.43 Task 53 28 3.86 2.71 Task 29 36 3.58 3.03 Task 45 72 3.47 2.60 Task 45 112 3.40 2.65 Task 41 6 3.83 3.33 Task 40 28 3.79 3.57 Task 41 35 3.77 3.14 Task 41 71 3.82 3.10 Task 11 111 3.68 2.94 Task 13 6 3.83 2.83 Task 11 27 3.78 3.19 Task 37 34 3.32 2.21 Task 16 71 3.63 2.41 Task 41 108 3.81 3.12 Task 5 6 3.50 2.00 Task 54 25 3.96 3.24 Task 35 32 4.06 3.63 Task 11 71 3.63 2.82 Task 13 97 3.47 2.71 Task 52 6 3.00 2.33 Task 52 25 3.92 2.72 Task 52 31 3.74 2.65 Task 18 65 3.77 3.14 Task 52 92 3.88 2.57 Task 18 5 4.00 2.80 Task 13 25 3.64 2.76 Task 13 31 3.68 2.77 Task 13 64 3.39 2.69 Task 18 90 3.77 3.16 Task 3 5 4.00 3.00 Task 27 23 3.26 2.78 Task 27 27 3.30 2.70 Task 48 62 3.74 3.27 Task 48 87 3.72 3.21 Task 4 5 3.80 2.20 Task 12 22 4.50 2.73 Task 12 26 4.42 2.65 Task 52 60 3.95 2.55 Task 34 84 3.44 2.75 Task 36 5 3.40 2.00 Task 34 21 3.10 2.52 Task 18 24 3.79 3.21 Task 34 57 3.58 2.84 Task 27 83 3.42 2.77 Task 26 5 3.00 2.20 Task 28 20 3.55 3.30 Task 3 24 3.50 2.54 Task 27 55 3.45 2.82 Task 26 80 3.39 2.74 Task 37 5 2.60 1.40 Task 18 19 3.74 3.32 Task 34 24 3.08 2.50 Task 26 55 3.35 2.73 Task 14 73 3.25 2.55 Task 12 4 4.00 2.25 Task 48 19 3.53 2.95 Task 4 23 3.65 2.30 Task 14 48 3.40 2.71 Task 12 72 4.24 2.65 Task 27 4 3.50 2.25 Task 3 19 3.37 2.42 Task 14 23 2.96 2.26 Task 12 46 4.13 2.65 Task 3 72 3.51 2.31 Task 35 4 3.25 2.00 Task 4 18 3.61 2.33 Task 48 22 3.59 2.95 Task 3 46 3.50 2.17 Task 4 70 3.80 2.21 Task 48 3 4.00 3.00 Task 26 17 3.53 3.00 Task 26 22 3.41 2.82 Task 4 45 3.87 2.13 Task 28 66 3.55 2.91 Task 34 3 3.00 2.33 Task 14 15 3.13 2.67 Task 28 21 3.52 3.24 Task 28 44 3.52 2.77 Task 25 62 3.08 2.45 Task 15 3 3.00 2.33 Task 25 15 3.00 2.73 Task 25 18 2.94 2.56 Task 25 41 3.10 2.39 Task 15 51 3.43 2.73 Task 25 3 2.67 1.67 Task 15 13 3.62 3.38 Task 5 16 3.56 2.19 Task 15 35 3.40 2.51 Task 5 40 3.38 2.10 Task 28 1 3.00 2.00 Task 5 10 3.60 2.30 Task 15 16 3.50 3.19 Task 5 24 3.25 2.04


73

Appendix D: List of Tasks by Importance Mean


Std. Dev


Std. Dev


Std. Dev


Std. Dev


Std. Dev

Task 1 144 4.58 0.642 Task 1 12 4.75 0.452 Task 1 36 4.64 0.683 Task 1 48 4.67 0.630 Task 42 94 4.53 0.683Task 42 143 4.55 0.699 Task 42 10 4.50 0.707 Task 42 36 4.58 0.770 Task 42 46 4.57 0.750 Task 1 93 4.53 0.653Task 17 141 4.40 0.755 Task 2 12 4.42 0.515 Task 12 22 4.50 0.802 Task 12 26 4.42 0.902 Task 8 92 4.46 0.747Task 8 143 4.38 0.787 Task 47 11 4.36 0.924 Task 17 33 4.39 0.827 Task 2 48 4.38 0.789 Task 2 90 4.39 0.745Task 2 141 4.31 0.836 Task 8 11 4.36 0.924 Task 2 36 4.36 0.867 Task 17 45 4.27 0.863 Task 17 93 4.30 0.831Task 33 138 4.26 0.757 Task 51 9 4.22 0.667 Task 46 35 4.29 0.825 Task 47 46 4.26 0.855 Task 21 90 4.30 0.771Task 21 136 4.25 0.748 Task 21 10 4.20 0.919 Task 33 37 4.27 0.769 Task 8 48 4.25 0.863 Task 33 88 4.26 0.750Task 46 137 4.24 0.853 Task 49 9 4.11 0.782 Task 32 34 4.26 0.751 Task 46 45 4.24 0.802 Task 46 89 4.21 0.885Task 12 72 4.24 1.041 Task 46 10 4.10 0.738 Task 47 35 4.23 0.843 Task 33 47 4.21 0.778 Task 20 90 4.21 0.855Task 20 135 4.13 0.871 Task 33 10 4.00 0.000 Task 8 37 4.22 0.854 Task 21 43 4.14 0.710 Task 12 46 4.13 1.108Task 35 122 4.13 0.802 Task 18 5 4.00 0.817 Task 35 28 4.18 0.772 Task 32 43 4.12 0.823 Task 6 86 4.13 0.794Task 47 137 4.09 0.873 Task 3 5 4.00 1.414 Task 21 33 4.12 0.650 Task 35 32 4.06 0.801 Task 35 87 4.13 0.804Task 32 134 4.05 0.826 Task 12 4 4.00 1.414 Task 30 36 4.08 0.841 Task 49 43 4.05 0.844 Task 7 88 4.08 0.834Task 7 135 4.04 0.827 Task 48 3 4.00 0.707 Task 20 32 4.06 0.716 Task 7 44 3.93 0.818 Task 22 88 4.05 0.870Task 6 130 4.02 0.876 Task 17 12 3.92 0.900 Task 49 34 4.03 0.870 Task 20 42 3.93 0.894 Task 23 85 4.02 0.886Task 22 136 4.01 0.865 Task 50 9 3.89 0.782 Task 7 33 4.00 0.791 Task 50 38 3.92 0.912 Task 9 91 4.01 0.876Task 49 133 3.99 0.857 Task 41 6 3.83 0.983 Task 6 32 4.00 0.950 Task 22 45 3.91 0.848 Task 30 91 4.00 0.894Task 30 140 3.97 0.881 Task 13 6 3.83 0.753 Task 54 25 3.96 0.790 Task 44 40 3.90 0.810 Task 32 88 4.00 0.830Task 23 128 3.96 0.891 Task 44 10 3.80 0.789 Task 22 36 3.94 0.791 Task 30 46 3.89 0.875 Task 47 88 3.98 0.871Task 44 126 3.95 0.920 Task 4 5 3.80 0.837 Task 44 30 3.93 0.828 Task 51 38 3.84 0.823 Task 52 60 3.95 0.832Task 50 119 3.91 0.863 Task 22 9 3.78 1.093 Task 50 29 3.93 0.961 Task 6 41 3.80 1.005 Task 49 87 3.94 0.867Task 52 92 3.88 0.970 Task 7 11 3.73 0.905 Task 52 25 3.92 1.077 Task 38 39 3.79 0.833 Task 44 83 3.94 0.967Task 24 126 3.87 0.933 Task 40 9 3.67 0.707 Task 23 29 3.86 0.789 Task 18 24 3.79 0.884 Task 24 82 3.94 0.866Task 9 142 3.86 0.912 Task 38 7 3.57 0.787 Task 53 28 3.86 0.891 Task 23 40 3.78 0.891 Task 50 78 3.87 0.843Task 41 108 3.81 0.855 Task 29 7 3.57 0.976 Task 38 32 3.84 0.847 Task 41 35 3.77 0.877 Task 4 45 3.87 1.236Task 38 121 3.81 0.820 Task 32 9 3.56 0.882 Task 40 28 3.79 0.833 Task 40 37 3.76 0.796 Task 41 71 3.82 0.850Task 4 70 3.80 1.199 Task 24 9 3.56 1.333 Task 11 27 3.78 0.934 Task 52 31 3.74 1.210 Task 29 81 3.81 0.910Task 29 120 3.77 0.923 Task 11 11 3.55 0.820 Task 41 29 3.76 0.872 Task 54 36 3.72 0.882 Task 38 79 3.81 0.818


74


Std. Dev


Std. Dev


Std. Dev


Std. Dev


Std. Dev

Task 18 90 3.77 0.887 Task 23 11 3.55 1.128 Task 18 19 3.74 0.733 Task 53 39 3.72 0.857 Task 54 75 3.77 0.909Task 54 114 3.76 0.886 Task 20 10 3.50 1.000 Task 10 30 3.73 0.907 Task 11 38 3.71 0.898 Task 18 65 3.77 0.897Task 48 87 3.72 0.898 Task 5 6 3.50 1.269 Task 51 29 3.72 0.841 Task 24 41 3.68 1.059 Task 48 62 3.74 0.848Task 40 121 3.72 0.906 Task 27 4 3.50 0.837 Task 31 32 3.72 1.054 Task 13 31 3.68 0.791 Task 40 81 3.69 0.931Task 51 126 3.71 0.857 Task 10 10 3.40 1.075 Task 24 32 3.72 0.991 Task 4 23 3.65 1.152 Task 51 85 3.65 0.869Task 11 111 3.68 0.874 Task 36 5 3.40 0.548 Task 9 37 3.65 0.978 Task 10 40 3.65 0.949 Task 43 76 3.64 0.743Task 53 125 3.67 0.896 Task 53 11 3.36 0.674 Task 43 34 3.65 0.849 Task 31 40 3.60 1.057 Task 19 78 3.64 0.882Task 31 132 3.64 0.892 Task 9 11 3.36 0.809 Task 13 25 3.64 0.810 Task 48 22 3.59 1.008 Task 31 89 3.64 0.815Task 43 120 3.64 0.807 Task 16 11 3.27 0.905 Task 15 13 3.62 1.121 Task 9 48 3.58 0.942 Task 16 71 3.63 0.975Task 10 128 3.61 0.974 Task 35 4 3.25 0.500 Task 4 18 3.61 1.243 Task 29 36 3.58 0.937 Task 11 71 3.63 0.849Task 19 119 3.59 0.933 Task 30 10 3.20 0.632 Task 5 10 3.60 0.966 Task 5 16 3.56 0.892 Task 53 83 3.61 0.908Task 16 114 3.59 1.020 Task 54 11 3.18 0.874 Task 39 37 3.59 0.956 Task 43 41 3.56 0.896 Task 36 84 3.60 0.971Task 36 125 3.58 0.944 Task 43 7 3.14 1.069 Task 29 29 3.59 0.946 Task 28 21 3.52 0.928 Task 34 57 3.58 0.963Task 28 66 3.55 0.948 Task 31 8 3.13 0.991 Task 19 29 3.55 0.910 Task 36 38 3.50 0.893 Task 37 85 3.58 1.189Task 37 122 3.53 1.158 Task 6 9 3.11 0.928 Task 28 20 3.55 0.945 Task 3 24 3.50 1.095 Task 10 85 3.56 0.993Task 3 72 3.51 1.075 Task 19 9 3.11 1.364 Task 16 30 3.53 1.167 Task 15 16 3.50 1.251 Task 28 44 3.52 0.952Task 39 144 3.51 0.893 Task 39 11 3.09 0.539 Task 26 17 3.53 0.943 Task 39 48 3.48 0.899 Task 39 93 3.51 0.892Task 13 97 3.47 0.879 Task 45 8 3.00 1.549 Task 48 19 3.53 1.073 Task 16 41 3.46 1.098 Task 3 46 3.50 1.006Task 34 84 3.44 1.068 Task 52 6 3.00 1.414 Task 36 33 3.52 0.939 Task 19 38 3.45 1.032 Task 45 72 3.47 0.855Task 15 51 3.43 1.044 Task 26 5 3.00 2.000 Task 37 29 3.45 1.055 Task 26 22 3.41 0.908 Task 27 55 3.45 0.978Task 27 83 3.42 0.952 Task 34 3 3.00 0.000 Task 3 19 3.37 1.342 Task 37 34 3.32 1.065 Task 15 35 3.40 1.035Task 45 112 3.40 0.925 Task 15 3 3.00 0.707 Task 27 23 3.26 0.864 Task 27 27 3.30 0.869 Task 14 48 3.40 1.086Task 26 80 3.39 1.000 Task 28 1 3.00 1.000 Task 45 29 3.24 0.912 Task 45 37 3.19 1.023 Task 13 64 3.39 0.919Task 5 40 3.38 1.170 Task 25 3 2.67 1.155 Task 14 15 3.13 0.990 Task 34 24 3.08 1.213 Task 26 55 3.35 1.040Task 14 73 3.25 1.038 Task 14 8 2.63 0.744 Task 34 21 3.10 1.136 Task 14 23 2.96 0.928 Task 5 24 3.25 1.327Task 25 62 3.08 1.013 Task 37 5 2.60 0.894 Task 25 15 3.00 0.926 Task 25 18 2.94 0.938 Task 25 41 3.10 1.044


75

Appendix E: Lists of Tasks by Time Mean


Std. Dev


Std. Dev


Std. Dev


Std. Dev


Std. Dev

Task 42 143 4.22 0.865 Task 42 10 4.00 1.054 Task 42 36 4.31 0.786 Task 42 46 4.24 0.848 Task 42 94 4.19 0.883Task 46 137 3.85 0.946 Task 47 11 3.91 0.831 Task 46 35 4.00 1.057 Task 47 46 3.96 0.942 Task 46 89 3.79 0.910Task 44 126 3.67 1.117 Task 46 10 3.80 0.919 Task 47 35 3.97 0.985 Task 46 45 3.96 1.021 Task 44 83 3.60 1.178Task 47 137 3.61 0.964 Task 51 9 3.67 1.000 Task 44 30 3.90 0.960 Task 44 40 3.78 1.000 Task 8 92 3.55 0.999Task 33 138 3.59 0.925 Task 44 10 3.40 1.075 Task 30 36 3.86 0.961 Task 33 47 3.66 0.984 Task 33 88 3.53 0.896Task 21 136 3.46 0.934 Task 50 9 3.33 1.118 Task 35 28 3.86 0.848 Task 35 32 3.63 1.008 Task 21 90 3.51 0.951Task 8 143 3.45 1.026 Task 41 6 3.33 0.816 Task 33 37 3.76 0.983 Task 30 46 3.59 1.087 Task 47 88 3.40 0.917Task 7 135 3.37 0.887 Task 33 10 3.30 0.949 Task 32 34 3.74 0.963 Task 32 43 3.53 0.984 Task 7 88 3.39 0.903Task 32 134 3.36 0.984 Task 49 9 3.22 1.093 Task 40 28 3.57 0.997 Task 50 38 3.47 1.033 Task 17 93 3.34 1.016Task 35 122 3.34 0.986 Task 21 10 3.20 0.919 Task 50 29 3.52 1.022 Task 40 37 3.46 1.016 Task 48 62 3.27 0.978Task 30 140 3.34 0.957 Task 40 9 3.11 1.054 Task 7 33 3.45 0.869 Task 21 43 3.35 0.870 Task 32 88 3.26 0.977Task 17 141 3.33 1.005 Task 48 3 3.00 1.000 Task 8 37 3.41 1.092 Task 7 44 3.32 0.857 Task 29 81 3.25 1.055Task 50 119 3.29 1.068 Task 17 12 3.00 0.953 Task 17 33 3.39 1.029 Task 8 48 3.29 1.071 Task 6 86 3.24 0.880Task 40 121 3.28 1.058 Task 3 5 3.00 0.707 Task 21 33 3.39 0.864 Task 17 45 3.29 1.014 Task 35 87 3.24 0.976Task 48 87 3.21 1.036 Task 8 11 2.91 0.944 Task 15 13 3.38 0.870 Task 49 43 3.28 1.054 Task 30 91 3.23 0.870Task 29 120 3.18 1.053 Task 7 11 2.91 0.701 Task 18 19 3.32 0.885 Task 28 21 3.24 0.889 Task 23 85 3.21 0.977Task 23 128 3.16 0.994 Task 29 7 2.86 1.069 Task 38 32 3.31 0.931 Task 18 24 3.21 0.977 Task 50 78 3.21 1.073Task 6 130 3.16 0.843 Task 13 6 2.83 0.983 Task 28 20 3.30 0.865 Task 15 16 3.19 1.047 Task 40 81 3.20 1.066Task 18 90 3.16 0.947 Task 11 11 2.82 0.874 Task 49 34 3.29 1.060 Task 38 39 3.18 0.942 Task 18 65 3.14 0.950Task 41 108 3.12 1.011 Task 18 5 2.80 1.304 Task 20 32 3.28 1.023 Task 41 35 3.14 1.089 Task 41 71 3.10 0.988Task 20 135 3.08 0.962 Task 22 9 2.78 1.202 Task 31 32 3.25 1.078 Task 20 42 3.14 1.095 Task 9 91 3.08 1.035Task 24 126 3.04 0.862 Task 32 9 2.78 0.667 Task 54 25 3.24 0.831 Task 51 38 3.11 1.110 Task 24 82 3.07 0.813Task 31 132 3.02 0.992 Task 45 8 2.75 0.707 Task 23 29 3.21 1.013 Task 11 38 3.08 0.997 Task 20 90 3.07 0.909Task 1 144 3.01 1.031 Task 23 11 2.73 0.905 Task 11 27 3.19 1.039 Task 23 40 3.08 0.997 Task 1 93 3.06 1.030Task 9 142 3.01 1.062 Task 20 10 2.70 1.252 Task 41 29 3.10 1.145 Task 54 36 3.06 0.860 Task 31 89 3.01 0.935Task 49 133 3.00 1.187 Task 24 9 2.67 0.866 Task 6 32 3.09 0.777 Task 31 40 3.05 1.131 Task 22 88 2.99 0.941Task 22 136 2.99 0.951 Task 6 9 2.67 0.500 Task 24 32 3.09 0.995 Task 29 36 3.03 1.028 Task 39 93 2.94 0.930Task 11 111 2.94 0.917 Task 1 12 2.67 0.651 Task 29 29 3.07 1.033 Task 6 41 3.00 0.742 Task 2 90 2.91 0.882Task 38 121 2.93 0.929 Task 54 11 2.64 0.809 Task 1 36 3.06 1.094 Task 22 45 3.00 0.953 Task 49 87 2.86 1.222


76


Std. Dev


Std. Dev


Std. Dev


Std. Dev


Std. Dev

Task 28 66 2.91 0.988 Task 53 11 2.64 0.809 Task 22 36 3.06 0.893 Task 24 41 3.00 0.975 Task 34 57 2.84 0.922Task 2 141 2.89 0.934 Task 30 10 2.60 0.966 Task 43 34 3.03 1.029 Task 1 48 2.96 1.010 Task 19 78 2.83 0.945Task 39 144 2.88 0.960 Task 38 7 2.57 0.787 Task 26 17 3.00 1.061 Task 48 22 2.95 0.999 Task 27 55 2.82 0.841Task 43 120 2.87 0.943 Task 10 10 2.50 0.707 Task 36 33 3.00 1.000 Task 43 41 2.93 1.034 Task 11 71 2.82 0.833Task 51 126 2.83 1.059 Task 9 11 2.45 0.820 Task 39 37 2.97 1.040 Task 36 38 2.87 1.044 Task 43 76 2.82 0.905Task 19 119 2.80 1.038 Task 19 9 2.44 1.590 Task 9 37 2.97 1.142 Task 9 48 2.85 1.091 Task 38 79 2.80 0.853Task 54 114 2.78 0.948 Task 43 7 2.43 0.976 Task 48 19 2.95 1.026 Task 26 22 2.82 1.053 Task 28 44 2.77 1.008Task 27 83 2.77 0.888 Task 2 12 2.42 0.515 Task 51 29 2.93 1.100 Task 10 40 2.80 1.018 Task 26 55 2.73 0.804Task 34 84 2.75 0.955 Task 52 6 2.33 1.211 Task 2 36 2.92 1.079 Task 39 48 2.79 1.010 Task 51 85 2.72 1.007Task 26 80 2.74 0.868 Task 34 3 2.33 1.155 Task 10 30 2.90 1.094 Task 2 48 2.79 0.988 Task 14 48 2.71 1.071Task 15 51 2.73 1.002 Task 15 3 2.33 1.528 Task 19 29 2.83 1.071 Task 45 37 2.78 0.917 Task 13 64 2.69 0.974Task 13 97 2.71 0.989 Task 31 8 2.25 1.035 Task 45 29 2.79 0.978 Task 13 31 2.77 1.055 Task 54 75 2.68 0.961Task 36 125 2.70 1.055 Task 27 4 2.25 0.500 Task 27 23 2.78 1.043 Task 19 38 2.74 1.201 Task 36 84 2.65 1.058Task 10 128 2.70 1.030 Task 12 4 2.25 0.957 Task 13 25 2.76 1.091 Task 27 27 2.70 0.993 Task 12 46 2.65 0.875Task 12 72 2.65 0.966 Task 26 5 2.20 0.837 Task 25 15 2.73 1.163 Task 53 39 2.69 1.030 Task 10 85 2.62 1.012Task 45 112 2.65 0.856 Task 4 5 2.20 0.837 Task 12 22 2.73 1.162 Task 12 26 2.65 1.129 Task 45 72 2.60 0.816Task 52 92 2.57 1.122 Task 39 11 2.18 0.603 Task 52 25 2.72 1.275 Task 52 31 2.65 1.253 Task 52 60 2.55 1.048Task 14 73 2.55 1.106 Task 16 11 2.09 0.831 Task 53 28 2.71 1.117 Task 25 18 2.56 1.149 Task 15 35 2.51 0.919Task 53 125 2.54 0.920 Task 36 5 2.00 1.000 Task 14 15 2.67 1.175 Task 3 24 2.54 1.250 Task 53 83 2.48 0.861Task 25 62 2.45 0.862 Task 35 4 2.00 0.000 Task 16 30 2.63 1.189 Task 34 24 2.50 1.022 Task 16 71 2.41 1.022Task 16 114 2.43 1.056 Task 28 1 2.00 0.000 Task 34 21 2.52 1.030 Task 16 41 2.49 1.121 Task 25 41 2.39 0.737Task 3 72 2.31 1.070 Task 5 6 2.00 0.894 Task 3 19 2.42 1.346 Task 4 23 2.30 1.295 Task 3 46 2.17 0.973Task 4 70 2.21 1.089 Task 25 3 1.67 0.577 Task 37 29 2.34 1.233 Task 14 23 2.26 1.137 Task 4 45 2.13 0.894Task 5 40 2.10 1.105 Task 14 8 1.50 0.535 Task 4 18 2.33 1.414 Task 37 34 2.21 1.200 Task 5 24 2.04 1.160Task 37 122 2.07 1.046 Task 37 5 1.40 0.548 Task 5 10 2.30 1.160 Task 5 16 2.19 1.047 Task 37 85 2.02 0.988


77

Appendix F: Demographic Data 1. What is your role in the delivery of online courses? Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)

Count Percent Count Percent Count Percent Count Percent Count Percent Administrator 24 16.2 2 16.7 7 18.9 9 18.4 14 14.6

Instructor 140 94.6 11 91.7 35 94.6 46 93.9 91 94.8 Instructional designer and/or

developer 70 47.3 7 58.3 17 45.9 24 49 45 46.9

Subject matter expert 59 39.9 6 50.0 15 40.5 21 42.9 37 38.5 Technology expert 21 14.2 3 25.0 5 13.5 8 16.3 13 13.5

Other 7 4.7 0 0 2 5.4 2 4.1 5 5.2 2. Do you have experience as an online student or trainee?

Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)

Count Percent Count Percent Count Percent Count Percent Count Percent Yes 109 74.7 11 91.7 27 73 38 77.6 69 71.9 No 37 25.3 1 8.3 10 27 11 22.4 25 26.0

3. Please rate your level of expertise in using technology for instruction. Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

Very high 39 26.5 4 33.3 10 27.0 14 28.6 23 24.0 High 66 44.9 6 50.0 15 40.5 21 42.9 44 45.8

Moderate 38 25.9 2 16.7 12 32.4 14 28.6 24 25.0 Low 4 2.7 0 0 0 0 0 0 4 4.2

Very low 0 0 0 0 0 0 0 0 0 0


78

4. Are you currently teaching online courses in a military setting?

Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

Yes 49 33.8 12 100 37 100 49 100 0 0 No 96 66.2 0 0 0 0 0 0 96 100

5. Which of the following is a delivery mode for your online courses? Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

Asynchronous Only 79 54.8 9 75 18 48.6 27 56.3 50 52.1 Synchronous Only 4 2.8 0 0 1 2.7 1 2.1 3 3.1

Both 61 42.4 3 25 17 45.9 20 41.7 40 41.7 6. What type of education or training have you completed that prepared you to be an instructor? (Please check all the apply) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

College or university courses 132 89.2 12 100 34 91.9 46 93.9 83 86.5 Training courses or workshops 101 68.2 8 66.7 26 70.3 34 69.4 66 68.8

Instructional certificate programs 51 34.5 5 41.7 14 37.8 19 38.8 31 32.3 On-the-job training 96 64.9 9 75 25 67.6 34 69.4 60 62.5

None 1 0.7 0 0 0 0 0 0 1 1.0 7. What type of education or training have you completed that prepared you to be an online course instructor? (Please check all the apply) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

College or university courses 69 46.6 6 50.0 15 40.5 21 42.9 48 50.0 Training courses or workshops 103 69.6 5 41.7 31 83.8 36 73.5 65 67.7

Instructional certificate programs 39 26.4 2 16.7 14 37.8 16 32.7 21 21.9 On-the-job training 112 75.7 10 83.3 28 75.7 38 77.6 72 75.0

None 2 1.4 0 0 0 0 0 0 1 1.0


79

8. Please select the range that best represents your years of teaching experience in any environment (classroom or distance). Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

Less than a year 3 2.0 1 8.3 0 0 1 2.0 2 2.1 1 to 5 24 16.2 5 41.7 5 13.5 10 20.4 13 13.5

6 to 10 34 23.0 3 25.0 10 27.0 13 26.5 20 20.8 11 to 15 23 15.5 2 16.7 5 13.5 7 14.3 16 16.7 16 to 20 19 12.8 0 0 6 16.2 6 12.2 13 13.5 Over 20 45 30.4 1 8.3 11 29.7 12 24.5 32 33.3

None 0 0 0 0 0 0 0 0 0 0 9. Please select the range that best represents your years of teaching experience in an online environment. (Please do not include blended or hybrid courses) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

Less than a year 13 8.8 3 25.0 2 5.4 5 10.2 8 8.3 1 to 5 46 31.3 4 33.3 8 21.6 12 24.5 33 34.4

6 to 10 69 46.9 4 33.3 21 56.8 25 51.0 43 44.8 11 to 15 12 8.2 0 0 5 13.5 5 10.2 7 7.3 16 to 20 4 2.7 1 8.3 1 2.7 2 4.1 2 2.1 Over 20 3 2.0 0 0 0 0 0 0 2 2.1

None 0 0 0 0 0 0 0 0 0 0 10. Please select the range that best represents the number of online courses you have taught. (Please do not include blended or hybrid courses) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

1 to 5 41 27.7 4 33.3 4 10.8 8 16.3 33 34.4 6 to 10 30 20.3 2 16.7 4 10.8 6 12.2 24 25.0

11 to 15 23 15.5 3 25.0 5 13.5 8 16.3 14 14.6 16 to 20 18 12.2 0 0 6 16.2 6 12.2 11 11.5 21 to 25 2 1.4 0 0 0 0 0 0 2 2.1

More than 25 30 20.3 3 25.0 18 48.6 21 42.9 9 9.4 None 4 2.7 0 0 0 0 0 0 3 3.1


80

11. Please select the range that best represents the number of different online courses you have taught (Please do not count iterations of the same course). Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

1 to 2 70 47.3 8 66.7 9 24.3 17 34.7 51 53.1 3 to 4 36 24.3 2 16.7 9 24.3 11 22.4 25 26.0 5 to 6 23 15.5 1 8.3 9 24.3 10 20.4 12 12.5

More than 6 19 12.8 1 8.3 10 27 11 22.4 8 8.3 12. Please select the category that best represents the level of interaction among students and between you and your students during your typical online course. [Adapted from Bedore and Goins (2001) Online instructional strategies: How many students fit in an online course?] Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

Highly Interactive 55 37.7 4 33.3 16 43.2 20 40.8 33 34.4 Moderately Interactive 65 44.5 6 50 16 43.2 22 44.9 42 43.8

Minimally Interactive 20 13.7 2 16.7 4 10.8 6 12.2 14 14.6 Other 6 4.1 0 0 1 2.0 1 2.7 5 5.2

13. Please select the category that best represents the taxonomic level of the learning outcomes for your typical online course. Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

Higher order 105 72.4 8 66.7 29 80.6 37 77.1 65 67.7 Lower order 36 24.8 4 33.3 6 16.7 10 20.8 26 27.1

Other 4 2.8 0 0 1 2.8 1 2.1 3 3.1 14. Please select the range that best represents the length of your typical online course. Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

Less than a month 5 3.4 2 16.7 0 0 2 4.1 3 3.1 1 to 2 month 19 12.9 4 33.3 8 21.6 12 24.5 7 7.3 3 to 4 month 115 78.2 6 50.0 26 70.3 32 65.3 80 83.3 5 to 6 month 7 4.8 0 0 2 5.4 2 4.1 5 5.2

More than 6 month 1 0.7 0 0 1 2.7 1 2 0 0


81

15. Please select the range that best represents the number of students in your typical online course. Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

1 to 10 12 8.2 1 8.3 3 8.1 4 8.2 8 8.3 11 to 20 64 43.5 5 41.7 19 51.4 24 49.0 39 40.6 21 to 30 52 35.4 1 8.3 11 29.7 12 24.5 39 40.6

More than 30 19 12.9 5 41.7 4 10.8 9 18.4 9 9.4 16. Please select the range that best represents the age of the majority of your online students. Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

18 to 25 14 9.6 0 0 2 5.4 2 4.1 12 12.5 26 to 33 51 34.9 5 41.7 14 37.8 19 38.8 32 33.3 34 to 41 39 26.7 4 33.3 14 37.8 18 36.7 19 19.8 42 to 49 5 3.4 2 16.7 0 0 2 4.1 3 3.1 Over 50 0 0 0 0 0 0 0 0 0 0

Don't know 37 25.3 1 8.3 7 18.9 8 16.3 28 29.2 17. What is your gender? Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

Male 66 44.6 12 100 22 59.5 34 69.4 31 32.3 Female 82 55.4 0 0 15 40.5 15 30.6 65 67.7

18. Please select the range that best represents your age. Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96) Count Percent Count Percent Count Percent Count Percent Count Percent

18 to 25 0 0 0 0 0 0 0 0 0 0 26 to 33 11 7.5 2 16.7 1 2.7 3 6.1 8 8.3 34 to 41 23 15.8 3 25.0 5 13.5 8 16.3 14 14.6 42 to 49 30 20.5 4 33.3 10 27 14 28.6 16 16.7 Over 50 82 56.2 3 25.0 21 56.8 24 49.0 57 59.4


82

19. What is the highest level of education you have completed? Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)

Count Percent Count Percent Count Percent Count Percent Count Percent High school 0 0 0 0 0 0 0 0 0 0

Some college courses 0 0 0 0 0 0 0 0 0 0 Associate/Technical Certificate 1 0.7 0 0 0 0 0 0 1 1.0

Bachelor 7 4.7 2 16.7 0 0 2 4.1 5 5.2 Master 88 59.5 6 50.0 21 56.8 27 55.1 58 60.4 Ph. D. 38 25.7 4 33.3 12 32.4 16 32.7 22 22.9

Other (Ed. D., ABD, JD, DD) 13 8.8 0 0 4 10.8 4 8.2 9 9.4


83

Appendix G: Summary Data in Task Order Task 1 - Review the course for accuracy (e.g., content accuracy, schedule/procedure updates, equipment updates, course syllabus accuracy and functionality) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)

Perform Count Percent Count Percent Count Percent Count Percent Count PercentYes 144 97.3 12 100.0 36 97.3 48 98.0 93 96.9No 4 2.7 0 0.0 1 2.7 1 2.0 3 3.1

Importance

Very High 95 66.0 9 75.0 26 72.2 35 72.9 57 61.3High 39 27.1 3 25.0 8 22.2 11 22.9 28 30.1Moderate 9 6.3 0 0.0 1 2.8 1 2.1 8 8.6Low 1 0.7 0 0.0 1 2.8 1 2.1 0 0.0Very Low 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 Total (n) 144 12 36 48 93 Mean 4.58 4.75 4.64 4.67 4.53

Std. Deviation 0.642 0.452 0.683 0.630 0.653

Time spent Much More 9 6.3 0 0.0 2 5.6 2 4.2 7 7.5More 41 28.5 1 8.3 13 36.1 14 29.2 26 28.0About the Same 46 31.9 6 50.0 9 25.0 15 31.3 31 33.3Less 39 27.1 5 41.7 9 25.0 14 29.2 24 25.8Much Less 9 6.3 0 0.0 3 8.3 3 6.3 5 5.4

Total (n) 144 12 36 48 93Mean 3.01 2.67 3.06 2.96 3.06

Std. Deviation 1.031 0.651 1.094 1.010 1.030 Task 2 - Make changes as needed to maintain course accuracy (e.g., content updates, schedule and/or procedure updates) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)

Perform Count Percent Count Percent Count Percent Count Percent Count Percent Yes 141 95.3 12 100.0 36 97.3 48 98.0 90 93.8No 7 4.7 0 0.0 1 2.7 1 2.0 6 6.3

Importance

Very High 75 53.2 5 41.7 19 52.8 24 50.0 48 53.3High 50 35.5 7 58.3 13 36.1 20 41.7 30 33.3Moderate 14 9.9 0 0.0 3 8.3 3 6.3 11 12.2Low 1 0.7 0 0.0 0 0.0 0 0.0 1 1.1Very Low 1 0.7 0 0.0 1 2.8 1 2.1 0 0.0

Total (n) 141 12 36 48 90Mean 4.40 4.42 4.36 4.38 4.39

Std. Deviation 0.755 0.515 0.867 0.789 0.745


84


Total (n) 141 12 36 48 90Mean 2.89 2.42 2.92 2.79 2.91

Std. Deviation 0.934 0.515 1.079 0.988 0.882 Task 3 - Verify that contact information for student support is accurate (e.g., help desk, mentors, records access) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 72 5 19 24 46 Mean 3.51 4.00 3.37 3.50 3.50

Std. Deviation 1.075 0.707 1.342 1.251 1.006


Total (n) 72 5 19 24 46 Mean 2.31 3.00 2.42 2.54 2.17

Std. Deviation 1.070 0.707 1.346 1.250 0.973 Task 4 - Verify that any required equipment, tools, and instructional materials that are not part of the instructional website are available (e.g., manuals, textbooks) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)



85

Importance


Total (n) 70 5 18 23 45 Mean 3.80 3.80 3.61 3.65 3.87

Std. Deviation 1.199 0.837 1.243 1.152 1.236


Total (n) 70 5 18 23 45 Mean 2.21 2.20 2.33 2.30 2.13

Std. Deviation 1.089 0.837 1.414 1.295 0.894 Task 5 - Verify the existence and practicality of a back-up plan for personnel or resources required for course delivery Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 40 6 10 16 24 Mean 3.38 3.50 3.60 3.56 3.25

Std. Deviation 1.170 0.837 0.966 0.892 1.327


Total (n) 40 6 10 16 24 Mean 2.10 2.00 2.30 2.19 2.04

Std. Deviation 1.105 0.894 1.160 1.047 1.160


86

Task 6 - Prepare for subject areas where learners may have questions or difficulty mastering the course content Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 130 9 32 41 86 Mean 4.02 3.11 4.00 3.80 4.13

Std. Deviation 0.876 0.928 0.950 1.005 0.794


Total (n) 130 9 32 41 86 Mean 3.16 2.67 3.09 3.00 3.24

Std. Deviation 0.843 0.500 0.777 0.742 0.880 Task 7 - Identify critical instructional areas, personalized or relevant examples, and additional materials Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 135 11 33 44 88 Mean 4.04 3.73 4.00 3.93 4.08

Std. Deviation 0.827 0.905 0.791 0.818 0.834


87

Time spent

Much More 13 9.6 0 0.0 4 12.1 4 9.1 8 9.1More 46 34.1 2 18.2 10 30.3 12 27.3 34 38.6About the Same 56 41.5 6 54.5 17 51.5 23 52.3 31 35.2Less 18 13.3 3 27.3 1 3.0 4 9.1 14 15.9Much Less 2 1.5 0 0.0 1 3.0 1 2.3 1 1.1

Total (n) 135 11 33 44 88 Mean 3.37 2.91 3.45 3.32 3.39

Std. Deviation 0.887 0.701 0.869 0.857 0.903 Task 8 - Maintain expertise in subject area Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 143 11 37 48 92 Mean 4.38 4.36 4.22 4.25 4.46

Std. Deviation 0.787 0.924 0.854 0.863 0.747


Total (n) 143 11 37 48 92 Mean 3.45 2.91 3.41 3.29 3.55

Std. Deviation 1.026 0.944 1.092 1.071 0.999 Task 9 - Maintain expertise in online instructional techniques/teaching strategies Overall (n=148) Military C (n=49) Military A (n=12) Military B (n=37) Non-Military (n=96)

Perform Count Percent Count Percent Count Percent Count Percent Count PercentYes 142 95.9 11 91.7 37 100.0 48 98.0 91 94.8No 6 4.1 1 8.3 0 0.0 1 2.0 5 5.2


88

Importance


Total (n) 142 11 37 48 91 Mean 3.86 3.36 3.65 3.58 4.01

Std. Deviation 0.912 0.809 0.978 0.942 0.876


Total (n) 142 11 37 48 91 Mean 3.01 2.45 2.97 2.85 3.08

Std. Deviation 1.062 0.820 1.142 1.091 1.035 Task 10 - Maintain expertise in course-related technologies Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 128 10 30 40 85 Mean 3.61 3.40 3.73 3.65 3.56

Std. Deviation 0.974 1.075 0.907 0.949 0.993


Total (n) 128 10 30 40 85 Mean 2.70 2.50 2.90 2.80 2.62

Std. Deviation 1.030 0.707 1.094 1.018 1.012


89

Task 11 - Use appropriate technology to support instructional administration (e.g., for communication, accessing and storing information) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 111 11 27 38 71 Mean 3.68 3.55 3.78 3.71 3.63

Std. Deviation 0.874 0.820 0.934 0.898 0.849


Total (n) 111 11 27 38 71 Mean 2.94 2.82 3.19 3.08 2.82

Std. Deviation 0.917 0.874 1.039 0.997 0.833 Task 12 - Maintain online security and privacy (e.g., passwords, email addresses, grades) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 72 4 22 26 46 Mean 4.24 4.00 4.50 4.42 4.13

Std. Deviation 1.041 1.414 0.802 0.902 1.108


90

Time spent


Total (n) 72 4 22 26 46 Mean 2.65 2.25 2.73 2.65 2.65

Std. Deviation 0.966 0.957 1.162 1.129 0.875 Task 13 - Provide guidance to learners for obtaining necessary instructional materials, equipment, tools, and other required materials that are not part of the instructional website Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 97 6 25 31 64 Mean 3.47 3.83 3.64 3.68 3.39

Std. Deviation 0.879 0.753 0.810 0.791 0.919


Total (n) 97 6 25 31 64 Mean 2.71 2.83 2.76 2.77 2.69

Std. Deviation 0.989 0.983 1.091 1.055 0.974 Task 14 - Provide guidance to learners on using course-related technologies (e.g., create a technical problem discussion area for learners to assist each other with technical problems) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)



91

Importance


Total (n) 73 8 15 23 48 Mean 3.25 2.63 3.13 2.96 3.40

Std. Deviation 1.038 0.744 0.990 0.928 1.086


Total (n) 73 8 15 23 48 Mean 2.55 1.50 2.67 2.26 2.71

Std. Deviation 1.106 0.535 1.175 1.137 1.071 Task 15 - Diagnose technical problems related to the course (e.g., log-in access, uploading and/or attaching files) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 51 3 13 16 35 Mean 3.43 3.00 3.62 3.50 3.40

Std. Deviation 1.044 1.000 1.121 1.095 1.035


Total (n) 51 3 13 16 35 Mean 2.73 2.33 3.38 3.19 2.51

Std. Deviation 1.002 1.528 0.870 1.047 0.919


92

Task 16 - Ensure all course activities comply with organizational rules, regulations, and policies (e.g., copyright restrictions, ethical standards, and security guidelines) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 114 11 30 41 71 Mean 3.59 3.27 3.53 3.46 3.63

Std. Deviation 1.020 0.905 1.167 1.098 0.975


Total (n) 114 11 30 41 71 Mean 2.43 2.09 2.63 2.49 2.41

Std. Deviation 1.056 0.831 1.189 1.121 1.022 Task 17 - Maintain record-keeping related to course activities (e.g., student attendance and/or participation, grades) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 141 12 33 45 93 Mean 4.31 3.92 4.39 4.27 4.30

Std. Deviation 0.836 0.900 0.827 0.863 0.831


93

Time spent


Total (n) 141 12 33 45 93 Mean 3.33 3.00 3.39 3.29 3.34

Std. Deviation 1.005 0.953 1.029 1.014 1.016 Task 18 - Maintain scheduled hours for student-initiated contact Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 90 5 19 24 65 Mean 3.77 4.00 3.74 3.79 3.77

Std. Deviation 0.887 1.414 0.733 0.884 0.897


Total (n) 90 5 19 24 65 Mean 3.16 2.80 3.32 3.21 3.14

Std. Deviation 0.947 1.304 0.885 0.977 0.950 Task 19 - Provide guidelines for communication protocol (e.g., netiquette guidelines, examples of appropriate communication, how and when to contact instructor, time frame for instructor feedback) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)



94

Importance


Total (n) 119 9 29 38 78 Mean 3.59 3.11 3.55 3.45 3.64

Std. Deviation 0.933 1.364 0.910 1.032 0.882


Total (n) 119 9 29 38 78 Mean 2.80 2.44 2.83 2.74 2.83

Std. Deviation 1.038 1.590 1.071 1.201 0.945 Task 20 - Communicate learning goals and objectives for each instructional component Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 135 10 32 42 90 Mean 4.13 3.50 4.06 3.93 4.21

Std. Deviation 0.871 1.269 0.716 0.894 0.855


Total (n) 135 10 32 42 90 Mean 3.08 2.70 3.28 3.14 3.07

Std. Deviation 0.962 1.252 1.023 1.095 0.909


95

Task 21 - Provide directions for assignments and other instructional activities (e.g., high quality examples of similar assignments/activities/discussions, uploading/downloading assignments or activities) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 136 10 33 43 90 Mean 4.25 4.20 4.12 4.14 4.30

Std. Deviation 0.748 0.919 0.650 0.710 0.771


Total (n) 136 10 33 43 90 Mean 3.46 3.20 3.39 3.35 3.51

Std. Deviation 0.934 0.919 0.864 0.870 0.951 Task 22 - Provide assessment criteria (e.g., minimum requirements for course website login, discussion postings, late submissions policies) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 136 9 36 45 88 Mean 4.01 3.78 3.94 3.91 4.05

Std. Deviation 0.865 1.093 0.791 0.848 0.870


96

Time spent


Total (n) 136 9 36 45 88 Mean 2.99 2.78 3.06 3.00 2.99

Std. Deviation 0.951 1.202 0.893 0.953 0.941 Task 23 - Use strategies to gain and maintain learner attention (e.g., invite virtual guests, arrange student debates, create discussions, provide examples that relate to instruction and to application settings) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 128 11 29 40 85 Mean 3.96 3.55 3.86 3.78 4.02

Std. Deviation 0.891 1.128 0.789 0.891 0.886


Total (n) 128 11 29 40 85 Mean 3.16 2.73 3.21 3.08 3.21

Std. Deviation 0.994 0.905 1.013 0.997 0.977 Task 24 - Establish the relevance of the instruction to learning goals and/or job performance Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)



97

Importance


Total (n) 126 9 32 41 82 Mean 3.87 3.56 3.72 3.68 3.94

Std. Deviation 0.933 1.333 0.991 1.059 0.866


Total (n) 126 9 32 41 82 Mean 3.04 2.67 3.09 3.00 3.07

Std. Deviation 0.862 0.866 0.995 0.975 0.813 Task 25 - Provide rewards and incentives for student participation (e.g., team competitions, recognitions) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 62 3 15 18 41 Mean 3.08 2.67 3.00 2.94 3.10

Std. Deviation 1.013 1.155 0.926 0.938 1.044


Total (n) 62 3 15 18 41 Mean 2.45 1.67 2.73 2.56 2.39

Std. Deviation 0.862 0.577 1.163 1.149 0.737


98

Task 26 - Guide learners to set personal learning goals Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 80 5 17 22 55 Mean 3.39 3.00 3.53 3.41 3.35

Std. Deviation 1.000 0.707 0.943 0.908 1.040


Total (n) 80 5 17 22 55 Mean 2.74 2.20 3.00 2.82 2.73

Std. Deviation 0.868 0.837 1.061 1.053 0.804 Task 27 - Guide learners to create action plans to achieve learning goals (e.g., identify learning resources and/or learning strategies) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 83 4 23 27 55 Mean 3.42 3.50 3.26 3.30 3.45

Std. Deviation 0.952 1.000 0.864 0.869 0.978


99

Time spent


Total (n) 83 4 23 27 55 Mean 2.77 2.25 2.78 2.70 2.82

Std. Deviation 0.888 0.500 1.043 0.993 0.841 Task 28 - Guide learners' ongoing self-evaluation of their personal learning strategies Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 66 1 20 21 44 Mean 3.55 3.00 3.55 3.52 3.52

Std. Deviation 0.948 0.000 0.945 0.928 0.952


Total (n) 66 1 20 21 44 Mean 2.91 2.00 3.30 3.24 2.77

Std. Deviation 0.988 0.000 0.865 0.889 1.008 Task 29 - Use strategies to increase interaction among learners (e.g., group projects, group problem solving, learner presentations, peer critiques, icebreakers) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)



100

Importance


Total (n) 120 7 29 36 81 Mean 3.77 3.57 3.59 3.58 3.81

Std. Deviation 0.923 0.976 0.946 0.937 0.910


Total (n) 120 7 29 36 81 Mean 3.18 2.86 3.07 3.03 3.25

Std. Deviation 1.053 1.069 1.033 1.028 1.055 Task 30 - Use strategies to create a friendly and open environment (e.g., initiate contact with individual learners, use personal stories and/or relevant personal examples during instruction) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 140 10 36 46 91 Mean 3.97 3.20 4.08 3.89 4.00

Std. Deviation 0.881 0.632 0.841 0.875 0.894


Total (n) 140 10 36 46 91 Mean 3.34 2.60 3.86 3.59 3.23

Std. Deviation 0.957 0.966 0.961 1.087 0.870


101

Task 31 - Use strategies to solicit sharing of personal knowledge and experience from all learners Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 132 8 32 40 89 Mean 3.64 3.13 3.72 3.60 3.64

Std. Deviation 0.892 0.991 1.054 1.057 0.815


Total (n) 132 8 32 40 89 Mean 3.02 2.25 3.25 3.05 3.01

Std. Deviation 0.992 1.035 1.078 1.131 0.935 Task 32 - Use questions that promote discussion among learners (e.g., open-ended and divergent questions) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 134 9 34 43 88 Mean 4.05 3.56 4.26 4.12 4.00

Std. Deviation 0.826 0.882 0.751 0.823 0.830


102

Time spent


Total (n) 134 9 34 43 88 Mean 3.36 2.78 3.74 3.53 3.26

Std. Deviation 0.984 0.667 0.963 0.984 0.977 Task 33 - Use questions that promote higher order thinking (e.g., questions that require learners to compare, analyze, evaluate, or prioritize relevant information or concepts) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 138 10 37 47 88 Mean 4.26 4.00 4.27 4.21 4.26

Std. Deviation 0.757 0.817 0.769 0.778 0.750


Total (n) 138 10 37 47 88 Mean 3.59 3.30 3.76 3.66 3.53

Std. Deviation 0.925 0.949 0.983 0.984 0.896 Task 34 - Provide collaboration guidelines (e.g., assigned roles within a group, guidelines for resolving conflicts, guidelines for peer feedback) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)



103

Importance


Total (n) 84 3 21 24 57 Mean 3.44 3.00 3.10 3.08 3.58

Std. Deviation 1.068 2.000 1.136 1.213 0.963


Total (n) 84 3 21 24 57 Mean 2.75 2.33 2.52 2.50 2.84

Std. Deviation 0.955 1.155 1.030 1.022 0.922 Task 35 - Model appropriate communication behavior (e.g., language, acknowledge diverse viewpoints, active listening) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 122 4 28 32 87 Mean 4.13 3.25 4.18 4.06 4.13

Std. Deviation 0.802 0.500 0.772 0.801 0.804


Total (n) 122 4 28 32 87 Mean 3.34 2.00 3.86 3.63 3.24

Std. Deviation 0.986 0.000 0.848 1.008 0.976


104

Task 36 - Require learners to comply with communication guidelines Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 125 5 33 38 84 Mean 3.58 3.40 3.52 3.50 3.60

Std. Deviation 0.944 0.548 0.939 0.893 0.971


Total (n) 125 5 33 38 84 Mean 2.70 2.00 3.00 2.87 2.65

Std. Deviation 1.055 1.000 1.000 1.044 1.058 Task 37 - Use strategies to address undesirable interaction behavior (e.g., intervene with students engaging in disruptive conversations, give specific suggestions for improvement) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 122 5 29 34 85 Mean 3.53 2.60 3.45 3.32 3.58

Std. Deviation 1.158 0.894 1.055 1.065 1.189


105

Time spent


Total (n) 122 5 29 34 85 Mean 2.07 1.40 2.34 2.21 2.02

Std. Deviation 1.046 0.548 1.233 1.200 0.988 Task 38 - Use strategies to maintain the learners' focus on the learning activities Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 121 7 32 39 79 Mean 3.81 3.57 3.84 3.79 3.81

Std. Deviation 0.820 0.787 0.847 0.833 0.818


Total (n) 121 7 32 39 79 Mean 2.93 2.57 3.31 3.18 2.80

Std. Deviation 0.929 0.787 0.931 0.942 0.853 Task 39 - Share supplemental learning resources with the learners (e.g., links to relevant websites) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)



106

Importance


Total (n) 144 11 37 48 93 Mean 3.51 3.09 3.59 3.48 3.51

Std. Deviation 0.893 0.539 0.956 0.899 0.892


Total (n) 144 11 37 48 93 Mean 2.88 2.18 2.97 2.79 2.94

Std. Deviation 0.960 0.603 1.040 1.010 0.930 Task 40 - Provide summation for learning activities and assignments (e.g., threaded discussions) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 121 9 28 37 81 Mean 3.72 3.67 3.79 3.76 3.69

Std. Deviation 0.906 0.707 0.833 0.796 0.931


Total (n) 121 9 28 37 81 Mean 3.28 3.11 3.57 3.46 3.20

Std. Deviation 1.058 1.054 0.997 1.016 1.066


107

Task 41 - Use strategies to promote the transfer of learning to the job environment (e.g., realistic examples and practice activities, discussions on how to promote transfer or barriers to transfer) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 108 6 29 35 71 Mean 3.81 3.83 3.76 3.77 3.82

Std. Deviation 0.855 0.983 0.872 0.877 0.850


Total (n) 108 6 29 35 71 Mean 3.12 3.33 3.10 3.14 3.10

Std. Deviation 1.011 0.816 1.145 1.089 0.988 Task 42 - Assess learners' attainment of learning objectives (e.g., exams, papers, projects, portfolios) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 143 10 36 46 94 Mean 4.55 4.50 4.58 4.57 4.53

Std. Deviation 0.699 0.707 0.770 0.750 0.683


108

Time spent


Total (n) 143 10 36 46 94 Mean 4.22 4.00 4.31 4.24 4.19

Std. Deviation 0.865 1.054 0.786 0.848 0.883 Task 43 - Provide opportunities for learners to self-assess their performance Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 120 7 34 41 76 Mean 3.64 3.14 3.65 3.56 3.64

Std. Deviation 0.807 1.069 0.849 0.896 0.743


Total (n) 120 7 34 41 76 Mean 2.87 2.43 3.03 2.93 2.82

Std. Deviation 0.943 0.976 1.029 1.034 0.905 Task 44 - Monitor group discussions (e.g., quality and frequency of participation) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)



109

Importance


Total (n) 126 10 30 40 83 Mean 3.95 3.80 3.93 3.90 3.94

Std. Deviation 0.920 0.789 0.828 0.810 0.967


Total (n) 126 10 30 40 83 Mean 3.67 3.40 3.90 3.78 3.60

Std. Deviation 1.117 1.075 0.960 1.000 1.178 Task 45 - Monitor learner attitudes (e.g., reactions to instructional activities) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 112 8 29 37 72 Mean 3.40 3.00 3.24 3.19 3.47

Std. Deviation 0.925 1.414 0.912 1.023 0.855


Total (n) 112 8 29 37 72 Mean 2.65 2.75 2.79 2.78 2.60

Std. Deviation 0.856 0.707 0.978 0.917 0.816


110

Task 46 - Provide feedback on the accuracy of learners' statements or performances



Importance


Total (n) 137 10 35 45 89Mean 4.24 4.10 4.29 4.24 4.21

Std. Deviation 0.853 0.738 0.825 0.802 0.885


Total (n) 137 10 35 45 89Mean 3.85 3.80 4.00 3.96 3.79

Std. Deviation 0.946 0.919 1.057 1.021 0.910

Task 47 - Provide feedback to learners on how to correct errors



Importance


Total (n) 137 11 35 46 88Mean 4.09 4.36 4.23 4.26 3.98

Std. Deviation 0.873 0.924 0.843 0.855 0.871


111

Time spent


Total (n) 137 11 35 46 88Mean 3.61 3.91 3.97 3.96 3.40

Std. Deviation 0.964 0.831 0.985 0.942 0.917

Task 48 - For group projects, provide feedback on the performance of each group as a whole, and on individual contributions toward the group's goals



Importance


Total (n) 87 3 19 22 62Mean 3.72 4.00 3.53 3.59 3.74

Std. Deviation 0.898 0.000 1.073 1.008 0.848


Total (n) 87 3 19 22 62Mean 3.21 3.00 2.95 2.95 3.27

Std. Deviation 1.036 1.000 1.026 0.999 0.978

Task 49 - Follow-up individually with learners who fail to participate in course activities




112

Importance


Total (n) 133 9 34 43 87Mean 3.99 4.11 4.03 4.05 3.94

Std. Deviation 0.857 0.782 0.870 0.844 0.867


Total (n) 133 9 34 43 87Mean 3.00 3.22 3.29 3.28 2.86

Std. Deviation 1.187 1.093 1.060 1.054 1.222

Task 50 - Document all evaluation data (e.g., instructional activities undertaken to support personal evaluation as an instructor)



Importance


Total (n) 119 9 29 38 78Mean 3.91 3.89 3.93 3.92 3.87

Std. Deviation 0.863 0.782 0.961 0.912 0.843


Total (n) 119 9 29 38 78Mean 3.29 3.33 3.52 3.47 3.21

Std. Deviation 1.068 1.118 1.022 1.033 1.073


113

Task 51 - Provide instructor input for course improvement (e.g., instructional materials, learning activities, instructional technologies, support systems, equipment) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 126 9 29 38 85 Mean 3.71 4.22 3.72 3.84 3.65

Std. Deviation 0.857 0.667 0.841 0.823 0.869


Total (n) 126 9 29 38 85 Mean 2.83 3.67 2.93 3.11 2.72

Std. Deviation 1.059 1.000 1.100 1.110 1.007 Task 52 - Evaluate your performance as an online instructor Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance

Very High 26 28.3 0 0.0 10 40.0 10 32.3 16 26.7High 39 42.4 4 66.7 6 24.0 10 32.3 28 46.7Moderate 19 20.7 0 0.0 6 24.0 6 19.4 13 21.7Low 6 6.5 0 0.0 3 12.0 3 9.7 3 5.0Very Low 2 2.2 2 33.3 0 0.0 2 6.5

Total (n) 92 6 25 31 60 Mean 3.88 3.00 3.92 3.74 3.95

Std. Deviation 0.970 1.549 1.077 1.210 0.832


114

Time spent


Total (n) 92 6 25 31 60 Mean 2.57 2.33 2.72 2.65 2.55

Std. Deviation 1.122 1.211 1.275 1.253 1.048 Task 53 - Solicit input from learners for course improvement (e.g., instructional materials, learning activities, instructional technologies, support systems, equipment) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)


Importance


Total (n) 125 11 28 39 83 Mean 3.67 3.36 3.86 3.72 3.61

Std. Deviation 0.896 0.674 0.891 0.857 0.908


Total (n) 125 11 28 39 83 Mean 2.54 2.64 2.71 2.69 2.48

Std. Deviation 0.920 0.809 1.117 1.030 0.861 Task 54 - Use evaluation data to recommend course revisions (e.g., to accommodate learners needs, problems with the instructional design, course content, delivery systems, media, equipment) Overall (n=148) Military A (n=12) Military B (n=37) Military C (n=49) Non-Military (n=96)



115

Importance


Total (n) 114 11 25 36 75 Mean 3.76 3.18 3.96 3.72 3.77

Std. Deviation 0.886 0.874 0.790 0.882 0.909


Total (n) 114 11 25 36 75 Mean 2.78 2.64 3.24 3.06 2.68

Std. Deviation 0.948 0.809 0.831 0.860 0.961

Date post:	06-Aug-2020
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

Final IDS Duties Tasks - DECADE Consulting Instructo… · list of ranked tasks performed as part...

Documents