Post on 03-Feb-2018
transcript
Module 2.1 Design Document Penny Reed /Heidi Weber
Topic: Earth and Space Science, Grade 2.
I. Analysis Phase
A. Background Information
The focus of this Web-based Learning System(WBLS) is to expand student
knowledge about the phases of the moon during science class. It is designed for
second grade students with a variety of learning styles, cognitive abilities, socio-
economic backgrounds, and average computer skills. The objective of this WBLS
is to provide students an opportunity to view current photos of the moon from any
computer at any time. The learning objectives include:
• Describe how the moon appears each day
• Identify the five main phases of the moon
• Predict the next moon phase
• Demonstrate the correct order of the phases of the moon
B. Problem Analysis
Students need to learn the material to have a better understanding how the solar
system works and to understand why they may see the moon appear differently
each day. Due to weather conditions, earlier bedtimes of second graders, and time
of sunset, a WBLS is beneficial for ‘observing’ and recording phases of the moon.
Websites allow students to view actual and current phases of the moon.
The gap between actual student learning and optimal learning is that these
students have come to second grade with a first grade science background. The
actual learning is that student’s prior knowledge is that the moon moves across
the sky and that the moon looks different at different times. The optimal learning
is that students will be able to describe how the moon appears different each day,
identify the five main phases of the moon, predict the next moon phase, and
demonstrate the correct order of the phases of the moon.
By using this WBLS students will be able to observe the moon’s phases anytime,
anywhere despite weather conditions, or other obstacles in observing the moon.
Students will also be able to participate in an on-line discussion responding to
instructor/peer posted questions and peer responses. The WBLS will be
supported through teacher-led direct instruction, class discussion, and various
hands-on activities. A WBLS is appropriate to solve this gap because it is a
highly motivational teaching strategy that incorporates credible websites, that are
accessible through public school Internet connections.
C. Instructional Situation Analysis
• Learning Goal and Outcome Level
Given websites, personal observation chart, and other supporting information, the
students will be able to describe how the moon appears each day, identify the five
main phases of the moon, predict the next moon phase, and demonstrate the
correct order of the moon phases.
Table 1 – Categories of Learning Descriptions
Category of
Learning
Learned
Capability
Verb
Explanation Example
Knowledge View Recall learned material,
exploring
Observe at the moon on a
nightly basis
Comprehension Describe Grasp the meaning of
new concepts
Describe which part of the
moon is lit (using first
quarter, half, third quarter,
full moon, new moon)
Illustrate Compare the illuminated
shape of the moon from one
night to the next by
completing a drawing
Application Demonstrate Use information in a
new concrete situation
Students will be able to
accurately name the four
main phases of the moon on
an observation chart (new,
full, ¼, ½, ¾)
Predict Students accurately predict
the shape of the illuminated
portion of the moon for the
next night
Synthesize Create Organize information
in the correct order
Creating a mini-book about
the moon, putting the phases
in the correct order
Evaluation Justify Judge value and
purpose with relevant
criteria
Student justifies the order of
his/her completed booklet
Attitudes Complete Assume responsibility
to complete the project
Complete daily drawing
either at home or at school
Motor Skills Fine motor
skills
Able to perform skills
on an individual basis
Create a mini-book, cutting,
pasting, coloring, fold
• Learning Context
The purpose of this analysis is to analyze the environment and the elements that
are included in the environment.
The WBLS will be used in an elementary school setting. The WBLS can be
accessed either in individual classroom settings, the school library computer lab,
or the school computer lab (20 desktop Dell computers). The table below shows
the elements for the WBLS to be a successful learning event for students.
Main Elements Aspects
Organizational
Infrastructure
Computers are Dell desktops, located either
in individual classrooms, the school library,
or the school computer lab. All computers
are connected to the Internet, using
Microsoft’s Internet Explorer. The WBLS
is owned by the creators, but can be
accessed by students, and managed by the
staff (teachers) as it is housed on the
school’s server.
Allocation and competencies
of personnel
Instructor is currently a member of the
District Technology Committee, is
attending a local university to obtain a
Master’s of Instructional Technology, and
has an elementary certification. The district
technical support staff is encouraging and
supportive when new applications of
technology are implemented. The school
administrative team is eager to showcase the
integration of technology in the classroom.
Learner location and
technology
Students are located in the same physical
location as the instructor. The district is
located in a suburban district in northeast
Ohio. The technology requirements for this
WBLS include a computer with Internet
connection.
• Learner Background
In this section of the analysis phase, the learners, or users, of the WBLS are
described. An in-depth analysis of the learner is conducted so that the WBLS is
relevant and interesting for the learner.
Main Element in
Learner Analysis
Aspects
General
Characteristics • A fairly even number of boys and girls
• Class size 20-25 students
• Second graders ranging in age from is 7-8 years old
• Have computer, keyboarding, and Internet
experience.
• Comparably diverse socio-economic backgrounds
• At least half read on grade level, others range from
one to three grade levels above or below grade
level
• Learning styles and abilities are diverse
Motivation • In the beginning of the project, motivation is high.
Students enjoy using the computer, matching the
phases of the moon to the description
• As the project progresses, students may lose
interest, therefore it will be important to add a new
dimension periodically to re-engage students.
• Students are interested in the environment and the
world around them
Prior knowledge,
Communication
skills,
Technology Skills
• Students know that the moon looks different at
various times throughout the month
• Students have a basic knowledge of how to
communicate ideas effectively using adverbs,
adjectives and verbs correctly
• Students have a basic knowledge of the computer.
Abilities and
Disabilities • Abilities and disabilities will be taken into
consideration are accounted for and WBLS can be
accommodated according to a student’s needs.
Other learner
characteristics • This WBLS takes into consideration the three main
learning styles – visual, auditory, and kinesthetic
• Educational enrichment needs can be incorporated
as needed
• For learners who may need concepts repeated,
presented in a different way, or are absent from the
classroom, the WBLS can meet these needs
Learning Content
The purpose of this section is to analyze the content the learner is to learn. This
includes the sequence of the presentation, what skills and subskills are needed by
the learner in order to complete the project.
------------------------------------------------------------------------------------------------------------
Phases of the Moon
1.0 Movement of the
moon in relation to
the earth and sun
(background
information)
1.1 Show United
Streaming video,
read books to
provide basic
knowledge
2.0 Observation
Calendar
2.1 Explain how to
complete the observation
calendar, illustrations, and
labeling the phase
2.2 As a class, view a
website showing the current
phase of the moon and
complete the first drawing
and label
3.0 Communication
3.1.1 Classroom discussion
3.1.2 On-line discussion
forum
Recognition that the
moon moves across
the sky
Recognize that the
moon looks different
at different times
Task-Objective-Assessment Item Blueprint (TOAB)
Learning Task Item and Number Objective Outcome Assessment Item
Step 1.0 F2F Interaction 1.1 Demonstrate movement of
the moon in relation to the earth
and sun (background
information)
1.2 Show United Streaming
video, read books to provide
basic knowledge
Defined Concept
Intellectual Skill
Verbal Information
Step 2.0 Observation Calendar 2.1 Explain how to complete the
observation calendar,
illustrations and labeling the
phase
2.2 View websites complete first
drawing of the moon phase
Concrete Concept
Motor Skill
Define Concepts
Step 3.0 Communication
3.1 Classroom Discussion
3.2 On-line discussion forums
Intellectual Skill
Verbal Information
Step 4.0 Library Corner
4.1 View video on why the
moon appears to change shape
4.2 Read a Reader’s Theater
about the moon phases
Concrete Concepts
Verbal Information
Planning for Evaluation of WBLS
Topic: Earth and Space Science, Grade 2
II. Evaluation Planning Stage The purpose of this stage is to plan for formative and summative evaluation of the
WBLS product. Formative evaluation is conducted during the design and development
phase and provides feedback to the designer. This feedback can give the designer
suggestions for improvement prior to full implementation of the WBLS and occurs in a
recursive loop allowing for many modifications in the design. The summative evaluation
is conducted after the full implementation of the WBLS with the user.
Evaluation Criteria Explanation Data Source
Effectiveness (mastery of
goals and success of WBI)
Determine if actual learning
has occurred with the
WBLS
Student mini-book
Observation Calendar
Discussion participation
Efficiency (delivered in a The WBLS was developed Document time to prepare
timely, or cost saving
manner)
well before the unit was to
be taught
materials and evaluation of
student products
Documentation of student
time on WBLS activities
Appeal (gain and
maintain learner attention
and interest; usability
The WBLS was easy to use
for second graders; the
colors were appropriate and
text size and color were
pleasing. New components
were introduced throughout
the project to maintain
student interest. Determine
ease of navigation of
bookmarked sites and
discussion forum
Instructor and student
opinions, review by peers;
interview of students to
determine degree of
difficulty in accessing
websites and participating
in the on-line discussion
forum
• Formative Evaluation Plan
Formative evaluation is conducted during the design and development phase of the
WBLS. This evaluation will take place as needed during the design/development stage.
The purpose is to provide feedback to the designer for improvement of the design, ease of
use, and ensure that it is visually appealing. This stage is important to the overall
effectiveness of the product as it gives the designer an opportunity to work out any bugs.
The designer will conduct formative evaluation during the WBLS design phase with
fellow teachers (regular and special education), friends, and family members. These
evaluators will review the WBLS and respond to the designer in an interview using the
questionnaire.
Formative evaluation in the final WBLS prototype development will take place
several times using small group and one-to-one tryout methods. Members of the small
group will typically include the target audience, but due to availability of students this
group would include the instructor’s family and/or other teachers or friends willing to
participate. Age level participants will be used in a one-to-one tryout to ascertain the
suitability and gain their perspective of the WBLS.
• List of Materials to be Examined (what is being evaluated?)
� Design Plans
• Objectives
• Assessment Items
• Clustered Objectives
• Instructional Strategies
• Motivational Strategies
� Prototype of WBLS
• Preliminary storyboards
• Interface (how the WBLS will look)
• Navigational features
• WBLS prototype
Formative Evaluation Plan:
Who are the stakeholders?
Primary – Designer, Instructor
Secondary – Students, Administration, Parents
What is being evaluated?
The design of the WBLS, the websites, discussion forum, materials
Who are the evaluators and reviewers?
Grade level teachers, special education teachers, friends willing to help
What are the evaluation methods? Questionnaire and Interview (see next section)
What types of decisions need to be made as the WBLS design plan and
prototype is develop and revised? Are the suggestions for revision valid and will the suggestions make the
WBLS more effective.
Formative Evaluation Plan Questionnaire
Evaluation Criteria Sample Questions Methods
and Tools
Effectiveness
• Goals
• Content
• Technology
• Message
Design
Does the goal match the ODE indicator?
Are the goals and objectives clear and achievable?
Are the goals and objectives appropriate for the
method of delivery?
Do the activities promote learning?
Is the information complete, covering the content
properly?
Are sites accessible through the school server
(filters)?
Was the time frame of the WBLS appropriate?
Are directions clear and concise?
Interview
Interview
Interview
/Observation
Interview/
Observation
Efficiency
• Goals
• Content
• Technology
• Message
Design
Are goals clear and concise?
Is it appropriate to our grade level?
Is content information clearly and concisely
presented?
Are websites and discussion forum easy to navigate?
Interview
Interview
Interview
Interview
Is the structure of the content clear?
Appeal
• Goals
• Content
• Technology
• Message
Design
Are students aware of the goals of the WBLS?
Is the content interesting?
Are there any typographical, punctuation, grammar,
or spelling errors?
Is navigation easy?
Is the vocabulary appropriate for the user?
Are there limited distractions on the sites?
Are the colors, typeface, and font emphasis used
appropriately to enhance learning?
Does it have good navigational design?
Are the graphics relevant and appropriate to the
WBLS?
Interview/
Observation
Interview/
Observation
Interview
Observation
Interview /
Observation
Results of the formative assessment will be delivered to the designer verbally
through an informal interview.
• Preliminary Summative Evaluation Plan
Summative evaluation takes place after the final implementation and students have
completed the WBLS. The purpose is to gather data to determine the effectiveness,
efficiency, and appeal of the final product. Summative evaluators will be fellow teachers
(regular ed and special education), students and the instructor. The method for fellow
teachers and students will take place in an interview/discussion format. The instructor
will evaluate the WBLS based on periodic observation and continual reflection.
• List of Materials to be Examined (what is being evaluated?)
o Design Plans
• Objectives
• Assessment Items
• Clustered Objectives
• Instructional Strategies
• Motivational Strategies
� Final Version of WBLS
• Interface (how the WBLS looks)
• Navigational features
• WBLS
Summative Evaluation Planning Questionnaire
Instructional Strategy Plan WBLS
Topic: Earth and Space Science, Grade 2
This section of the design document is the Instructional Strategy Plan. This section describes the instructional and motivational strategies that will be used throughout the WBLS. These strategies will be used during the month-long unit. These strategies have not been broken down into a lesson-by-lesson format.
I. Unit Overview
Description Instructional Strategies for phases of the moon unit
Orientation to Learning • Assist learner recall of prior knowledge, skills and experiences
Anticipation Guide – to access prior knowledge
Discuss results Anticipation Guide (classroom discussion)
Review Moodle use for answering questions and discussion with classmates and teacher
• Provide an overview of the unit
This WBLS is blended instruction between face-to-face direct instruction and self-directed learning using the web-based instructional units. Students will be viewing and responding to educational videos; reading trade books, listening to teacher read
Evaluation Criteria Sample Questions Data Sources
Effectiveness
How did the final product
meet the objectives?
What did the learner like?
What did the learner find
easy or difficult to use?
Mini-book
Observation Calendar
Discussions
Interview (informal)
Efficiency
Were the learners able to
participate in and complete
the activities in the time
allotted?
Teacher observation
Percentage of completed
activities
Informal interview with
users
Appeal
Will students want to
participate in another
WBLS?
Teacher observation
Percentage of completed
activities
Informal interview with
users
alouds, partner reading, individual reading; completing graphic organizers; journal writing about the earth, moon, and stars; letter writing (moon to the earth or the earth to the moon) accessing interactive websites, completing personal moon observation chart; performing hands on activities; interacting with classmates and teacher through Moodle (Learning Management System); entrance slips, exit slips; Reader's Theater; demonstrations; experiments; web searches formative assessments, summative assessment at the of the unit
• State the Learning Objective Students will be able to: Given websites, personal observation chart, and other supporting information, the students will be able to describe how the moon appears each day, identify the five main phases of the moon, predict the next moon phase, and demonstrate the correct order of the moon phases.
• Explain relevance of instruction
The gap between actual student learning and optimal learning is that these students have come to second grade with a first grade science background. The actual learning is that student’s prior knowledge is that the moon moves across the sky and that the moon looks different at different times. The optimal learning is that students will be able to describe how the moon appears different each day, identify the five main phases of the moon, predict the next moon phase, and demonstrate the correct order of the phases of the moon. Throughout this process, students will have a broader knowledge base about relationship between the sun, the moon, and the earth. This concept will be transferred to students through the use of a teacher-led, large group game called Rivet.
• Provide directions on how to start, navigate, and proceed through the instruction
• Letter to parents explaining the unit and what students will be expected to do.
• Model how to complete the Observation Chart.
• Demonstrate how to access websites organized on mybookmarks.com.
• Show the end product (Moon Phases Booklet).
• Teacher uses Interwrite board to navigate to first discussion question in Moodle, modeling how to respond in the
discussion forum. • On a daily basis, students will be
provided with instructions and expectations for that day’s activities.
II. Instruction on the Content
Instruction on the content is the area of the design document that shows what instructional strategies will be used during the presentation of the content.
Instruction on the content subcomponents
Instructional Strategies for Moon Phases Instructional Unit
Present Content Information
• Direct instruction through presentations using text, audio, multimedia, streaming audio/visual, lectures, etc.
• Demonstration led by teacher how moon reflects sunlight, how moon revolves and rotates around the earth always showing the same side to the earth
• Hands-on experiments • Modeling by instructor how to complete observation chart and how to log in and complete discussions in Moodle
• Graphic Organizers such as Venn diagrams, I Am poem,
t-charts, The Most Important Thing about… • Teacher-led discussions • Student-led discussions • In-class reading assignments • Web searches for information • Simulations on the Internet (mybookmarks.com website)
• Collaborative, cooperative, and independent learning situations
Provide Learning Cues • Moodle discussion with teacher created prompt
• Remind students to observe the moon nightly and complete the observation chart
• Before, during, after reading strategies
• Mnemonic (Left/last; right/first • On-line games (mybookmarks.com website)
• Web research • Journal writing • Moodle discussions • On-line vocabulary practice • Projects- completion of observation chart, mini book, and summary of readings
• Photos of the Moon • Virtual Field Trip (mybookmarks.com website)
• Journals response journals • Graphic Organizers
Present Opportunities for Practice
• Class discussion – student to student feedback
• Moodle discussion threads • Teacher feedback on activities • Self-correcting feedback on Internet activities
Provide feedback on the practice performance
• Review previous learning throughout the unit
• Summarize readings/activities at end of each lesson (exit slips)
• End lesson with preview of the next topic or task
• Use text, video, audio wrap-ups of the lesson
• Instruction comments on progress of on-line discussion forum
Provide review and close the unit of instruction
• Review game such as Rivet or tic tac toe in collaborative group, no summative
assessment
III. Measurement of Learning
Measurement of Learning outlines the methods used to assess student learning. These methods can take the form of diagnostic, formative, or summative assessments.
Measurement of Learning Subcomponents
Instructional Strategies for Moon Phases Instruction
Assess performance or progress • An informal pretest is given to students in the form of the Anticipation Guide prior to instruction to determine student background knowledge.
• Individual pre and post-learning will be assessed via a KWL graphic organizer prior to reading activities
• Exit and Entrance slips • Require completion of mini-book project
• Require completion of observation chart
• Checklists and rubrics to score reading, writing, and discussion participation
Advise learners of scores • There is no summative posttest
as second grade science is intended for exposure to concepts, not mastery
• Provide project guidelines for learners to self-check their work.
IV. Summary and Close
This section of the design document provides for a summary and closure of the learning. This component allows for the closure of learning, reviewing the work, and transferring the learning to new and different situations.
Summary and Close Overview of the Instructional Strategies for Moon Phases
Provide for opportunities for retention of learning
• Summarize and review key concepts regularly and often.
• Think-Pair-Share • Lunar Café Activity (small groups share knowledge retained throughout the unit, by recording facts on chart paper in a timed method. Students then rotate to the next chart to read peer’s comments and their own facts. This continues on until the rotation is complete.
Provide for remediation of unmet objectives
• Summarizing the lessons • Due to this being exposure and not mastery, remediation is not required, but the student not meeting the objectives is encouraged to review the products, websites, or reading materials
Enhance and enrich learning • Students want to continue the observation chart for an additional month
• Provide website list for students to access at home with parent permission
• Suggest additional reading materials
• Encourage continued learning through the use of school or public library resources
• Provide an open discussion forum for the sharing of new knowledge through Moodle
Task Analysis The task analysis portion of the design document takes the learning strategies presented in the previous section and turns them into tasks that the learners or users of the WBLS (Lunar Lab) will perform in order to complete this unit. There are several parts to the Task Analysis – a prioritized task list showing the core, important, and nice to have features – a functional requirement list again with a core, important, and nice to have list of requirements based on the importance or
frequency of use – a task flow diagram showing which tasks are on which screen in the WBLS.
Part I – General Task List
Initial Task List
Using the Lunar Lab WBLS the learner will be able to:
♦ Log into/out of Lunar Lab WBLS in Moodle
♦ Read text
♦ Navigate through Lunar Lab efficiently
♦ View and respond to discussion forum
♦ View videos
♦ View photos
♦ Post questions to instructor
♦ Access, open, and view links to outside resources
A. Prioritized Task List
(Key)
� Core Functionality
� Important Functionality
o Nice to have Functionality
Frequent by Many
� Log in and out of Lunar Lab WBLS
� Read Text
� Navigate through Lunar Lab efficiently
Frequent by Few
o Access, open, and view links to outside resources
I. Occasional by Many � View and respond to discussion forum
o View Photos
Occasional by Few
o Post questions to instructor
o View Videos
Part II – Functional Requirements
Initial Functional Requirements List
• Ability to log in and out of system
• Ability to scroll
• Ability to left click on mouse to select item
• Ability to right click
• Ability to navigate
• Ability to view text
• Ability to open, view, and access links
• Ability to type
• Ability to respond to questions posed by peers and
instructor
• Ability to compose a new post
• Ability to view video clips
• Ability to view graphics
• Ability to use spell check
• Ability to share enrichment materials with class
B. Function Requirement List
(key)
� Core Functionality
� Important Functionality
o Nice to have Functionality
Main Screen
� Ability to log in and out of system
� Ability to scroll
� Ability to navigate
� Ability to view text
� Ability to open, view and access links
� Ability to left click on mouse to select item
� Ability to view graphics
Discussion Forum (Moodle Link) Screen
(from Lunar Lab to Highland Local Schools Homepage, this is a
series of links to access Moodle for security reasons)
� Ability to scroll
� Ability to left click on mouse to select item
� Ability to navigate
� Ability to view text
(Highland Local Schools Homepage to Online
Academy Screen)
� Ability to scroll
� Ability to left click on mouse to select item
� Ability to navigate
� Ability to view text
(Online Academy to Log-in Page Screen)
� Ability to type in user name and password
� Ability to left click on mouse to select item
� Ability to navigate
� Ability to view text
(Log-in Screen to Class Page Screen)
� Ability to scroll
� Ability to left click on mouse to select item
� Ability to navigate
� Ability to view text
� Ability to open, view and access links
(Class Page Screen to Discussion Topic Screen)
� Ability to scroll
� Ability to left click on mouse to select item
� Ability to navigate
� Ability to view text
� Ability to open, view and access links
(Discussion Topic Screen to Add a new discussion
topic screen or response screen)
� Ability to navigate
� Ability to view text
� Ability to type
� Ability to respond to questions posed by peers and
instructor
� Ability to compose a new post
� Ability to use spell check
� Ability to scroll
(discussion topic screen to reply screen)
� Ability to right click
� Ability to navigate
� Ability to view text
� Ability to type
� Ability to respond to questions posed by peers and
instructor
� Ability to compose a new post
� Ability to scroll
� Ability to use spell check
Library Screen
� Ability to scroll
� Ability to navigate
� Ability to view text
My Bookmarks Screen
� Ability to scroll
� Ability to left click on mouse to select item
� Ability to navigate
� Ability to view text
� Ability to open, view and access links
� Ability to view graphics
� Ability to view video clips
Please see next page for Task Flow Diagram
C. Task Flow Diagram for Lunar Lab WBLS
Moodle On-Line
Discussion Forum
Moodle is access
through Highland Local
Schools Main Website
due to the need for on-
line security
My Bookmarks • Select links to
websites outside of
WBLS
• Select websites
used in WBLS
instruction
• Select video links,
interactive games,
virtual field trip,
digital photos
Main Page Screen
• Select Moodle On-Line
Discussion Forum
• Select Book Library link
• Select My Bookmarks
link
• View Fun Facts
• Read Unit overview
Book Library
View titles of books
related to unit
Module 3.3 Scenario
Task for both scenarios: Students are to check the phase of the moon and complete
the observation chart showing the phase of the moon for that day, post their
response to the question posted on the class discussion forum. The posted question
is, “Given today’s moon phase, when do you predict the next new moon will
happen?”
Scenario #1 – Student is in a hurry and wants to be as efficient as possible in his or her
work effort
Background:
Student A knows that it is permissible to complete this task in class the following day
because the teacher cannot require students to perform computer-based tasks at home.
During science class, when the teacher shows the previous evening’s moon phase, student
A quickly traces her nickel and shades in the illuminated portion using her white pencil.
Student A listens to class discussion and then correctly labels the current phase. After
this, the class goes to the computer lab to complete the Moodle discussion.
Structure of Tasks:
Student A walks to the computer lab with the teacher. Student waits until the teacher
assigns her to a computer. Student sits down. Turns on the computer. Waits for it to
boot up. Logs in to the system using the class username and password. Logs onto
Internet. Types in the Lunar Lab URL. Double left clicks the Moodle Discussion Forum
icon. The next screen to appear is the Highland Local Schools homepage. Student left
clicks on the On-Line Academy tab at the top of the homepage. The next screen to
appear is the “Available Courses in On-Line Academy.” Student A scrolls down to
locate her class. Student left clicks (once) on correct class name. The next screen is the
student log in screen. Student types in user name, hits the tab key to move cursor to
password field. Raises hand, cannot remember password. Teacher gives Student A the
password. Types in password. Using the mouse, student puts cursor over log in button
and left clicks (once). On the next screen the student reads the discussion topics, finding
the one to be completed that day. Using the mouse, the student moves the cursor to that
topic and left clicks (once) on that topic. On the next screen, the student reads the
question and can see who has participated so far in the discussion. Using the mouse the
student places the mouse over the “Add a new discussion topic” button. Left click
(once). Cursor appears in the subject field and student types the subject – “The next new
moon.” Student moves cursor with the mouse to the first line of the composing field and
begins to type a response to the posted question. Since student did not view the moon the
previous night, or play close attention to the classroom discussion, it takes quite a while
for Student A to form the response. Once the response is complete, the student uses the
mouse to place the cursor on the right scroll bar and while holding the left button down
on the mouse, and scrolls down to the bottom of the page. Student moves the cursor to
the “Post to Forum” button using the mouse and left clicks (one) to post the reply to the
forum. On the next screen student moves the cursor to the “Continue” button, left clicks
(one) to return to the discussion page. Student scrolls to the bottom of the page using the
right scroll bar and left mouse button. Then she moves the cursor to the “log out” button
(beside her name), left clicks (one) and logs out of the class page. Then she moves the
cursor to the top right of the Highland On-line Academy page and left clicks the red “X”
to exit out of the academy. Using the mouse, student places the cursor on the red “X” to
exit out of the Highland homepage. Using the mouse, student places the mouse over the
start button, left clicks on it. Using the mouse, student places the mouse over the shut
down button, left clicks on it. Using the mouse, student places the mouse over the shut
down button, left clicks on it to shut down the computer.
II. Scenario # 2- Student wants to have an efficient and effective collaborative work
with his/her teammates in the WBLS and achieving at a high level of performance.
Background:
Student B has always put forth his/her best effort in fully completing class assignments
neatly in order to get the most out of their education. When presented with today’s class
assignment, Student B was excited to complete the work!
At home, Student B makes a point to observe the moon in person by going outside during
dark hours and viewing the moon. After observing the way the moon appears in the sky,
Student B takes out the moon observation chart and completes the drawing for that day.
Students B then colors in the illuminated portion of the moon as seen and relates this to
what has been learned in class about the different phases. Student B believes that the
current moon phase is a waxing moon in the first quarter. After labeling the phase
underneath the drawing of the moon on the calendar, Student B logs onto the class
website to check the current moon phase. Student B navigates from the main page to the
mybookmarks.com site to select the website previously posted by the classroom teacher.
Student B accesses the website, http://www.briancasey.org/artifacts/astro/moon.cgi,
opens and views the current moon phase and confirms that what was observed and drawn
was a waning crescent. Student B also notices that the website says the current moon
phase is 13% full and states that it is 2 1/2 days before the New Moon. Student B is
excited to return to school and share this new knowledge with classmates and the teacher.
Structure of Tasks:
Student B walks to the computer lab with the teacher. Student quickly chooses a
computer and sits down. Turns on the computer. Waits for it to boot up. Logs in to the
system using the class username and password. Logs onto Internet. Types in the Lunar
Lab URL. Double left clicks the Moodle Discussion Forum icon. The next screen to
appear is the Highland Local Schools homepage. Student left clicks on the On-Line
Academy tab at the top of the homepage. The next screen to appear is called “Available
Courses in On-Line Academy.” Student scrolls down to locate her class. Student left
clicks (once) on correct class name. Next screen is the student log in screen. Student
types in user name, hits the tab key to move cursor password field. Types in password.
Using the mouse, student puts cursor over log in button and left clicks (one). On the next
screen the student reads the discussion topics, finding the one to be completed that day.
Using the mouse, the student moves the cursor to that topic and left clicks (once) on that
topic. On the next screen, the student reads the question and can see who has participated
so far in the discussion. Using the mouse the student places the mouse over the “Add a
new discussion topic” button. Left click (once). Cursor appears in the subject field and
student types the subject – “The next new moon.” Student moves cursor with the mouse
to the composing field and begins to type her response to the posted question. Since
Student B accessed the website the previous night, the answered response was arrived at
quickly and the student was able to efficiently complete the assignment. Student shared
additional information learned from the research, enriching the learning experience of
other students. Once the response is complete, using the mouse and the holding the left
button down on the mouse, student places the cursor on the right scroll bar and scroll
down to the bottom of the page. Student moves the cursor to the “Post to Forum” button
using the mouse and left clicks (once) to post the reply to the forum. On the next screen
student moves the cursor to the “Continue” button, left clicks (once) to return to the
discussion page. Student then views other class members who have replied and using the
mouse places the cursor over the discussion topic of a classmate and left clicks. This
brings up the discussion response of the classmate for student b to read. Student B views
text, carefully reading classmate’s response and chooses to reply, creating a collaborative
learning environment. To reply, Student B moves the cursor to the word reply on the
right side of the response page left clicks. This brings up the composition page. Student
places cursor in the composing field using the mouse. Student types a thoughtful and
thorough response to classmate’s posting. Student B poses a question to the classmate to
continue the collaborative discussion. Student B moves the cursor to the spell check icon
in the tool bar and left clicks (one) the icon. Student views the task window and makes
the decision to either accept or ignore the suggested spellings. Student moves the cursor
to the “Post to Forum” button using the mouse and left clicks (once) to post the reply to
the forum. On the next screen student moves the cursor to the “Continue” button, left
clicks (once) to return to the discussion page. Student B repeats this process until the end
of the class period creating many collaborative learning opportunities with classmates.
This becomes an effective web-based learning system and the student is achieving at a
high level of performance. Student scrolls to the bottom of the page using the right scroll
bar and left mouse button. Then she moves the cursor to the “log out” button (beside her
name), left clicks (one) and logs out of the class page. Then she moves the cursor to the
top right of the Highland On-line Academy page and left clicks the red “X” to exit out of
the academy. Using the mouse, student places the cursor on the red “X” to exit out of the
Highland homepage. Using the mouse, student places the mouse over the start button,
left clicks on it. Using the mouse, student places the mouse over the shut down button,
left clicks on it. Using the mouse, student places the mouse over the shut down button,
left clicks on it to shut down the computer.
Summary:
We decided to use the same task for both scenarios so that we could compare the tasks
that the students would do. The scenarios are detailed, so that readers can understand
what the users are doing throughout the task. The objective of “predicting when the next
new moon will occur” is reached. This is a sub-objective of the main objectives, “Given
websites, personal observation chart, and other supporting information, the students will
be able to describe how the moon appears each day, identify the five main phases of the
moon, predict the next moon phase, and demonstrate the correct order of the moon
phases”.
Module 3.4 Specification Report
The link to our Wiki page is:
http://mrsweber.pbworks.com
Module 4.1 – Expert and User Review
Part I. Evaluation Planning Stage/with modifications for formative
*Modifications made to this document after the design/development
stage prior to field testing with potential users, content experts, and web
design experts are in italics.
The purpose of this stage is to plan for formative and summative evaluation of the
WBLS product. Formative evaluation is conducted during the design and development
phase and provides feedback to the designer. This feedback can give the designer
suggestions for improvement prior to full implementation of the WBLS and occurs in a
recursive loop allowing for many modifications in the design. The summative evaluation
is conducted after the full implementation of the WBLS with the user.
Evaluation Criteria Explanation Data Source
Effectiveness (mastery of
goals and success of WBI)
Determine if actual learning
has occurred with the
WBLS
Student mini-book
Observation Calendar
Discussion participation
Efficiency (delivered in a
timely, or cost saving
manner)
The WBLS was developed
well before the unit was to
be taught
Document time to prepare
materials and evaluation of
student products
Documentation of student
time on WBLS activities
Appeal (gain and
maintain learner attention
and interest; usability
The WBLS was easy to use
for second graders; the
colors were appropriate and
text size and color were
pleasing. New components
were introduced throughout
the project to maintain
student interest. Determine
ease of navigation of
bookmarked sites and
discussion forum
Instructor and student
opinions, review by peers;
interview of students to
determine degree of
difficulty in accessing
websites and participating
in the on-line discussion
forum
• Formative Evaluation Plan
Formative evaluation is conducted during the design and development phase of the
WBLS. Feedback is given to the designer for improvement to the design to make it
easier to use or more visually appealing. This stage is important to the overall
effectiveness of the product as it gives the designer an opportunity to work out any bugs.
Formative evaluation will take place several times with a one-to-one tryout and small
group. Members of these groups will include the instructor’s family and/or other teachers
or friends willing to participate.
A final formative evaluation took place prior to full implementation to determine if the
WBLS met the goals of the designers, was visually appealing, was easy to navigate, if the
content was appropriate for the users, and if the design was appropriate.
Formative Evaluation Plan:
Who are the stakeholders?
Primary – Designer, Instructor
Secondary – Students, Administration, Parents
What is being evaluated?
The design of the WBLS, the websites, discussion forum, materials
Who are the evaluators and reviewers?
Grade level teachers, special education teachers, friends willing to help
What are the evaluation methods?
Questionnaire (see next section)
What types of decisions need to be made as the WBLS design plan and
prototype are develop and revised?
Are the suggestions for revision valid and will the suggestions make the
WBLS more effective.
Formative Evaluation Plan Questionnaire/Formative Evaluation Questions that were
added to this are in italics
Evaluation Criteria Sample Questions Methods
and Tools
Effectiveness
• Goals
• Content
• Technology
• Message
Design
Does the goal match the ODE indicator?
Are the goals and objectives clear and achievable?
Are the goals and objectives appropriate for the
method of delivery?
Do the activities promote learning?
Is the information complete, covering the content
properly?
Are sites accessible through the school server
(filters)?
Are the sites accessible through the WBLS?
Was the time frame of the WBLS appropriate?
Are directions clear and concise?
Interview
Interview
Observation
Observation
Field Test
Interview
Efficiency
• Goals
• Content
• Technology
• Message
Design
Are goals clear and concise?
Do you understand what to do for each of the lab
activities?
Is it appropriate to our grade level?
Is content information clearly and concisely
presented?
Are websites and discussion forum easy to navigate?
Is the structure of the content clear?
Interview
Interview
Observation
Interview
Observation
Interview
Appeal
• Goals
Are students aware of the goals of the WBLS?
Interview
• Content
• Technology
• Message
Design
Is the content interesting?
Are there any typographical, punctuation, grammar,
or spelling errors?
Is navigation easy?
Do you like the colors on the pages?
Did you like the pictures?
Do the pictures help you understand what to do?
Should the graphics be on the other pages too?
Is the vocabulary appropriate for the user?
Are there limited distractions on the sites?
Are the colors, typeface, font emphasis used
appropriated to enhance learning?
Does it have good navigational design?
Interview
Observation
Interview
Observation
Interview
Observation
The designer/developer will receive the results of the formative evaluation
through a written summary of the interviews and observation of the users, content
experts, and web design experts.
Background of Users:
The WBLS was implemented with 2 females age 8. They just completed second grade.
Both were familiar with Moodle and both had the ability to type and use basic
navigational skills.
Background of Content Experts:
One content expert is a male, age 50. He is an amateur astronomer and has a telescope
in his back yard. Another is a female, age 49 and is currently a teacher with certification
in general science and biology 7-12. The third expert is a college student studying in the
food and agriculture and environmental sciences department, and is 20 years old.
Background of Web Design Expert:
One web design expert is a female, age 27. She is a master’s student seeking her
master’s in instructional technology. The second web design expert is a
computer/technology teacher in a suburban district. She is 45 years old and has been
designing web pages for 2 years. The final web design expert is 18 years old, and has
been using the Internet for many years. He is conversant in navigation and design of web
sites, and will be majoring in computer technology in the fall.
Part 2. Relevant Documents/Guides/Instruction
A. Questions for Web Design Expert Review:
Are the tasks easy to understand?
Are the directions on the WBLS easy to understand?
Do the links work?
Is the layout visually appealing?
Are there any typos, grammar errors, punctuation, or spelling errors?
B. Questions for Content Expert Review
Are websites appropriate for second grade?
Is the content on the websites correct?
Are the book titles appropriate?
C. Questions for Users Review
Were the directions easy to follow?
Did you like the pictures, did they help you understand what to do?
Are the colors okay, or would you like it to be more colorful?
Did you understand what to do for each activity?
Should there be graphics on each page that is part of the Wiki?
Anecdotal Observations (notes taken while students are navigating
through the WBLS).
3. Results/Findings from the Expert Reviews a. Web Design Experts
Interview/Observation Results (conducted simultaneously)
• Design is clean, kid friendly
• Site is easy to navigate
• Move the Lunar Library and My Bookmarks links closer together so
there is not so much white space
• Add a return to front (home) page link on the library page and the
reflection page so users can easily return to the home page
• Change TeacherTube video to open in a separate page
• Add OR to Lab 1 so students know they can do either or both activities
• Add directions how to get to the appropriate site for lab 1 (1,2,3 or
bullets
• Add directions how to get to the appropriate site for lab 2 (1,2,3 or
bullets
• Add wording on lab 4 (library activities) to tell students how to get
back to Lunar Lab
• Spelling error on Reflection Page – should be ‘choose’
• For students on IEPs, you could print off the front page and have them
check off the boxes as they complete a task.
• Change wording at the bottom of the home page, I thought you could
click on it
• Tell students how to log off the lab
• I like how the bookmarks page had other activities to do, not just the
sites that you wanted them to go to.
• Like the graphics
• Just the right balance of text and graphics
• Like the set up with the boxes for the activities
b. Content Experts
Interview/Observation Results (Conducted simultaneously)
• Found error on moon fun fact in the side bar
• Directions are easy to follow
• The site was attractive and easy to find the content
• The websites chosen were appropriate for second grade students
• Websites on the bookmarks site had varied information for the diverse
learners in second grade
• Library books listed were adequate
• Comment: “When I logged on to the site, I was confused as to what to
do. You should add something on the front page so parents/students
know what you expect them to do with your site.
c. Potential Users
Interview Results:
� Both students liked the pictures, but one did not like the bookworm
picture.
� Both students like the colors, one wanted more colors on the main
page though.
� Neither found anything confusing about the site.
� Both understood the lab activities
� There were two links that did not work
� There should be graphics on all the pages on the Wiki
Observation Results:
� When reading the directions aloud, the highlighted information
was spoken in a louder tone, so she understood that this was
important information
� Because of their familiarity with my bookmarks, they wanted to go
to all the sites, see if there were any games (this was a distraction)
� Two of the website links did not work –my bookmarks and the
reader’s theatre
� Both students were able to navigate to Moodle through the various
screens due to privacy/security concerns.
� Both students were familiar with typing and basic navigational
skills necessary to complete the WBLS effectively and efficiently.
� One student was confused by the orange moon at the top and
wanted to click on it.
4. Ideas for Modification for the Design Products
Based on feedback received from the experts and users we will modify the WBLS
as follows:
� Fix the two links that did not work
� Add the bookworm graphic to the library page
� Add a graphic to the Moon Message page
� We will not add more colors or graphics to the main (front) page as we
feel this will be too distracting.
� Add wording to the lab 1 and lab 2 to explain to open the moon phases
folder
� Change wording at the bottom of the front page from “End Lunar Lab”
to “When you have completed Lab 5, you are finished with the Lunar
Lab Activities. You did it! Remember to log out of this site by
clicking on the words “sign out” in the upper right hand corner.
� Move two graphics and links (library and bookmarks) closer to the
center to make the triangle more compact and reduce white space
� Fix spelling error
� Fixed TeacherTube link so it opens in its own window outside of
Lunar Lab
� Added wording on Lunar Library to tell students how to return to
Lunar Lab when the are finished viewing the video or reading the
reader’s theatre
� Moved links in moon phases folder to be first or second choice
depending on lab
� Add wording to Labs 1 and 2 to direct students on how to access the
appropriate link
� Add the “click here to return to home(front) page” on the library and
reflection pages
� On lab 1 add the word or so students would know they could do either
activity
� Change the daily fun fact to be less confusing when it is read
� Added wording to front page to explain what to do with the sites,
activities to avoid confusion.
Module 4.2 Usability Test
1. Usability Test Plan
The usability test took place at the home of Penny Reed on June 29, 2009
from 1:00 to 3:00 pm. There were two participants, both were eight years
old. Both were familiar with Moodle, MyBookmarks, had basic
keyboarding and navigational skills. These participants were selected
based on their availability, proximity to the testing site, and felt
comfortable with the instructor and the testing site.
The learning goal for the usability test is to discover the usefulness,
learnability, efficiency, and satisfaction level of the WBLS. There were
several computers with Internet access available for the users, which were
connected to the Internet and ready for them to use. To prepare the uses,
we gave them a brief lesson of the objectives for using the site. We also
discussed with them why they were chosen to test out the site.
During the usability test, data was collected through observation and an
interview. We took anecdotal records of our observations and their
comments. After students finished the activities, interviewed each user
individually so that their answers would not be tainted.
2. Findings/Results of the Usability Test
� From observations we found that the link to the Reader’s Theater
and Mybookmarks didn’t function.
� From observations we found that the students were able to
remember how to navigate to the Moodle page.
� From observations we found that the students remembered how to
sign into and out of Moodle.
� From observations we found that students remembered how to
navigate the mybookmarks site (once we put them on the site)
� From observations we found that there was little need for them to
seek assistance to either navigate through the site or complete the
activities.
� From observation we found that the intended pace of the class was
appropriate for the age level and time given.
� From observation we found that the readability of the content was
appropriate for this grade level.
� From the observation we found that one student wanted to click the
orange moon thinking that it was a clickable area.
� From observation we found that students wanted to access the
interactive games instead of sites necessary for the activity.
� During the observation the designers realized that they neglected to
put their names on the front page of the WBLS.
� From observed behavior, we discovered that the students were
eager to participate in the usability test. They sat down at the
computer and quickly sat down to work, exploring the site, testing
out the links.
� From the interview, we learned that the topic was interesting to
them.
� From the interview, we learned that the site was appealing (design
and colors were appropriate). There were limited graphics and no
distractions.
� From the interview, we learned that students liked the individual
aspect of the WBLS.
� From the interview, we learned that students wanted to have
graphics on all the pages instead of just the home page.
3. Suggestions for Improvement
� Based on the results of this usability test we will:
o Fix the links that did not work
o Add a graphic to the library page
o Add a graphic to the moon message page
o Add the designers names to the front page of the WBLS
� Based on the observed behaviors of the participants we
discovered that the WBLS did not create frustration and
successfully met our objectives.
� Strengths:
o Students were eager to complete the task
o The site was appealing and easy to navigate
o Users were familiar with several aspects of the site
(Moodle and mybookmarks)
o Students were efficient in using the site, accomplishing
the age appropriate tasks with ease.
� Weakness
o The weakness was the limited number of users for the
test.
o Another weakness was performing this test during the
summer. Had this been implemented during the school
year, students would have had the benefit of face to
face instruction.
Because of the time frame, the test was conducted before complete testing of the
prototype was accomplished
Module 4.3 Implementation Plan and Summative Evaluation
a) Implementation Plan
The purpose of the Implementation Plan is to determine the timeline, allocate the
duties of personnel, and outline learning tasks and associated facilitation
strategies. This plan is based on the designers/instructor experiences and
suggestions gained from the usability test. The plan allows for the designers to
reflect upon the design if the learning objectives have been achieved, if a learning
community has been created and sustained in the WBLS.
i) Time Schedule of the Implementation
• Design and development of activities including maintaining the
Wiki and Moodle sites will be completed during the
instructor’s planning times and/or own time-
Both sites can be accessed from the instructor’s home
Estimated time commitment required is 15 hours
Instructor Duties Estimated Time
Requirement
Set up student
usernames/passwords
0.5 hour
Develop and/or collect online
materials, activities, and
discussion questions
7 hours
Teach course (content, learning
tasks, instruction on navigating)
2.5 hours
Provide feedback on activities
both off and on line
4 hours
Miscellaneous
activities/troubleshoot technical
problems
1 hour
Total hours for unit 15 hours
• Budget concerns will not need to be addressed since the
District has already implemented a Moodle site as part of the
school’s website and the class Wiki site is a free Internet tool-
this would also include technical support at no cost
• Lunar Lab activities are projected to cover a four week time
period due to the implementation of the project to observe and
record moon phases for one month
• All other tasks in Lunar Lab are designed to be completed
during two weeks
ii) Personnel and Duties Allocation
WBI Instructor:
• Design, develop, and maintain class Wiki website
• Implement and maintain District Moodle classroom website
• Develop personal biography section-introduce the instructor
• Provide unit objectives, course goals, and assignment
requirements
• Maintain site information to keep it current and that all links
are active
• Establish rapport and maintain communication between
learners and instructor
• Review content to check for accuracy and completeness
• Provide students with user names, passwords, and
instructions on accessing and navigating Wiki and Moodle
sites
• Design place, time, and topics for discussion
Administrative and Technical Personnel
• Provide the infrastructure needed to institute and maintain
District Moodle site
• Maintain network infrastructure
• Maintain computer equipment including computers,
monitors, web cams, microphones, mouse, LCD projectors,
and all connecting wiring
• Provide WBI instructor with technology support for
instruction
• Update software/hardware when necessary
• Enroll class members as participants on the class Moodle site
iii) Learning Tasks V.S. Facilitation/Strategies
The WBI is based on for a second grade class of 28 students
• Initial instruction of the WBI will take place during regular
class time
III. Learning Tasks IV. Facilitation Strategies Describe movement of the
moon in relation to the earth and
sun (learn content knowledge)
Pose questions for accessing prior
knowledge and as a discussion starter;
demonstrate with models/students the
earth-sun-moon movements, encourage
class discussion with additional
questions; summarize main points
Learn content knowledge Pose question before viewing
UnitedStreaming video to provide a
purpose on content to be discussed;
provide quiet uninterrupted viewing of
video; follow up with class discussion to
summarize major concepts covered
Read books to gain basic
content knowledge about the
moon phases
Read grade level fiction and nonfiction
books to stimulate class discussion and
acquire moon content knowledge;
provide grade level books for students to
read independently or in small groups;
provide opportunities for discussion
and/or reflection about knowledge
learned through readings; provide
graphic organizers to organize content
knowledge
Demonstrate how to access and
navigate Lunar Lab website,
activities, and how to participate
Model behavior and directions
necessary for accessing and using the
WBLS through the use of a LCD
in discussion forums projector and Internet websites; give
learners opportunity to ask questions;
inform participants of expectations for
completing activities; provide time for
learners to practice accessing websites
and activities to avoid possible
frustrations later when working
independently
Complete the observation
calendar, illustration, and
labeling the phase
Explain how to complete the
observation calendar, model one day’s
response by viewing website together,
completing drawing of the moon, and
labeling the phase
View website to complete first
drawing of the moon phase
Provide a observation chart with current
moon phase completed for student self-
checking
Complete responses in online
class discussion forums (Wiki
discussion & Moodle
discussion-threaded responses)
Asynchronous discussion-keep on track
and relevant by providing students with
guidelines for participating and by using
password protected forums; state &
model expectations for responses;
provide opportunities that allow learners
to assist each other with technical
support; personalize feedback by
responding to individual postings; use
threaded discussion (Moodle) to permit
participants to share experiences and ask
questions among themselves; provide
netiquette guidelines for the discussion
responses
View video on why the moon
appears to change shape
Provide a discussion response to verify
viewing of video
Read the Reader’s Theater
about moon phases
Provide own paper copy of Reader’s
Theater with character part highlighted;
form small groups to practice for live
performances
Work collaboratively and as
part of a group
Allow for a mix of small-group and
whole-group activities for peer support
on problem-solving tasks; share
cooperation strategies
Have opportunities to
differentiate in learning
activities
Include independent tasks; additional
activities; extra informational sources;
be flexible with students abilities and
responses
Acquire additional technology
skills
Provide opportunities for interacting
with various websites (Moodle, Wiki,
Mybookmarks); applying a variety of
keyboarding skills; practice login and
logging out procedures and keeping
passwords secure
Practice time management skills Provide ample time to complete grade
appropriate tasks and inform students of
how long the assignment should take;
allow for extra opportunities to finish
work if needed; Provide links that work
Summative Evaluation F2F
Class Discussion (reflection)
Correct any misconceptions about the
moon that may have occurred during the
online learning & activities; summarize
major concepts in discussions and
readings as a way to follow up on unit
objectives; ask students to reflect on unit
to share opinions of learning through
this type of activity; provide an
opportunity for students to share if they
enjoyed completing this type of activity
and would they want to learn this way in
future lessons
b. Summative Evaluation Plan
The summative evaluation takes place after the final implementation of the Lunar Lab
WBLS. The purpose of the summative evaluation is to determine the effectiveness of the
final product, if learning goals have been met, if stakeholders were satisfied with its
outcomes, if value has been added, and if students would like to do this type of learning
strategy again. The plan also lays the groundwork for the collection of data and its
analysis and the reporting of the findings to the stakeholders.
i) Summative Plan
Who are the Stakeholders? Primary: The Designers/Instructors
Secondary: The School District
The Students
What is being evaluated? Student Learning in Lunar Lab
Through the context of effectiveness,
efficiency and appeal
Who are the evaluators? o The designers (internal)
o Other 2nd grade teachers,
principal(external)
o The users of Lunar Lab (2nd
graders)
What are the evaluation methods and tools o Informal interview, survey,
observation
o Products produced during face-to-
face class meetings
o Discussion forum postings
How are data collected and analyzed? o Interview data collected with use of
tape recorder and transcribed
(students)
o Pen and paper survey and interview
(teachers, administrator)
o Products are graded by the
instructor and discussed with students
to clear up any misconceptions
o Microsoft Excel will be used to
analyze the data. Common threads
among short answer responses will be
looked for
How are evaluation results reported? o Designers discuss summative
evaluations to determine effectiveness,
efficiency and appeal of Lunar Lab
o Using Excel, tables and charts of
responses will be made to determine
frequency of responses, trends in the
data, and any outliers
ii) Questions to ask Evaluators during Summative Evaluation
The following table lists the questions that to be asked during the summative evaluation
of Lunar Lab. There are three areas of consideration – Instructional Goals and Content,
Learners and Instructors, and Context.
Area of Consideration Questions to be asked
Instructional Goals and Content Effectiveness
o Could students accurately predict when a moon phase
would occur?
o Could students order the phases of the moon
correctly?
o Were students able to communicate their thoughts
concisely in the discussion forums to demonstrate
learning?
o Can students properly identify the five phases of the
moon?
Efficiency
o Were the goals of Lunar Lab achieved?
o Could students access Lunar Lab from school/home
and understand the activities?
Appeal
o Do students want to participate in another WBLS
learning activity?
Learners and Instructors Effectiveness
o Were the stakeholders pleased with the outcomes,
products, and activities?
o Do users feel Lunar Lab was worth the time spent on
it?
o Were the activities and informational websites
informative?
Efficiency
o Was the Wiki easy to use?
o Were there some design features that made the Wiki
difficult to use?
o Was Moodle easy to use?
o Did Moodle lend itself to collaborative learning?
Appeal
o Did users find Lunar Lab interesting?
o Would users want to participate in another WBLS
learning activity?
o Did the graphics help you understand what to do?
Context Effectiveness
o Was the computer lab and/or library a good place to
work on this?
o Did you like doing Lunar Lab as a station in the
classroom?
o If you used it at home, did it work as well as it did at
school?
o If you used it at home, were you as successful in
completing the activities as you were at school?
Efficiency
o Was the site easy to access?
o Did all the links open?
o Were you able to navigate back to Lunar Lab from
the external links?
Appeal
o Were the authors clearly identified?
o Did the graphics and photos enhance or detract Lunar
Lab?