The following notes were prepared by Sebnem Gür, Umut Lagap, and Virginia Couch

to share what we learned while preparing our first quiz using CMS on 21.04.20.

We hope this will be helpful to other instructors in our faculty.

Step 1 - Adding Quiz and General Information

Go to the course page on CMS.

On the course dashboard in CMS, while editing

mode is on, click ‘Add an activity or resource’.

Choose ‘Quiz’.

In the next screen a title and a description should

be provided for the quiz.

Check the box below if you want the description to

be visible on the course page.

NOTES ON THE CMS QUIZ INTERFACE

NOTES ON THE CMS QUIZ INTERFACE

1

Step 2 - Timing

In the Timing section check ‘Enable’

then select the time period to take the

quiz (Open the quiz – Close the quiz).

This is NOT how long the quiz takes

but the time within which students can

take the quiz.

Set the duration of the quiz using ‘Time

Limit’.

‘When time expires’ lets you choose to

auto-submit (accept) or reject

unfinished attempts when time expires.

A ‘Submission Grace Period’ can also

be set.

Submission Grace Period: “When time expires” is set to 'Allow a grace period to submit, but not

change any responses', the grace period is the amount of extra time that is allowed.

Step 3 - Grade

For Grading there are two settings, ‘Grade

Category’ and ‘Grade to Pass’.

‘Attempts Allowed’ defines how many times a

student can take the quiz, within the Open and

Close times, within the given time limit (see

Timing).

» To prevent drawbacks caused by connectivity

issues, it is advised to choose ‘Attempts allowed’

greater than one. This means that the students will

be able to take the quiz more than once. However,

each attempt begins with no answers selected.

If you have allowed multiple attempts, this requires

selection of a ‘Grading Method’.

NOTES ON THE CMS QUIZ INTERFACE

2

Grade Category: This setting controls the category into which this activity's grades are placed in the

gradebook.

Grade to Pass: This setting determines the minimum grade required to pass. The value is used in

activity and course completion and in the gradebook, where pass grades are highlighted in green and

fail grades in red.

Grading Methods: When multiple attempts are allowed, the following methods are available for

calculating the final quiz grade:

Highest grade of all attempts

Average (mean) grade of all attempts

First attempt (all other attempts are ignored)

Last attempt (all other attempts are ignored)

Step 4 - Layout

In Layout, ‘New Page’ setting shows the number

of questions that will be visible in each screen.

» During our quiz slow internet connection caused

time loss while navigating through the pages.

Having multiple questions in pages, hence

reducing the number of pages to load, can yield

better performance.

‘Navigation Method’ defines whether the student

can skip between pages.

» Notice that when ‘Sequential’ is chosen for

navigation method, students can not go back or

forward between questions. A warning to the

students before the quiz is therefore essential.

.

NOTES ON THE CMS QUIZ INTERFACE

3

Step 5 – Question Behaviour

The ‘Shuffle within questions’ setting changes the

order of the answer choices. This setting only

applies to questions that have multiple parts, such

as multiple choice or matching questions.

» Multiple choice questions with answers such as

‘Numbers I & II’ or ‘All of the above’ should be

rephrased since the placement of the choices will

be random.

‘How questions behave’ has multiple settings (see

next page for details). ‘Adaptive mode (no

penalties)’ is the setting that allows students to

change their answer any number of times for a

question before advancing.

OTHER NOTES ABOUT STEP 5 - QUESTION BEHAVIOUR:

Deferred feedback: Students must enter an answer to each question and then submit the entire quiz

before anything is graded or they get any feedback.

Adaptive mode and Adaptive mode (no penalties): Allows students to have multiple attempts at any

question before moving on to the next question. This behaviour requires that the "Whether correct"

box is ticked under "During the attempt" in the "Review options" section.

Manual grading: Used for essay questions, but this behavior allows you to choose to have every

question in the quiz manually graded.

Interactive mode: After submitting one answer, and reading the feedback, the student has to click a

'Try again' button before they can try a new response. They can be given hints to help them. Once the

student has got the question right, they can no longer change their response. Once the student has got

the question wrong too many times, they are graded as wrong (or partially correct) and are shown the

feedback and can no longer change their answer. You can set up the quiz to give different feedback

after each try the student makes. The number of tries the student gets is the number of hints in the

question definition plus one.

Immediate feedback: Similar to interactive mode, in that the student can submit their response

immediately during the quiz attempt, and the response can be graded. However, they can only submit

one response, and they cannot change it later.

Deferred feedback or Immediate feedback with Certainty-based marking (CBM): With CBM, the

student does not only answer the question, but they also indicate how sure they are that they got the

question right. The grading is adjusted by the choice of certainty, so students have to reflect honestly

on their own level of knowledge in order to get the best mark.

NOTES ON THE CMS QUIZ INTERFACE

4

Conditional questions: If using the Interactive with multiple tries or Immediate Feedback behaviour,

and with the navigation method set to 'Free', it is possible to make the display of a question dependent

on a previous question being answered first. In this case, the question editing page will display

padlock icons to the right of each question.

Step 6 – Review Options

In the ‘Review Options’ section, among other settings, it is possible to show/hide the marks (score)

and right answer during four different stages of the quiz.

» When multiple attempts have been chosen (in Grade section), the ‘Right Answers’ and

‘Whether Correct’ box should be unticked, except in the ‘After the quiz is closed’ stage.

» Hiding the ‘Marks’ at all stages is advised in case of any problems with the questions.

The attempt: Whether the student can review the attempt at all.

Whether correct: This covers both the textual description 'Correct', 'Partially correct' or 'Incorrect', and

any coloured highlighting that conveys the same information.

Marks: The numerical marks for each question, and the overall attempt score.

Specific feedback: Feedback that depends on what response the student gave.

General feedback: General feedback is shown to the student after they have completed the question.

Unlike specific feedback, which depends on the question type and what response the student gave, the

same general feedback text is shown to all students.

You can use the general feedback to give students a fully worked answer and perhaps a link to more

information they can use if they did not understand the questions.

Right answer: an automatically-generated summary of the correct response. This can be limited, so

you may wish to consider turning this option off and explaining the correct solution in the general

feedback for the question.

Overall feedback: The feedback given at the end of the attempt, depending on the student's total mark.

NOTES ON THE CMS QUIZ INTERFACE

5

Step 7 - Appearence

Although we chose to display the picture, no

pictures were seen during the quiz.

Step 8 - Extra Restrictions on Attempts

Under ‘Extra Restrictions on Attempts’ it is

possible to set a password, and under

‘Browser Security’ it is possible to set a safe

browser that blocks every other window from

being seen.

» Middle East Technical University offers

‘Safe Exam Browser’ to be used. Note that

while using Safe Exam Browser it is not

possible to use an online dictionary or send

messages to a Teams meeting.

Since our quiz was open-book, we have no

experience with Safe Exam Browser. When

using Safe Exam Browser, are students visible

on Teams?

If "Full screen pop-up with some JavaScript security" is selected,

The quiz will only start if the student has a JavaScript-enabled web-browser.

The quiz appears in a full screen popup window that covers all the other windows and has no

navigation controls.

Students are prevented, as far as is possible, from using facilities like copy and paste.

NOTES ON THE CMS QUIZ INTERFACE

6

Step 9 - Overall Feedback

‘Overall feedback’ is text that is shown after a quiz has been attempted. Specify additional grade

boundaries to show text such as ‘Good job!’, depending on the grade.

Step 10 - Common Module Settings

» Under ‘Common Module Settings’ a crucial

one is the ‘Visible’ setting. When set to

‘Show’, the quiz will be visible for the

students in the course dashboard but not

accessible until the Open Time.

There are ‘Group mode’ and ‘Grouping’

settings to divide the students into groups and

arrange their access to the quiz (see below).

We did not use the Group settings.

NOTES ON THE CMS QUIZ INTERFACE

7

Finish the settings for the quiz by selecting

‘Save and display’ in the blue box at the

bottom.

In the next screen choose ‘Edit Quiz’ to add

quiz questions.

“Group Mode” has 3 options:

No groups - There are no sub groups, everyone is part of one big community.

Separate groups - Each group member can only see his/her own group; others are invisible.

Visible groups - Each group member works in his/her own group but can also see other groups.

The group mode defined at course level is the default mode for all activities within the course. Each

activity that supports groups can also define its own group mode. However, if the group mode is

forced at course level, the group mode setting for each activity is ignored.

Grouping: A grouping is a collection of groups within a course. If a grouping is selected, students

assigned to groups within the grouping will be able to work together.

NOTES ON THE CMS QUIZ INTERFACE

8

Step 11 – Adding Questions

After creating a quiz and determining the quiz settings mentioned above, questions should be added by

clicking on the Add button. The system provides three ways to add questions: a new question, from a

question bank, and a random question.

** Marking Shuffle option mixes questions for each student while taking quiz.

A- Adding a new question

Select a question type > Click on “Add”

CMS allows to users to create

different types of questions. In

this example, multiple choice

type questions will be added.

NOTES ON THE CMS QUIZ INTERFACE

9

Save in category : shows which category the new question is going to be saved in

Question name : a name should be applied to the question to find the question in “Question Bank”

If there is only one correct answer, “One

answer only” should be selected.

“Shuffle the choices” mixes the answer

choices for each student.

NOTES ON THE CMS QUIZ INTERFACE

10

After saving a question, it is added to “Question Bank” automatically.

In the “Answers” section, the grade should be defined as 100% if there is only one correct answer.

5 or more choices can be added to any question. If you have less options, leave the other options

empty.

NOTES ON THE CMS QUIZ INTERFACE

11

B- Adding a question from the Question Bank

Use this option to choose questions from an earlier Question Bank.

“Question Bank” is listed under “Quiz administration”. You can see, add, and change questions in

the Question bank or add different categories for questions, like “Midterm 1” and “Midterm 2”.

These categories can be exported for another course later.

NOTES ON THE CMS QUIZ INTERFACE

12

Giving a name or number to each question helps to find the question in the Question bank. “Show

question text in the question list” can also be marked to see the question text. On this page, you

can edit, delete and preview questions.

NOTES ON THE CMS QUIZ INTERFACE

13

C- Adding Random Questions

Use this option when there are more questions in the Question bank than you need for each

quiz. For example, you can choose 5 questions from 50 questions in the Question bank.

The system will then choose different random questions for each student from your Question

bank.

NOTES ON THE CMS QUIZ INTERFACE

14

After choosing questions with one of these 3 methods, the maximum grade should be

assigned.

Question grades can be individually specified if grades will be different. If all questions have

the same grade they can be left as “1.00”. The system automatically calculates students’

grades according to a maximum grade based on the number of questions.