Setting Multiple Choice Questions

Tips & Tricks

2 of 34

Difficulty

type

Description

of the type Translates to the following guidelines

1 Simple Requires 30secs to less than 2mins

2 Medium Requires 2 mins to less than 5 mins

3 Complex Requires 5 mins to 10 mins

Type Name

Guideline(s) - These are mere indicative for

ease of use and by no means a defn for

each of the categories.

Example - The examples

given here are some sort of

multiple choice qs belonging

to that category.

C Conceptual

These type of questions are answered by

learning a "new" concept that is taught

Example: Which of the

following choices is not a

function of "Thrashing"

A Analytical

These type of questions require a good amount

of analysis to answer

Example: Which one is better

in general for average waiting

time - premptive or non-

premptive scheduling ?

M Memory

These type of qs are answered by just

remembering and recalling from the material

(course, book etc.)

Example: How many layers are

there in OSI/ISO Network

model?

P

Practice-

based

These type of questions check for the

understanding and applying the concept in a

practical way

Example: How does one find

out the "configuration" of a

machine?

3 of 34

Conceptual

• Conceptual

Questions which require the candidate to

Demonstrate understanding of facts and ideas

by organizing, comparing, translating,

interpreting, giving descriptions, and stating

main ideas

Example:

Which of the following choices is not a function of

"Thrashing“

4 of 34

Analytical

• Analytical

Questions, which require the candidate to

examine and break information into parts by

identifying motives or causes and thereafter,

make inferences and find evidence to support

generalizations

Example: Which one is better in general for average

waiting time - premptive or non-premptive scheduling ?

5 of 34

Memory

• Memory-recall

Questions, which require the candidate to

exhibit memory of previously-learned

materials by recalling facts, terms, basic

concepts and answers

Example: How many layers are there in OSI/ISO

Network model?

6 of 34

Practice based

• Practice-based

Questions, which require the candidate to

solve problems in certain situations by

applying, acquired knowledge, facts,

techniques and rules in different ways

Example: How does one find out the "configuration" of a

machine?

7 of 34

Multiple Choice Question /

Objective question ?

Multiple choice question (or item) is one in which a student chooses one answer from a

number of choices supplied. A multiple choice question consists of a :

stem - the text of the question

options - the choices provided after the stem

the key - the correct answer in the list of options

distracters - the incorrect answers in the list of options

In RDBMS, the key that helps in establishing relationship between tables is

a) Primary Key

b) Candidate key

c) Foreign key

d) Composite Key

Stem

Key

Distracters

8 of 34

Step by Step guidelines

to set

Effective Multiple Choice Questions

9 of 34

Writing Stem

Present a single, definite statement to be completed or

answered by one of the several given choices

A. Weak question: Page fault

a. Is an error in the specific page

b. Is a reference to a page of the program which is not in the primary

memory

c. Is an access to a page belonging to another program

d. Is a reference to a page belonging to another program

B. Improved question: Page Fault in the Demand Paging memory management scheme of

Operating system refers to

a. an error in the specific page

b. a reference to a page of the program which is not in the primary

memory

c. Is an access to a page belonging to another program

d. Is a reference to a page belonging to another program

10 of 34

Writing Stem - Avoid unnecessary and

irrelevant material

A. Weak question:

Which of the following is not true for MOM – Message Oriented Middleware, a client/server

infrastructure which is asynchronous in nature?

a. Client initiates communication by putting messages on the message queue

b. No interaction with server process

c. Server uses another queue to send response back

d. Server uses the same message queue used by client to send responses back

B. Improved question:

Which of the following is not true for MOM – Message Oriented Middleware?

a. Client initiates communication by putting messages on the message queue

b. No interaction with server process

c. Server uses another queue to send response back

d. Server uses the same message queue used by client to send responses back

Example A contains material irrelevant to the question.

11 of 34

Writing Stem - Use of Language

Use clear, straightforward language in the stem of the item.

Questions that are constructed using complex wording may become a test of reading

comprehension rather than an assessment of whether the student knows the subject

matter.

A : Weak question

As the level of fertility approaches its nadir, what is the most likely

ramification for the citizenry of a developing nation?

a.a decrease in the labour force participation rate of women

b.a dispersing effect on population concentration

c.a downward trend in the youth dependency ratio

d.a broader base in the population pyramid

e.an increased infant mortality rate

B : Improved question

A major decline in fertility in a developing nation is likely to produce

a.a decrease in the labour force participation rate of women

b.a dispersing effect on population concentration

c.a downward trend in the youth dependency ratio

d.a broader base in the population pyramid

e.an increased infant mortality rate

Ref: GRE - Sociology Test 1997-1999

12 of 34

Writing Stem - Use of Negatives

Use negatives sparingly. If negatives must be used, capitalize, underscore, embolden or

otherwise highlight

A : Weak question

Which of the following is not a symptom of osteoporosis?

a.decreased bone density

b.frequent bone fractures

c.raised body temperature

d.lower back pain

B : Improved question

Which of the following is a symptom of osteoporosis?

a.decreased bone density

b.raised body temperature

c.hair loss

d.painful joints

13 of 34

Writing Stem - Duplication

Put as much of the question in the stem as possible, rather than

duplicating material in each of the options.

A : Weak question

Data Control Language GRANT is used for

a. Allocating privileges for the users/roles on tables only

b. Allocating privileges for the users/roles on views only

c. Allocating privileges for the users/roles on both tables and views

d. Allocating privileges for the users/roles on only the rows of the table

B : Improved question

Data Control Language GRANT is used for allocating privileges for

users/roles

a. on tables only

b. on views only

c. on both tables and views

d. on only the rows of the table

14 of 34

Writing Distracters

For single response MCQ's, ensure that there is only one correct response.

A : Weak question In RDBMS, Join operation between the two tables based on equality

condition “=“ is called

a. Equi Join

b. Cross Join

c. Outer Join

d. Inner Equi Join

B : Improved question In RDBMS, Join operation between the two tables based on equality

condition “=“ is called

a. Equi Join

b. Cross Join

c. Outer Join

d. Inner Join

In Example A, both options 'a' and 'd' could be considered to be correct.

15 of 34

Writing Distracters

Use only plausible and attractive alternatives as distracters

Which of the following is an example for Network Operating System (NOS)?

a. Novell Netware

b. Windows XP

c. Java

d. Dot Net

A : Weak question

B : Improved question

Which of the following is an example for Network Operating System (NOS)?

a. Novell Netware

b. Windows XP

c. Window 2000

d. Windows 98

In Example A, options 'c' and 'd' are not serious distracters

16 of 34

Writing Distracters

If possible, avoid the choices "All of the above" and "None of the above".

• An "all of the above" alternative could be exploited by a test-wise students who will

recognise it as the correct choice by identifying only two correct alternatives.

• A student who can identify one wrong alternative can then also rule this response out.

• Clearly, the student's chance of guessing the correct answer improves as they employ these

techniques.

17 of 34

Additional Tips

• Distracters based on common student errors or misconceptions are very effective - Identify which incorrect responses appear most frequently and use them as distracters for a

multiple choice version of the question.

• Correct statements that do not answer the question are often strong distracters.

• Do not create distracters that are so close to the correct answer that they may

confuse students who really know the answer to the question.

• Provide a sufficient number of distracters – This avoids the likeliness of choosing the

right answer by guessing

18 of 34

Complicated form of MCQ

Multiple true / false MCQ’s

Which of the following statements regarding Client-Server Computing are true ?

a. Client Server Computing can be divided into three components, a Client Process, a Server

Process and a Middleware in between them for their interaction.

b. The Client-Server environment is typically homogeneous and single-vendor. The hardware

platform and operating system of Client and Server are usually the same.

c. Database, Transaction Servers are more advanced and sophisticated types of Servers than

Disk and File Servers.

d. Client-Server Computing is merely a transitional phase in the larger trend towards fully

distributed computing.

i) Only a ii) Only a &b iii) only a, c & d iv) Only a, b & d

In the above example options ‘a‘,’c’ &’d’ are correct options

21 of 34

True / False Questions

• A true-false questions is a specialized form of the multiple-

choice format in which there are only two possible

alternatives. Suggestions for writing true-false questions:

Include only one main idea in each item.

As in multiple choice questions generally, use negatives sparingly.

Use statements which are unequivocally true or false.

Avoid lifting statements directly from assigned reading, lecture notes or other course

materials so that recall alone will not permit a correct answer.

Generally avoid the use of words which would signal the correct response to the test-

wise student. Absolutes such as "none", "never", "always", "all", "impossible" tend to be

false, while qualifiers such as "usually", "generally", "sometimes" "often" are likely to be

true

22 of 34

True / False Questions ( Contd…)

“C” is an object oriented programming language:

a. TRUE

b. FALSE

c. NOT ANSWERED

Following type of questions must NOT be generated

Consider the following statements:

C is a structured language

C is an object oriented programming language

Choose the CORRECT option:

i) is TRUE and ii) is FALSE

ii) is TRUE and i) is FALSE

Both i) and ii) are TRUE

Both i) and ii) are FALSE

NOT ANSWERED

A : Weak question

B : Improved question

23 of 34

Suggestions for setting MCQ’s

• Identify the important and relevant topics

• There must be proper distribution of the questions among the topics. Do not include too many questions in one topic only

• All the questions must be within the scope

• The answer options must also be within the scope

• The answer options must not be contradictory

• The wrong answer option must NOT be obvious

• The correct answer option must NOT be obvious

• The number of answer options must NOT be less than 4 or more than 5

• Questions must NOT be repeated

• Questions must be CAMP based as follows: • C – Conceptual (30%)

• A – Analytical (30%)

• M – Memory Based (10%)

• P – Practice Oriented (30%)

• Spelling and grammatical mistakes must NOT be there in the questions and answer options

• Proper capitalization and punctuation must be followed

• Check the total marks

24 of 34

Suggestions for setting MCQ’s

Setting MCQs to test recall of knowledge or basic application of factual knowledge is

easier compared to test for understanding of key concepts or to synthesise complex

information.

Guidelines:

• Whenever you prepare a multiple choice question, check yourself what it is testing. Can it be answered on factual recall or direct application of a (recalled) formula only? If a student gets the answer correct, does this demonstrate she has understood?

• Prepare a set of questions on the topics you want to test and have a peer review with your colleagues and ask for their feedback about what they think the questions are testing – This helps us to ensure that questions adhere to what we intend to test ?

25 of 34

References

• Isaacs, G. (1994) "About multiple choice questions" in Multiple

Choice Testing: Green Guide, No. 16, HERDSA, Cambelltown, NSW,

4-22.

• http://www.caacentre.ac.uk/resources/objective_tests/index.shtml

26 of 34

Thank You