Principles of Measurement

Lunch & Learn Oct 16, 2013 J Tobon & M Boyle

What is Measurement?

• the process of assigning numerals to variables to represent quantities of characteristics according to certain rules.

• Objective – directly observable (e.g., length)• Subjective – inferred from experience (e.g., sensations, perceptions, behaviour)

What are some examples of concepts you

have/would like to measure?

Measurement Steps

1. Identify and define the concept

2. Specify the objective and conditions of measurement

3. Determine if a useful measure exists

4. Decide whether to use existing measure or develop a new measure

Use Develop

Measurement Steps

5. Create a question/item pool and evaluate the relevance/importance of each item using a formal judgment approach and trim item pool

6. Choose an approach to scaling

7. Pre-test instrument with target respondents (interviews, focus groups, pilot data)

8. Conduct reliability and validity studies

Measurement Steps

Reliability and Validity studies

Basic Measurement Concepts

• Development• Evaluation

Basic Concepts…

What is a Concept?

• A mental representation, an abstract idea, the invented name for an attribute that we cannot observe directly but infer from the data of experience (indicators) e.g. SES, I.Q., anger

• Concepts are usually complex and need to be represented by multiple indicators

Basic Concepts…

What are the Objectives of Measurement?

1. To discriminate between individuals

2. To evaluate change within individuals

3. To classify individuals

What are the Conditions of Measurement?

Practical considerations associated with assessment and data collection (cost/burden)

Basic Concepts…

What is a Useful Measure?

1. Adequately represents the concept of interest

2. Satisfies the reliability/validity (empirical) requirements associated with the objectives of measurement

3. Meets practical requirements

Basic Concepts…

What is Scaling?

• The process of ‘scaling’ is an attempt to upgrade measurement – the numerical representation of attributes (e.g., increasing reliable variance or the meaning of score intervals)

Basic Concepts…

What are the Levels of Measurement?

• Nominal, ordinal, interval and ratio

• These levels determine the numerical operations available in an analysis

• Strength and form of association between variables will depend on capturing their underlying nature

Levels of Measurement

NO SCHOOLING SOME PRIMARY COMPLETED PRIMARY SOME SECONDARY SECONDARY COMPLETED SOME COMMUNITY ETC COMPLETED COMMUNITY SOME UNIVERSITY COMPLETED UNIVERSITY

1 2 3 4 5 6 7 8 9

0 3 6101314151617

IntervalOrdinal

0

1

Nominal

Rate your agreement with the following statements

SA A N D SD

I am knowledgeable about the signs and symptoms of student mental health issues

1 2 3 4 5

I am knowledgeable about appropriate actions to take to support student mental health at school

1 2 3 4 5

I am knowledgeable about legislation related to mental health issues (confidentiality, consent to treatment, etc.)

1 2 3 4 5

I am knowledgeable about school system services and resources for helping students with mental health issues

1 2 3 4 5

What is an Additive Scale?•multiple items, same response, added up

Basic Concepts…

Single Item Multiple Items

Signs & Symptoms MH Literacy

Basic Concepts…

What is a Good Additive Scale?

• It will be unidimensional

• It will cover the full range of meaningful values

• It will be scaled to the highest level of measurement

• It will be reliable and valid for its intended use

Basic Concepts…

What is reliability?• the extent to which measures provide

similar values on replication

What are the types of reliability?1. Internal consistency – alpha (item

sampling)2. Test-retest (b/w respondent)3. Inter-observer (b/w observer)

Levels of reliability constrain the magnitude of associations (Spearman Brown Prophecy)

(Predicted) = (Observed) / (Rel_x)(Rel_y)

0.24 rXY = = 0.50

0.60 x 0.80

Observed Predict

ed

Reliabilities of x & y

Importance of Reliability

Basic Concepts…

What is validity?• the extent to which measures

adequately represent a concept and are useful for its intended objective

What are the types of validity?• Old: face, content, criterion, construct • New: construct – test hypotheses based

on theoretical linkages with other measured concepts (nomological net)

Types of Hypotheses

• Differences between groups (discrimination)

• Sensitivity to change (evaluation)

• Accuracy of classification (prediction)

• Correlations with similar measures

Types of hypotheses should align with the objective of measurement

Summary

• Concepts define the subject matter (boundary) of different scientific disciplines

• Measurement quality (level) defines the sophistication scientific disciplines

• Measurement is a two-step process that works within practical constraints (objectives and assessment conditions) and scientific prerequisites (reliability and validity)