Post on 04-Apr-2018
transcript
7/29/2019 Introduction-Principles of Psychological Measurement
1/44
PRINCIPLES OF PSYCHOLOGICAL MEASUREMENT
3090.03
Instructor: Peter Papadogiannis, Ph.D.
York University
7/29/2019 Introduction-Principles of Psychological Measurement
2/44
Why is Psychological Testing Important?
1. Allows us to make important decisions aboutpeople.e.g. Early School Placement, College Entrance Decisions,Military Job Selections
2. Allows us to describe & understand behaviour
3. Measures personal attributes4. Measures performance
5. Saves time
6. Most economical
7. Its Scientific
7/29/2019 Introduction-Principles of Psychological Measurement
3/44
Psychological Test DefinitionIs a measurement instrument that consists of a
sample of behavior obtained under
standardized conditions and evaluated usingestablished scoring rules.
7/29/2019 Introduction-Principles of Psychological Measurement
4/44
Types of Reasoning Associated
with Psychological TestingInductive Reasoning - from Data to General
Theory.
Deductive Reasoning - from General Theory toData.
7/29/2019 Introduction-Principles of Psychological Measurement
5/44
Characteristics of Psychological Instruments
Behaviour Sampling
Standardization Scoring Rules
7/29/2019 Introduction-Principles of Psychological Measurement
6/44
Behaviour Sampling
It is a sample of behaviour.
It is not an exhaustive measure - it is too difficult
to evaluate every behavior.
Attempts to approximate the exhaustiveprocedure.
Does not necessarily require the respondent to
engage in overt behavior.
The test must somehow be representative of
behaviours that would be observed outside of the
testing situation
7/29/2019 Introduction-Principles of Psychological Measurement
7/44
StandardizationThe behavior sample is obtained under standardized
conditions.
Each individual taking a psychological or educational test
should be tested under essentially identical conditions.
For example, SAT administration instructions pertain to:
Seating Arrangements, Lighting Conditions, Noise Levels Interruptions,
Answering common questions
Standardization is vital because many test results are
referential in nature: Your performance is measured
relative to everybody elses performance.
7/29/2019 Introduction-Principles of Psychological Measurement
8/44
Standardization (cont.)
Standardization reduces between subject variability due
to extraneous variables.
Standardization is easier to obtain with tests designed to
be administered en masse.
Tests such as the Wechsler Adult Intelligence Scale, which
are administered individually, are less standardized.
The individual giving the test is an important variable.
They take special training to standardize the way they give
the test.
7/29/2019 Introduction-Principles of Psychological Measurement
9/44
Scoring RulesThere are established scoring rules for obtaining quantitative
information from the behavior sample.
Objective Scoring Rules: Most mass produced tests fall into this
category. Different qualified examiners will all come to the same score
for an identical set of responses.
Subjective Scoring Rules: When the judgement of the examiner is an
important part of the test, different examiners can legitimately come to
different conclusions concerning the same sample of behavior. There
conclusions should be similar, however.
Good standardized psychological tests all have a set of rules or
procedures for scoring responses to a test.
7/29/2019 Introduction-Principles of Psychological Measurement
10/44
Types of Tests
Most psychological tests can be sorted into 3
general categories:
1. Tests in which the subject performs a task.
2. Tests that involve observations of the subjects
behaviour within a particular context.
3. Self-report measures
7/29/2019 Introduction-Principles of Psychological Measurement
11/44
Tests of PerformanceReferred to as "Tests of Maximal Performance"
Subjects are given a well-defined task that they try to performsuccessfully.
Participant must know what he/she must
do in response to the task.
The subject exerts maximal effort to
succeed.
Performance tests are designed to uncover whatan individual can do, given the specific testconditions.
Examples
- Intelligence Tests, language proficiency- Biology test, flight simulator
B h i Ob i
7/29/2019 Introduction-Principles of Psychological Measurement
12/44
Behaviour Observation
Naturalistic observation
Involves observing the subjects behaviour and responses
in a particular context.
Differs from performance tests in that the subject does
not have a single, well defined task.
The observer can record duration &
intensity
Examples
- Examiner might observe children interacting
or an individual having a conversation or some other social interaction.
- Companies recruit observers to pose as salespeople to observeemployees behaviors. Subjects may be unaware they are being tested.
7/29/2019 Introduction-Principles of Psychological Measurement
13/44
Self Report Instruments- Participant is asked to report his or her feelings,
attitudes, beliefs, values.
When self-report makes sense:
Self-report relies upon the test takers awareness andhonesty.
It is the bestmethod to measure internal states - things
only the person themselves can be aware of and judge.
People are not always good judges of their ability
Provides an estimate
7/29/2019 Introduction-Principles of Psychological Measurement
14/44
Self Report Instruments (cont.)Many personality inventories such as the MMPI and the
16PF measures are based on self-report.
Clinicians include self-report measures as part of their
initial examinations of presenting clients.
Self-Report measures are frequently subject to self-
censorship.
People know their responses are being measured andwish to be seen in a favorable light. (self-serving bias)
Items are frequently included to measure the extent to
which people provide socially desirable responses.
7/29/2019 Introduction-Principles of Psychological Measurement
15/44
History of Test Developmentcirca 1000 BC.:Chinese introduced written tests to help fill civil service
positions Civil Laws, Military Affairs, Agriculture, Geography
1850 : The United States begins civil service examinations.
1885 : Germans tested people for brain damage
1890: James Cattell develops a "mental test" to assess college students .
Test includes measures of strength, resistance to pain, and reaction time.1905: Binet-Simon scale of mental development used to classify
mentally retarded children in France.
1914: World War I produces need in U.S. to quickly classify incoming
recruits. Army Alpha test and Army Beta test developed. Looked atpsychopathology.
1916 :Terman develops Stanford - Binet test and develops the idea of
Intelligence Quotient
7/29/2019 Introduction-Principles of Psychological Measurement
16/44
History of Test Development (cont.)
1920 - 1940:factor analysis, projective tests, and personality inventoriesfirst appear.
1941-1960 : vocational interest measures developed
1961-1980 : item response theory and neuropsychological testing
developed1980 - Present: Wide spread adaptation of computerized testing.
"Smart" Tests which can give each individual different test items develop
7/29/2019 Introduction-Principles of Psychological Measurement
17/44
Early Abuses of Tests in America Goddard (1906) began testing 378 residents and categorized them as
Idiot (ma below 2), imbecile (3-7), feebleminded (8-12), moron (foolish)
MA four years behind, were feebleminded
Goddards desire was to separate people out
Believed feeble minded people were the cause of most social problems
(thievery, laziness, alcoholism, prostitution, immorality).
Called for the colonization of morons to restrict their breeding. Further, he
believed that many immigrants were feeble minded.
Went to Ellis Island, administered tests translated from French to English to
Yiddish, Hungarian, , Italian, Russian, to farmers, laborers, who had just
crossed the Atlantic. Then interpreted results based on French norms.
Favored deportation for low IQ immigrants but then also in a humanitarian
gesture said we might be able to use moron laborer if only we are wiseenou h to train them ro erl .
7/29/2019 Introduction-Principles of Psychological Measurement
18/44
Early Abuses of Tests in America (cont.) Robert Yerkes, a Harvard psychology prof. Convinced the Department
of War that it should test all of its 1.75 million recruits for intelligence
tests, so they could be classified and given appropriate assignments(Goddard and Terman also chaired this committee).
Army Alpha
Army Beta Examinations
Produced evidence that supported segregation. Sounded dire warnings
that racial intermixture would inevitably cause a deterioration of
American intelligence. Later recanted: without foundation Probablythe result of cultural and language differences.
7/29/2019 Introduction-Principles of Psychological Measurement
19/44
Application of Psychological Measurement
Educational Testing
Personnel Testing
Clinical Testing
7/29/2019 Introduction-Principles of Psychological Measurement
20/44
Educational Testing Intelligence tests and achievement tests are used from an early age in
the U.S and Canada. From kindergarten on, tests are used for placement
and advancement.
Educational institutions have to make admissions and advancement
decisions regarding students. e.g, SAT, GRE, subject placement tests
Used to assess students for special education programs. Also, used in
diagnosing learning difficulties.
Guidance counselors use instruments for advising students.
Investigates school curriculum.
7/29/2019 Introduction-Principles of Psychological Measurement
21/44
Personnel Testing Following WW I, business began taking an active interest in testing job
applicants. Most government jobs require some civil service
examination.
Tests are used to assess: training needs, workers performance in
training, success in training programs, management development,
leadership training, and selection.
For example, at the Lally School of Management, the Myers -Briggs
type indicator is used extensively to assess managerial potential. Typetesting is used to hopefully match the right person with the job they are
most suited for.
7/29/2019 Introduction-Principles of Psychological Measurement
22/44
Clinical Testing
Tests of Psychological Adjustment and tests which canclassify and/or diagnose patients are used extensively.
Psychologist generally use a number of objective andprojective personality tests.
Neuropsychological tests which examine basic mental
function also fall into this category. Perceptual tests are
used detecting and diagnosing brain damage.
7/29/2019 Introduction-Principles of Psychological Measurement
23/44
Testing Activities of PsychologistsClinical Psychologists - e.g. Assessment of Intelligence, Assessment ofPsychopathology
Counseling Psychologistse.g. Career Interest Inventories, Skill Assessment
School Psychologists
e.g. Assessment of Academic progress, Readiness for School,
Social Adjustment
I/O Psychologists - e.g. Managerial potential,
Training Needs, Leadership Potential
Neuropsychologists- e.g., Assessment of Brain Damage, neurologicalimpairments.
Forensic Psychology- intersection between law andpsychology --needed for legal determinations
e. . Assessment for risk com etenc to stand trial child custod
7/29/2019 Introduction-Principles of Psychological Measurement
24/44
Information About TestsThe Mental Measurement Yearbook - A guide to all currentlyavailable psychological tests.
The MMY uses content classifications do describe tests:
1. Acheivement 2. Behavior Assessment
3. Developmental 4. Education5. English & Language 6. Fine Arts
7. Foreign Languages 8. Intelligence and Aptitude
9. Mathematics 10. Neuropsychological
11. Personality 12. Reading
13. Science 14. Sensory-Motor
15. Social Studies 16. Speech and Hearing
17. Vocations
7/29/2019 Introduction-Principles of Psychological Measurement
25/44
Ethics In Psychological Testing
Given the widespread use of tests, there is considerable
potential for abuse.
A good deal of attention has therefore been devoted to the
development and enforcement of professional and legalstandards.
The American Psychological Association (APA) has taken
a leading role in the development of professional standardsfor testing.
A i P h l i l A i ti Ethi l G id li
7/29/2019 Introduction-Principles of Psychological Measurement
26/44
American Psychological Association Ethical Guidelines:
The investigator has the responsibility to make a careful evaluation of its ethical
acceptability.
The investigator is obliged to observe stringent safeguards to protect the rights of
human participants.
The researcher must evaluate whether participants are considered Subject at risk or
Subject at minimal risk - No appreciable risk (physical risk, mental harm).
The principal investigator always retains the responsibility for ensuring ethical
practice in research. That is, the principal researcher is responsible for the ethical
practices of collaborators, assistants, employees, etc. (all of whom are also responsible
for their own ethical behavior).
Except in minimal-risk research, the investigator establishes a clear and fair
agreement with participants that clarifies the obligations and responsibilities of each.
Must explain all aspects of the research that may influence the subjects decision to
participate. Explains all other aspects that the participants inquire about.
American Ps chological Association Ethical G idelines (cont ):
7/29/2019 Introduction-Principles of Psychological Measurement
27/44
American Psychological Association Ethical Guidelines (cont.):
In research involving concealment or deception, the research considers the
special responsibilities involved.
Individuals freedom to decline, and freedom to withdraw, is respected.
Researcher is responsible for protecting participants from physical and mental
discomfort, harm, and danger that may arise from research procedures. If there
are risks, the participants must be aware of this fact.
After the data are collected the investigator provides participants with
information about the nature of the study and attempts to remove any
misconceptions that may have arisen.
The investigator has the responsibility to detect and remove any undesirableconsequences to the participant that may occur due to the research.
The information obtained from the participant should be treated
confidentially unless otherwise agreed upon with the participant.
I f d C
7/29/2019 Introduction-Principles of Psychological Measurement
28/44
Informed Consent
Participants must be fully informed as to the purpose and
nature of the research that they are going to be involved in.
Participants must be fully informed about the procedures used
in the research study.
After getting this information, the participants must provide
consent for their participation.
Participants must be informed about their right to
Confidentiality and their right to withdrawal without penalty.
D b i fi
7/29/2019 Introduction-Principles of Psychological Measurement
29/44
Debriefing
Post-administration debriefing should:
-Restate purpose of the research.
-Explain how the results will be used (usually.
emphasize that the interest is in the group
findings).-Reiterate that findings will be treated
confidentially.
-Answer all of the respondents questions fully.-Thank the participant!
P ti i t F db k
7/29/2019 Introduction-Principles of Psychological Measurement
30/44
Participant Feedback
In clinical research, or research with interpretive
instruments, there may be the need to provide more in-
depth feedback about individuals responses (e.g.,
Research on Emotional Intelligence).
In such cases, first and foremost, it is critical that this
kind of detailed feedback be given by a qualified
individual.
At Least 4 parties are involved in Professional Test Use:
7/29/2019 Introduction-Principles of Psychological Measurement
31/44
At Least 4 parties are involved in Professional Test Use:
(1) Testing professionals: the test developer and
publisher
(2) Testing professionals: the individuals who
administer the testing procedure
(3) The user: the organization or practice that will
eventually use the information to make certain
decisions
(4) The test taker
DEVELOPING/SELECTING APPROPRIATE TESTS
7/29/2019 Introduction-Principles of Psychological Measurement
32/44
DEVELOPING/SELECTING APPROPRIATE TESTS
Define what each test measures and what the test should be used for.
Describe the population(s) for which the test is appropriate.
Accurately represent the characteristics, usefulness, and limitations of
tests for their intended purposes.
Describe the process of test development.
Provide evidence that the test meets its intended purpose(s).
Provide either representative samples or complete copies of test
questions, directions, answer sheets, manuals, and score reports to
qualified users.
DEVELOPING/SELECTING APPROPRIATE TESTS (Cont )
7/29/2019 Introduction-Principles of Psychological Measurement
33/44
DEVELOPING/SELECTING APPROPRIATE TESTS (Cont.)
Indicate the nature of the evidence obtained concerning the
appropriateness of each test for groups of different racial, ethnic, or
linguistic backgrounds who are likely to be tested.
Describe the population(s) represented by any norms or comparison
group(s), the dates the data were gathered, and the process used to select
the samples of test takers.
When feasible, make appropriately modified forms of tests or
administration procedures available for test takers with handicapping
conditions. Warn test users of potential problems in using standard norms
with modified tests or administration procedures that result in non-
comparable scores.
DEVELOPING/SELECTING APPROPRIATE TESTS (Cont )
7/29/2019 Introduction-Principles of Psychological Measurement
34/44
DEVELOPING/SELECTING APPROPRIATE TESTS (Cont.)
When a test is optional, provide test takers or their parents/guardians
with information to help them judge whether the test should be taken, or
if an available alternative to the test should be used.
Provide test takers the information they need to be familiar with the
coverage of the test, the types of question formats, the directions, and
appropriate test-taking strategies. Strive to make such informationequally available to all test takers.
Provide test takers or their parents/guardians with information about
rights test takers may have to obtain copies of tests and completed
answer sheets, retake tests, have tests rescored, or cancel scores.
Tell test takers or their parents/guardians how long scores will be kept
on file and indicate to whom and under what circumstances test scores
will or will not be released.
R ibilit f Th T t
7/29/2019 Introduction-Principles of Psychological Measurement
35/44
Responsibility of The Tester1. Have competence in test administration, interpretation and
feedback.
2. Have an understanding of basic psychometrics and scoring
procedures and be competent in interpretation, and apply
scientific knowledge and professional judgment to the results.
3. Take responsibility for the selection, administration, and
scoring, the analysis, interpretation and communication of test
results.
4. Be familiar with the context of use: the situation, purpose,
setting in which a test is used.
R ibilit f Th T t ( t )
7/29/2019 Introduction-Principles of Psychological Measurement
36/44
Responsibility of The Tester (cont.)
5. Have knowledge of legal and ethical issues related to testuse
6 . Awareness of ethnic or cultural variables that could
influence the results:
7. Have the ability to determine language proficiency
8. Have knowledge of important racial, ethnic, or cultural
variables relevant for individuals or groups to whom tests are
administered.
Iss es to Address ith the Testee
7/29/2019 Introduction-Principles of Psychological Measurement
37/44
Issues to Address with the Testee1. Informed consent - Assuring confidentiality, freedom to
withdraw, purpose of assessment, What kinds of attributes are
being measured?
2. Who is the client?Individual, Group, Employer
3. What happens with results, who has access to it
4. Where will the data be stored, how, and for how long
5. Time frame in which results are to be considered valid
6. Who will be the payer, and how much
7. Where will the assessment take place8. Are the facilities appropriate, conducive for testing
9. Will there be follow-up assessments or feedback
Factors Not Under the Testers Control
7/29/2019 Introduction-Principles of Psychological Measurement
38/44
Factors Not Under the Testers Control
Some factors are not under the control of the administrator :
1. How fatigued a test taker is.
2. Motivation level of the test taker.
3. Physical Discomfort
4. Test Anxiety
Ethnic and Cultural Variables
7/29/2019 Introduction-Principles of Psychological Measurement
39/44
Ethnic and Cultural VariablesKnowledge of attitudes of various racial,
ethnic, or cultural groups toward testing.Ability to determine language proficiency.
Ability to determine the potential effects of
different test settings on different racial,ethnic, or cultural groups.
Knowledge of specific biases that have beendemonstrated for particular tests forindividuals or groups of individuals fromparticular racial, ethnic, or cultural minoritygroups.
Test Fairness
7/29/2019 Introduction-Principles of Psychological Measurement
40/44
Test FairnessPeople with different values often disagree over the fairness
of some testing practices.
Factors that affect testing fairness:
1. Obstacles that prevent people from performing well2. Test may provide unfair advantage to some people
3. Some tests are not valid and used in wrong situations
4. Some tests are used for purposes that are inherently
objectionable
Test Use & Test Fairness
7/29/2019 Introduction-Principles of Psychological Measurement
41/44
Test Use & Test FairnessA test is most likely to be seen as unfair when:
1. It is the sole basis for the decision.2. The consequences of doing poorly on the test is harsh
Ways to reduce concerns over test unfairness:1. Multiple assessment procedures
2. Use more intensive screening procedures for those
likely to be treated unfairly by a given test.
Types of Decisions
7/29/2019 Introduction-Principles of Psychological Measurement
42/44
Types of DecisionsTwo distinctions are very useful for classifying
decisions:
1. Individual or Institutional
2. Comparative or Absolute
Ethics References
7/29/2019 Introduction-Principles of Psychological Measurement
43/44
Ethics ReferencesWebsites www.apa.org and www.cpa.ca
APA
American Psychological Associations Code of Ethicsfor Psychologists
The Ethical Practice of Psychology in OrganizationsCPA
The Canadian Code of Ethics for Psychologists
Companion Manual
Guidelines for Non-discriminatory Practice
Guidelines for Educational and Psychological Testing
Practice Guidelines for Providers of Psychological
Service
Ethics References (cont )
7/29/2019 Introduction-Principles of Psychological Measurement
44/44
Ethics References (cont.)Websites for publications / reports
Rights and Responsibilities of Test Takers: Guidelines andExpectations(www.apa.org/science/ttrr.html)
Responsibilities in Providing Psychological Test Feedback toClients(http://kspope.com/sci/feedback1.shtml)
The Real World: It is Better to Receive than to Give--Practical Tips for Giving and Receiving PerformanceFeedback(www.siop.org/tip/TipOct00/12waclawski.htm)
Statement on the Disclosure of Test Data
(www.apa.org/science/disclosu.html) Test Security(American Psychologist, Dec. 1999, Vol.54,
No.12, p1078)
Statement on the Use of Secure Psychological Tests in the