Unit 2, PSY 4600

ScheduleTuesday and Thursday: LectureMonday, 9/23: 5:30-7:00 PM Instructional AssistanceTuesday, 9/24: Exam

1

2

Last unit: Respondent Behavioral Relations

S--->R

US--->UR CS--->CR

This unit: Operant Behavioral Relations

MO:SD/S∆:R--->Sc

Focus on operant consequences and SDs and S∆s

SO 2: Basic Behavioral Principles

3

1. ReinforcementA. PositiveB. Negative (difference between pos & neg?)

1. Escape (alarm clock, safety harness after chime)

2. Avoidance (safety harness before chime; child plays quietly)

2. Punishment3. Operant Extinction (withheld, not withdrawn)

Examples on page 18 of the Study Objectives

(terminates or avoids, e-aversive stim that comes before, TV screen clears, food reinforcement, avoid vs. pun, Decrease to avoid conseq – not correct;student asks question, professor says; pun dec; avoid incr; cannot increase a nonbehavior; extinction burst, taking truck away, sending a child from the table; language will be very clear in the examples I provide on the exam)

SO 2: Examples (in SOs)• Rafael gets a muscle cramp. He massages the

muscle and the cramp immediately decreases in severity. As a result, when Rafael gets a muscle cramp in the future, he massages it more often than he had done in the past.

• A student wants to make a copy. She inserts her Bronco Card into a copy machine and pushes the button. No copies are made. The student pushes and pushes the button, but still no copies are made. As a result, the student pushes the copy button on that machine less often.

4

SO 2 (Examples, cont.)• Barbara calls her little sister a scardy cat and the

little sister immediately begins to cry. As a result, Barbara calls her sister a scardy cat more often in the future.

• A worker is standing around with co-workers and puts on her hard hat before entering the construction area. Her supervisor sees this and immediately says “Hey, that’s great, Grace! Thanks for making safety first a reality!” As a result, Grace puts on her hard hat less often in the future.

5

(effect on behavior, can’t just look at the conseq; teachers and elementary school children, criticism,attention)

SO 2: Final Example

• Jake gets bitten by bugs when he walks in the woods. One day, he puts on a new kind of bug repellant and does not get bitten by bugs. As a result, in the future, he puts on that new kind of bug repellant before he walks in the woods more often.

6(exam examples, Tas use examples from your exams)

SO 3: Abbreviations: Unconditioned and Conditioned Reinforcers

7

Unconditioned Reinforcer

SR NOT UR UR=Unconditioned Response

Conditioned Reinforcer

Sr NOT CR CR=Conditioned Response

SO 4: Introduction

• Unit starts with animal training– Dolphin training, e.g., Shedd Aquarium, San Diego Sea

World, Georgia Aquarium – Behavioral enrichment in zoos, e.g., Honolulu Zoo,

Atlanta Zoo, Brookfield Zoo, Disney Land and Disney World, Busch Gardens-Orlando

– Animal training (dogs, cats, horses, etc.), Karen Pryor (Don’t shoot the dog), Mary Burch & Jon Bailey (How dogs learn), Mary Burch, (Citizen Canine – AKA), Gillette Obedience Training (Galesburg, MI), Applied animal training practicum (WMU, UMN-Duluth)

– “Clicker Training”: Clicker as an Sr (athletes)

8(Gulf oil spill; Binti Jua – Brookfield zoo, Otto Fad, Ken Ramirez)

SO 4: The Aggressive Bull ElephantSan Diego Zoo

• An aggressive elephant• Husbandry includes cutting off calluses on feet,

otherwise, eventually they can’t walk• G. Priest established a “click” as an Sr• Shaped the elephant to walk to a wall with hole in

it, put its foot through the hole, and stand patiently while the vet cut off the calluses.

9

SO 4: Advantage of Srs vs. SRs

10

When shaping, the delivery of the consequence must follow the appropriate behavior (the successive approximation to the target behavior) as immediately after the behavior as possible. Otherwise, some other inappropriate behavior may be reinforced.

If an SR, such as food, is used, it is going to be hard to:• deliver it immediately after the appropriate behavior

e.g., tossing a carrot in the cage of the elephant - by the time you react, the elephant has emitted a new behavior

• the animal is going to stop and eat the food, which halts the shaping process (or behavioral sequence)• It also prevents satiation

(a) Srs can often be delivered more immediately than SRs,(b) Srs don’t interfere with the behavioral sequence as SRs

may(c) Srs prevent satiation of the reinforcer

SO 5: Development and Testing of an Sr

11

When food deprived (MO):

Development

NS (click) / SR (carrot) (no behavior is necessary!)

NS becomes an Sr

Critical features:• The NS is paired with an SR (or Sr) (NOT a US!)• The NS precedes the SR when pairing takes place• No behavior is necessary• The NS becomes an Sr (NOT a CS!)

(How trainers made click; all in SOs,click, crticial features on slide - not for the exam, need to test)

SO 5: Development and Testing of an Sr

12

Testing

When food deprived (MO):

R (any response) ----> Sr (click)

*If R increases in frequency, the NS has become an Sr

Critical features:1. The Sr follows the response (operant relation)2. The Sr is presented alone (not with the SR)3. The R must increase in frequency in the future4. The Sr must occasionally be paired with the SR

(*essential - if the R doesn’t increase, no reinforcer, click critical features)

SO5: Sample test question

• Sample test question is at the end of SO5• Answer is at the end of the study objectives

for this unit• I guarantee that a similar question will be on

the exam

13

SO6: Difference BetweenRespondent Conditioning and Development of an Sr

14

The confusion: Both involve pairing an NS with another stimulus

Difference:

Respondent NS/US, or NS/CS Conditioning:

Development NS/SR, or NS/Sr

of an Sr:

(Respondent conditioning: NS becomes a CS-->CR; Sr NS becomes Sr; R-->Sr)

(Thought question: When will an NS become both a CS and an Sr? NS, click)

SO8: Behavioral Enrichment in Zoos• Behavioral interventions designed to improve the well

being and health of captive animals• Hal Markowitz started this work in the 1970s• Zoos have a very important function: protection of

endangered species, education of public– keep humans from destroying natural habitats– keep humans from killing off species of animals (ivory

tusks or furs)– protect and preserve species that are endangered due

to disease, natural disasters

15

(mother nature ain’t kind – Poling story)

Zoos• Many of us cringe when we think about

zoos – animals in prison• But over the years, zoos have been

attempting to make life better for the animals

16(but most zoos have come a long way..)

SO8: Two popular* approaches zoos have tried to make life better for animals

• Make the enclosures more naturalistic• Add toys, boomer balls

17Neither – terrifically effective – naturalistic enclosures first)

*popular, but ineffective

SO9: What’s the problem, even when enclosures are naturalistic?

• Naturalistic enclosures sometimes do have some benefits for the animals

• Certainly make us more comfortable

18

SO9: What’s the problem, even when enclosures are naturalistic?

• Fail to include the behavioral contingencies in the wild that reinforce species typical (and active) behavior

• Much of the behavior of free-ranging animals involves getting food (the only one mentioned by Chance), fighting off or fleeing predators, natural migration, securing mates and mating, establishing social hierarchies, etc.

• It’s the consequences of those behaviors that maintain much of the active behavior of wild animals– some behavior is, of course, genetic– over the years, they have discovered, however, that many behaviors

that were once considered inherited are learned

19(most groups, dominant male: stallions, mares; gorillas; ducklings following Momcloseness to object, following in the natural environment, bird’s songs)

SO9: What’s the problem, even when enclosures are naturalistic?

• In zoos, food is provided usually in the same place at the same time each day, animals are completely protected from predators, certainly cannot migrate to different locations, and are not subjected to threats of their domination from outside animals

• There is “no reason” for animals to be active• Behaviorally the reason to be active:

R (species typical behaviors) SR (food or other reinforcers)

• What happens if behaviors are not followed by reinforcement?

20(in a zoo, no one wants to see an antelope/Bambi killed, mauled, and eaten by a hyena)

21

Great enclosures, but no reinforcement for active behaviors

Toys and boomerballs, but noreinforcement for playing with them

SO10: Examples of Behavioral Enrichment

• Servals

22(Who can’t love a face like this? Click…Servals swim in the wild; naturalisticenclosures included ponds – servals didn’t’ swim. Guess what was missing?)

Enrichment for Servals:Honolulu Zoo

23(not squirmish about dead fish; only dead mammals; click; 5-gallon ice, cross-species)

Species typical behavior (swimming) with same reinforcement as in the wild (fish)

Enrichment for Elephants: Honolulu Zoo

24(variation on the same theme: elephant keggers; not beer!)

Species typical behavior (manipulating objects with trunk) with the same reinforcement as in the wild (food)

Enrichment for Langor Monkeys:Honolulu Zoo

25

Species typical behavior (grooming and foraging) with reinforcement (fruit loops)

(mop head on bungee cord, laced with fruit loops, last slide on this)

SO11: Delusions, hallucinations• Chance presents a number of very interesting

cases• Delusions and hallucinations - seeing, hearing

things that aren’t really there (little green men, voices, etc.)

• Sometimes they are organic or have physiological causes - brain injury, Alzheimer’s disease, drugs - sometimes they may be due to operant conditioning, but

• They often can be altered by operant conditioning procedures.

26(back to humans!)

SO11A: What was reinforcing the delusional behavior that her head was falling off?• Woman in a mental institution who believed her

head was falling off.• She seemed quite frightened when this was

occurring and the staff immediately tried to calm her down.

• The delusion got worse - she began to hear “popping” noises right before her head was going to fall off.

27

SO11A Cont.

Behavioral psychologists observed:• She had difficulty approaching staff and engaging

them in conversation.• She had poor social skills so when she did

approach them, the staff responded with annoyance.

• When her head was falling off or when she heard popping sounds prior to her head falling off, the staff paid attention to her and comforted her.

28

SO11B: Intervention componentsSolution?• Taught her better social skills• Taught staff to reinforce her appropriate

(actually, better) social behaviors• Taught staff to extinguish any behavior related to

her head falling offResult?

“Her head remained firmly attached to her body.”

29

(Very nice ethical procedure - next slide)

SO11C: Why is this such a nice example of an ethical intervention?

• Social interaction with staff was a powerful Sr for her (as evidenced by her delusional behaviors - popping sounds, head falling off)

• If they had only extinguished (not to mention punished) the delusional behavior, it would have deprived the woman of an important reinforcer for her - decreasing her “quality of life”

**Identified the powerful reinforcer for her, the one maintaining the inappropriate behavior, then arranged to have that same reinforcer provided for appropriate behavior, thus preserving her quality of life.

30(quality of life, enriched environment – number/quality of reinforcers for behaviorsNote, they did extinguish the delusional behaviors – “head following off”)

SO12: The Haggly Old Witch

• Patient was a young male “schizophrenic” in the psychiatric hospital

• Presenting problem: A haggly old witch kept following him around.

• Medication helped, but he continued to report that she was “dogging” him

• Intervention: Record the strength of his belief that the witch was really there on a 100-point rating scale. – 100 = Absolutely, positively certain– 0 = Witch is not there, it’s my imagination

31

SO12

• Reinforced expressions of doubt• After 26 days, the patient consistently reported

that the witch was all in his imagination!• Think of the implications - the intervention

consisted of “simply” reinforcing the verbal behavior of the client, and it changed his reality.

32

SO13

33

But how do we know that the schizophrenic patient really believed that the witch was no longer following him?

How do we know that the patient simply wasn’ttelling the therapist what he knew the therapistwanted to hear?

SO13• The patient always took a certain tranquilizer when

he believed the haggly old witch was there• The therapist recorded the number of tranquilizers

the patient took, using an ABAB reversal design• Surely enough, during treatment, he did not take as

many tranquilizers, and at the end of treatment he wasn’t taking any.

34

Meyerson & Michael: SDs and S∆s

• Definitions: SDs and S∆ (not for the exam)• My definition of an SD for this class (in SO14):

A stimulus that precedes a response and evokes that response because that particular response has been reinforced in its presence and not in its absence.

• Malott’s definition:A stimulus in the presence of which a response has been reinforced or punished.

• Pietras’ definition:An event that precedes an operant and sets the occasion for the behavior. They change the probability of behavior based on a history of differential reinforcement.

35(animation of this study on Dr. Johnson’s web site, link is on my web page)

SO15: Development and Testing of an SD

36

Development/Training: dolphin to jump and back flipimmediately after seeing a hand signal but not in its absence

SD (hand signal): R (jump and back flip) --->SR (food)

S∆ (no hand signal): R (jump and back flip) --->Ext (no food)

Testing: After repeated SD and S∆ training above, will thedolphin jump and do a back flip ONLY after the hand signal?

SD (hand signal): R (jump and back flip)

S∆ (no hand signal): NO R (does not jump/back flip)

(both SD and S∆ training necessary) (no ext in s∆ testing)

SO15: SD/S∆ Another Example

37

Development/Training: rat to press a lever immediately when a light is on and ONLY when the light is on.

SD (light on): R (press lever) --->SR (water)

S∆ (no light): R (press lever) --->Ext (no water)

Testing: After repeated SD and S∆ training above, will therat press the lever ONLY after the light on?

SD (light on): R (press lever)

S∆ (no light): NO R (no lever press)

SO15: Sample test question

• Sample test question is at the end of SO15• Answer is at the end of the study objectives

for this unit• I guarantee that a similar question will be on

the exam

38

SOs16 &17: Two Important Issues About SDs

• SDs immediately evoke a response - within 5 - 60 sec after the SD occurs.– They do NOT increase the future frequency of a

response. Only consequences affect the future frequency of a response. (-1 on exam)

• SDs precede responses– They do NOT precede other stimuli, e. g.,

yellow light and red light.

39

SOs 19-25: Meyerson & Michael, Introduction

• M&M developed a procedure to test the hearing of nonverbal children diagnosed with developmental disabilities. Ingenious procedure.

• Typical hearing test requires that individuals have a verbal repertoire, that is, have language

• Developed before federal regulations required such hearing tests

• Dickinson’s experience - ten years after the article, undiagnosed hearing problem with child who had spent 3 years in a mental institution

40

(Meyerson, deaf, rehabilitation psychology,Blough technique)

Lee Meyerson Jack Michael

41

Meyerson & Michael Description: Overview

• Ultimate goal/hearing test– Pull the right lever when the child heard a tone– Pull the left lever when he/she did NOT hear a tone

• Why pull left lever when he/she did NOT hear a tone?

42(answer not on slide)

Meyerson & Michael, Overview cont.

• Three 30-min phases– Stimulus discrimination training (SD/S∆ training)– Stimulus fading and stimulus generalization

(bringing lever pulling under the control of the critical stimuli - tone on/tone off)

– Hearing test

43

Basic M&M Apparatus

44

LeftLight

RightLight

LeftLever

RightLever

ReinforcementDispenser Tray

SO20: Phase 1: Two Discriminations: Right Lever Pull and Left Lever Pull

45

Right Lever Pull

SD: R1 ------------------> SR/Sr (VR8 schedule)Tone on (500 cps, 40 dbl) Right lever pull Edibles

R light on, L light off Trinkets

S∆: R1 ------------------> EXTTone off Right lever pull No ediblesR light off, L light on No trinkets

Left Lever Pull

SD: R2 ------------------> SR/Sr (VR8 schedule)Tone off Left lever pull Edibles

R light off, L light on Trinkets

S∆: R2 ------------------> EXTTone on (500 cps, 40 dbl) Left lever pull No ediblesR light on, L light off No trinkets

(SO19: two SD/S∆ disc, two responses. Three stimuli for SD and S∆)

(VR8 schedule-SO20 Nxt slide. If pulled wrong lever, reset the VR8 schedule for that lever)

NFE: M&M Reinforcement Schedule

• Technical Name: VR8• Reinforcement provided after an average of

8 responses– In this case, lever pulls

• What type of response pattern does it generate?

• Why did M&M use this schedule?

46(answers not on slide)

SO21: Why are there two discriminations?

• A “discrimination” consists of an SD and S∆ for a particular response

• There were two responses each with an SD and S∆ – Right level pull:

• SD (tone on, R light on, L light off)• S∆ (tone off, R light off, L light on)

– Left lever pull• SD (tone off, R light off, L light on)• S∆ (tone on, R light on, L light off)

47

Note carefully: it is NOT because the SD and S∆ are made up of more than one stimulus – the critical issueis that there two different responses, each with an SD and S∆

SO 22, Phase 2, Stimulus Fading

• Gradually made the lights over the two levers dimmer and dimmer until they would no longer light up)

• Not for the exam:But why did M&M use the lights as stimuli in the first phase if they were just going to fade them out in the second phase?

48

(both stimulus fading and generalization, 2nd training; answer not on slide; next slide, discriminations after training)

Phase 2: Discriminations after Stimulus Fading of Lights

49

Right Lever Pull

SD: R1 ------------------> SR/Sr (VR8)Tone on Right lever pull Edibles

Trinkets

S∆ : R1 ------------------> EXTTone off Right lever pull No edibles

No trinkets

Left Lever Pull

SD: R2 ------------------> SR/Sr (VR8)Tone off Left lever pull Edibles

Trinkets

S∆: R2 ------------------> EXTTone on Left lever pull No edibles

No trinkets

SOs 23 & 24 Phase 2 Generalization Training• Phase 1: one tone, a 500 cps tone, 40 dcbls• Phase 2: tones of different frequencies presented in

sequence (400, 500, 600, etc.)

50

Why was this stimulus generalization training necessary?

Pulling the right lever was reinforced only when the 500 cps tone was presented in the first phase but they wanted to child to pull the right lever when he/she heard any tone, not just the 500 cps tone.

(bold faced part is essential; explanation, next slide)

SO 24 Generalization Training, cont.

51

Explanation:

During phase 1, pulling the right lever was reinforced only after a 500 cps tone was presented. Wanted:

SD (any tone): R (right lever pull) SD (no tone): R (left lever pull)

But: SD (only 500 cps tone): R (right lever pull) SD (no tone or any other tone): R (left lever pull)

(expect stimulus fixation, DD and autistic, mom new hair cut, dad, new glasses)

SO 25: Meyerson & Michael Hearing Test

• Presented different tones, different loudness in steps of 5 to 10 decibles

• What lever would the child pull if a 600 cps tone at 100 dbls was presented and the child could hear it?

• What lever would the child pull if a 600 cps tone at 40 dbls was presented but the child could not hear it?

52

SO26: SDs & S∆s• Assume in M&M during training, a tone is presented, and the

child pulls the left lever (a “mistake” - more technically, the response has not come under appropriate stimulus control). The tone is what type of stimulus for the left lever pull?

Another example:Suppose Mom is teaching her young child the alphabet. She

reinforces “M” when she holds up a picture of the letter “M”. She extinguishes any other verbal response, like “N”.

The picture M is what type of stimulus for thevocal response of “N”?

53

SO 27: Automaticity of Reinforcement, M&M

• Even though the children did not have verbal behavior (language), they quickly acquired the appropriate discriminations, even though the procedures are, indeed, rather complicated and difficult to explain.

• Strong evidence for the automaticity of reinforcement, that is, that operant conditioning can and often does work without awareness.

i.e., individuals do not have to be able to describe, understand, or be aware of the relationship between the SDs, S∆s, and consequences in order for their behavior to be affected by them.

• Note carefully, it is not the behaviors they are “unaware of” – the point is that the conditioning process works without awareness of the individual.

54

(parents, teachers, informed consent, never work, Maria, if time)

U1 SO 13: Elicit (review)• Simple Rule:

USs elicit URs, CSs elicit CRs_________________________________• Elicit is only used in respondent relations• ONLY USs and CSs elicit responses:

Organisms do NOT• Only responses can be elicited

55

U1 SO13 Examples (review): Is elicit used correctly?

• When the temperature is cold, a person elicits a shivering response.

• The SD (a tone) elicited the behavior of turning in a circle by the pigeon.

• The CS (a tone) elicited the CR (salivation).• In lower order conditioning, the NS elicits the

US.

56

SO 18: Evoke

• Simple rule: USs evoke URs, CSs evoke CRs and SDs evoke Rs

57

Critical features: (you do not have to memorize these)

• Only antecedent stimuli can evoke– Organisms do not evoke responses– Behaviors do not evoke consequences– Reinforcement does not evoke behaviors

• Only responses can be evoked by stimuli

(evoke vs elicit - nxt slide)

SO 18: Evoke vs. Elicit

• Similarities– Only (antecedent) stimuli elicit and evoke – Only responses can be evoked

• Differences– Elicit is used ONLY in respondent relations, e.g., USs

elicit URs, CSs elicit CRs– Evoke can be used in either respondent or operant

relations, e.g., USs evoke URs, CSs evoke CRs, SDs evoke Rs

58

THE END• Questions? • See my web page for animated illustration

of M&M• Instructional assistance hours:

– Monday, 9/23, 5:30-7:00 PM– Wood Hall First Floor Lounge

• Amber will be conducting them

59

60