JOURNAL OF RESEARCH IN PERSONALITY 8, lo%109 (1974)

Some Implications for Psychological Research of Early

Versus Late Term Participation by College Subjects

RON EVANSI AND EDWARD DONNERSTEIN

Southern Illinois University

Several surveys of recent psychological literature have indicated that a large percentage of human subjects are introductory psy-

chology students who are given the option of selecting when to participate in research which is either implicitly or explicitly required of them. The present study was conducted to investigate the pos- sibility that those subjects who choose to participate early in the term differ in personality and in attitudes toward research from later term participants. Results indicated that early term males and females, as expected, were more academically oriented and more in- ternally controlled than late term subjects; and early term males were more achievement oriented than their late term counterparts. Additionally, females were found to feel more positive toward re- search participation than males; while a large percentage of all sub- jects reported both suspiciousness of experimenter intent and a general and unexpected willingness to voluntarily participate as sub- jects. Possible effects of these findings on research results and con- clusions were discussed, and suggestions for improvements aimed at reducing self-selection bias in college subject pools were made.

While nearly three decades have elapsed since McNemar (1946) first labeled psychology as “the science of the behavior of sophomores,” those remarks appear no less true today than they were in the mid-1940’s. Although most facets of psychology have changed during the intervening years, the basic source of data for researchers in human behavior con- tinues to be the readily available college student. In fact, a survey of published studies made by the present authors (Journal of Personality and Social Psychology, 1972, Vols. 21-24 and Journal of Experimental Social Psychology, 1972, Vol. 8) indicated, as do several previous surveys (Smart, 1966; Schultz, 1969; Carlson, 1971; Higbee & Wells, 1972) that approximately 70-80s of studies contained in major psychological journals reporting research with human subjects utilize college students

1 Requests for reprints should be sent to Ron Evans, Department of Psychology, Southern Illinois University, Carbondale, Illinois 62991.

102 Copyright @ 1974 by Academic Press, Inc. All rights of reproduction in any form reserved.

TIME OF SUBJECT PARTICIPATIOK 103

as subjects. In addition, Jung (1969) found that over 90% of subjects used by researchers at 52 major North American institutions were under- graduate students.

These statistics probably come as no great surprise to most researchers, nor should those which indicate that at least 2841% (with a large per- centage unspecified) of the studies surveyed by the present authors used introductory psychology students who participated to fulfill a course rc- quirement. These figures correspond closely to Jung’s report that 45% of all subjects used at the schools he surveyed were obtained by requiring research participation as part of the introductory course requirements.

Although many of these compulsory subject pools came into being at least in part as a reaction to well-established self-selection biases engen- dered by differences between volunteer and nonvolunteer subjects (see Rosenthal & Rosnow, 1969), recent research indicates that such pools may themselves be susceptible to various forms of self-selection by subjects.

Adair (1970)) for example, has shown that subjects who choose to ful- fill their participation requirement early in the term report significantly more positive attitudes toward research than do those who sign up late in t,he term. As noted by Adair, “The volunteer-non-volunteer bias emerges in the form of a beginning-of-the-term willing subject versus an end-of- the-term reluctant participant (Adair, 1973, p. 511.” Because such an analysis appears highly plausible the present investigation was under- taken to explore the hypothesis that subjects who sign up for research participation early in the term differ in predictable ways from those who delay their participation until much later.

Specifically, the early-term ‘%oIunteer,” conceptualized as a stu(lent who views research participation as another academic responsibility to be conscientiously pursued, was expected to be more internally controlled, higher in need for achievement and need for approval, more intelligent and more academically successful than his later-term counterparts. Also, in line with Adair’s findings, early term subject’s were expected to he more willing to participate in research and thus demonstrate more posi- tive attitudes toward their role as subjects.

METHOD

Subjects

Thirty-one male and 49 female subjects. all members of introductory psychology

classes at Southern Illinois University, served as participants in this research. Sub-

jects in this pool select from a number of experiments those they wish to participate in. While subjects are given the alternative of participating in four experiments or

104 EVANS AND DONNERSTEIN

writing reviews of four journal articles as a means of obtaining 12 “optional” points (three points per article or experiment) the fact that at least 95% of all students normally obtain the 12 points, and that less than 3% of those individuals choose to write reviews (as reported by course administrators) clearly suggests that students feel it necessary to participate as subjects. In fact, a brief questionnaire administered as part of the present study indicated that subjects rate “experimental participation is better than writing journal reviews” and “participation is necessary or my grade will suffer” as the two most critical factors among six which might have exerted some influence on their participation. Thus, for all practical purposes, the present pool is a compulsory one.

Design

Research folders entitled “Opinions and Experiments” were made available to subjects during the second and eighth weeks of a lo-wk spring quarter. The resultant design was thus a two (male-female) by two (early versus late) factorial. Because previous participation has been shown to affect attitudes toward research (Holmes, 1967) the experiment was restricted to subjects who had been in one and only one previous study.

Procedure

Subjects participated in groups of &14, with varying proportions of males and females. Subjects were asked to complete a booklet of questionnaires described as assessing some of the thoughts and attitudes held by those who participate in psychology experiments. The following dependent measures were randomly ordered in each booklet: The Rotter Internal-External Control Scale (Rotter, 1966) ; the Information subscale from the Wechsler Adult Intelligence Scale (Wechsler, 1955) ; the Marlowe-Crowne Social Desirability Scale (Crowne & Marlowe, 1969) ; the need for Achievement scale from the Edwards Personal Preference Schedule (Edwards, 1954) and a series of seven-point attitude rating scales (ranging from strongly disagree to strongly agree) designed by the present authors to assess suspiciousness of deception, resentment at being forced to participate, compliance with experi- menter requests to not discuss the study with potential subjects, inclination to try to sabotage (the “screw you” effect; Masling, 1966) the experiment; endorsement of compulsory subject pools and willingness to participate in experiments on a purely voluntary basis.

While no identification was required on each booklet, subjects were asked to leave their names on a blank card included in each set of questionnaires. This re- quest was made after all subjects had completed the experiment in order to guard against subjects possibly biasing their answers through fear of having their identities traced. These names were matched with each booklet after subjects left the experi- ment in order that Grade Point Average and American College Testing (ACT) composite scores could be obtained for each subject.

Because may of the factors assessed in this research could have a potential impact upon self esteem it was deemed unwise to indicate, for example, to late quarter subjects that they were expected to be less intelligent, etc., than early subjects. Consequently, subjects were thoroughly debriefed on the meaning of each scale included in the experiment, and provided with written feedback which included a time and place where results could be obtained.

TIME OF SUBJECT PARTICIPATIOS 105

RESULTS

JIean scores on the six individual difference measures are presented in Table 1. An analysis of variance with time (early versus late) and sex treated as factors on each measure revealed the following results:

fnterzal-external control. The only significant source of variation was that for time [F(1,57) = 10.15, p < .0027]. Thus, subjects participating early in the quarter appear to be more internal (x = 9.975) than late quarter subjects (J? = 13.626).

Intelligence. While early quarter subjects achieved higher mean scores on the WAIS Information subscale, this difference was only marginally significant [F (1,571 = 3.20, p < .0753]. A strong and unexpected main effect for sex was noted, however, [F( 1,57) = 12.19, p < .0013], with early quarter males showing significantly higher scores than either female group. This result suggests that sex differences found in early quarter studies may be contributed to by IQ differences.

Seed for achievement. Results revealed a significant interaction between time and sex [F(1,57) = 4.54, p < .0352]. As expected, late quarter males scored lower than early quarter males, although there were no significant differences across females.

LYeed for approval. No significant effects were found for t.his variable, although means were directionally similar to those observed for need achievement. Consequently while volunteer-nonvolunteer differences in need for approval have been reported (Rosenthal and Rosnow, 1969), such differences do not appear to be an important source of variation within compulsory subject pools such as the one sampled.

C;rcrde point average. Results revealed marginally significant effects for time ]F(1,571 = 2.84, p < .0934], sex [F(1,57) = 3.14, p < .0779], and time by sex [F(1,57) =3.54, p < .0618], While no change across the quart.er was found for female subjects, late quarter males tended t,o have lower CPA’s than their early quarter counterparts [t (57) = 2.51. p < .02, two tailed].

.4C’T composite. Higher ACT scores were observed for males (S = 23.258 versus X = 21.494), although this difference was found to be only of marginal significance [F(1,57) = 3.365, p < .0684]. More importantly, early term subjects were found to have scored significantly higher on the ACT [F(1,57) = 10.162, p < .0027], than had late term research participants.

Attitude responses. Mean attitude scores as measured on seven point rat*ing scales are shown in Table 2. WhiIe no time main effects were noted analysis of variance preformed on these scores did indicate a num- ber of strong sex differences. In fact, females were found to be less

106 EVANS AND DONNERSTZZIX

TABLE 1

MEAN INDIVIDUAL I~IPFERICNCE SCORES BY SEX .4~r1 TIME OF PARTICIPATION

Males

Early, Late, n = 10 n = 13

Females ____--.._

Early, Lat)e,

n = 24 n = 14

I-E control (Rotter I-E Scale) 9.70 14.54 10.25 12.71 Intelligence (WAIS Information) 22.20 20.76 19.21 17.36

Need achievement (Edwards Personal 15.70 12.77 13.63 1.5.71

Preferences) Need approval (Marlowe-Crowne 13.10 11.08 12.08 13.79

Scale) Grade point (5.000 possible) average 3.932 3.415 3.917 3.945

ACT composite scores 24.90 22.92 21.62 20.07

Note. Higher scores indicate, respectively, more external control, intelligence, nACH nApp, academic success (GPA) and academic potent.ial (ACT).

suspicious [F (1,57) = 6.846, p < .02], more compliant [F( 1,57) = 8.683, p < .005], less inclined toward “sabotage” [F(1,57) = 8.471, p < .006], less resentful toward participation [F(1,57) = 9.189, p < .004] and more willing to voluntarily participate in research [F(1,57) = 4.353, p < .04] than male subjects. In addition, a sex X time interaction [F(1,57) = 4.754, p < .04] on the item asking for attitudes toward Southern Illinois’ method of obtaining research subjects was noted, with late quarter females registering more approval for the current system than any other group.

TABLE 2 ME.W SCORES ON ATTITUDES TOWARD EXPERIMENTS

Male Female

Early, Late, Early, Late, n = 10 n = 13 n = 24 n = 14

Suspiciousness 4.00 5.00 3.50 3.29

Resentment 2.50 2.69 1.67 1.64

Compliance 5.40 4.62 6.21 5.86

Sabotage 2.00 2.54 1 .63 1.50

Method of obtaining subjects 4.30 5.08 5.21 5.14 Willingness to volunteer 2.80 1.62 3.25 3.21

Note. High scores indicate greater agreement. A score of one strongly disagree; four neutral; seven strongly agree. For item 6, scores indicate number of experiments subjects would volunteer for.

TIME OF SUBJECT PARTICIPATION 107

DISCUSSION

These results combine to strengthen Adair’s (1973) observation that while compulsory pools help to eliminate volunteer-nonvolunteer differ- ences, this aim is defeated by investigators who allow their subjects to select the time to fulfill their requirement. Since nearly 90% of subjects are allowed this option (Jung, 1969)) this particular form of self selection may foster nonrepresentativeness in many areas. The situation may be magnified even further by the present results indicating that males, who form the strong majority of psychological subjects (Holmes & Jorgensen, 1973) are especially likely to show differences across the quarter.

Unfortunately, it would appear that these differences might have numer- ous opportunities to affect psychological research. Carlson (1971) has reported, for example, that 78% of the studies she surveyed were single session experiments, many of which are no doubt completed quickly (i.e., in l-3 wk) at the beginning or end of the term. Similarly, later-term studies based upon pilot data gathered early in the term could also be affected by the differences discussed in this report. In all of these cases, and probably many more, the effects of early-late differences can poten- tially produce an over- or underestimation of results obtained.

To date, however, the only direct evidence on this issue seems to come from Underwood, Schwenn and Keppel (1964’)) who found no differences in verbal learning performance between early and late term subjects. This issue of performance differences has yet to be extended to areas of more direct concern to personality and social psychological researchers, however, and it appears that future research should be undertaken to assess what, if any, differential effects on performance result from the differences reported above.

Pending the evaluation of such effects, it would appear wise for researchers to sample across the term, and perhaps even across more than one term (one wonders, for example, if summer school subjects might be a “different breed” from those sampled at other times) in order to avoid potentially nonrepresentative results. Another alternative might involve shifting the emphasis, as some have suggested (Johnson, 1973; Jung 1969) from subject to experimenter selection of the time and the experi- ment. Under such a plan students in the subject pool could complete information forms at the beginning of each quarter. Data contained could range from age and GPA to scores on personality variables to he investigated by departmental researchers that term. From this point. researchers could randomly select subjects from those fitting their re- search needs and then initiate contacts and set up appointments thcm- selves. Since college subject pools will probably continue to he a major source of research participants for many investigators, a sp&em modeled

108 EVANS AND DONNERSTEIN

along this outline could aid greatly in eliminating self-selection biases which presently appear to be operating in many experimental situations.

In addition to personality differences several important features of the attitude responses also deserve some further comment. First, unlike pre- vious research of this type (Adair, 1970), late subjects did not indicate more negative attitudes toward research than early term subjects. Thus, contrary to predictions, differences in attitudes do not, in this pool at least, appear to affect time of participation.

Secondly, females, who form the clear minority of experimental sub- jects, seem to hold more of the attitudes most researchers tend to prize (i.e., willingness to participate) than do males. Given Carlson’s report that 74% of studies she surveyed where sex was a variable did indeed find sex differences, it does seem possible that these attitudinal differences may, in some cases, be translated into observed differences in perform- ance. Certainly, the issue warrants increased investigation.

Finally, in view of the essentially coercive nature of the subject pool under study, it is somewhat surprising that more negative attidudes to- ward research participation did not appear. Several authors (e.g., Argyris, 1970; Adair, 1973), have discussed the potential damage done to research by negativism assumed to result when students are required to participate as subjects. No such negativism appears in these results, however. Not a single subject indicated agreement with the item designed to assess the sabotage or ‘(screw you” effect (Masling, 1966), while only one male indicated resentment over being a subject. Most surprisingly, 50% of males and 84% of females indicated that they would voluntarily participate in at least two experiments, while 22% of males and 68% of females reported that they would volunteer (in the absence of any grade credit) for a number of experiments equal to or exceeding that currently required (four) by the introductory course at Southern Illinois. These results certainly suggest that the apparent high degree of interest shown toward psychology by undergraduates (News- week, 1973) may be yielding as a byproduct subjects who are genuinely interested in being of assistance to researchers in human behavior. While this phenomenon could be explained in other ways (i.e., resignation to the system, or dissonance reduction) it does appear worthwhile to continue to monitor subject attitudes with an eye toward making more active par- ticipants of interested subjects who currently play a rather passive role in most psychological research.

In conclusion, it seems clear that a great deal of research needs to be undertaken on the problems discussed in this report. Perhaps the situation was best summarized by Johnson (1973)) who wrote that “Better research will not be forthcoming until university researchers concern themselves

TIME OF SUBJECT PARTICIPATIOS 109

with the problems of obtaining human subjects (,p. ZO).” The present authors support this view.

REFERENCES

AD.~IR, J. Pre-experiment attitudes toward psychology as a determinant of subject behavior. Presented in symposium : “Methodological problems in re- search with human subject” at annual meeting of the Canadian Psychological Association, May, 1970, Winnipeg, Canada.

ADAIR, J. The Izuwzan subject: the sociul psychology oj the psychological experimellf.

Boston: Little, Brown, 1973. ARGTRIS. C. Some unintended consequences of rigorous research. Psychological

Bulletin, 1970, 70, 185-197. CARLSON. R. Where is the person in personality research? Psychological RuZ/etin.

1971. 75, 203-219. CROWNE. D., & MARLOWE, D. A new scale of social desirability independent of

psychopathology. Journal of Consulting Psychology, 1960, 24, 349-354. EDWARDS, 4. The Edwards personal preference schedule. New York: Psychological

Corp. 1954. HIGBEE. K.. & WELLS, M. Some research trends in social psychology during the

1969’s. American Psychologist, 1972 2’1, 963-966. HOLMES. D. Amourlt of experience in experiments as a determinant. of performance

in later expermients. Journal of Personality and Socicil Psychology, 1967. 7, 403-404.

HOLMES, D., & JORGENSEN, B. Do personality and social psychologists study men more than women? Representative Research in Social Psychology, 1971, 2, 71-76.

JOHNSON. It. The obtaining of experimental subjects. Canadian Psychologtit, 1973. 14, 298-211.

JIJNG. J. Current practices and problems in the use of college students for psycho- logical research. Canadian Psychologist, 1969, 10, 280-290.

MCKEMAR. Q, Opinion-attitude methodology. Psychological Bulletin. 1946. 43,

289374. MASLING, J. Role-related behavior of the subject and psychologist and its effect

upon psychological data. In D. Levine (Ed.), The Nebraska symposium. on motivation. Lincoln, NB: University of Nebraska Press, 1966. Pp. 67-103.

Newsweek. Psychology: Hot course on campus. May 21, 1973. ROTTER, J. Generalized expectancies for internal versus external control of rcin-

forcement. Psychologicul Monographs, 1966, 80, (1) (Whole No. 609). ROSENTHAL. R., & ROSNOW, R. The volunteer subject. In R. Rosenthal and

R. Rosnow (Eds.) Artifact in behavioral research. New York: Academic Press. 1969. Pp. 59-118.

SCHCLTZ. D. The human subject in psychological research. PsychologicaE Rullelitz. 1969, 72, 214-228.

SMART. R. Subject selection bias in psychological research. Canadin?l Psychologist,

1966. 7, 115121. UTNDERW~OD. B., SCHWENN, E.! &I KEPPEL, G. Verbal learning as related to point, in

time in the school term. Jou:-)zaI of Verbal Lenming and Verbal Behnvior,

1964, 3, 222-225.