+ All Categories
Home > Documents > The Hitchhiker's Guide to Altruism: Gene-culture Coevolution, and the Internalization of Norms

The Hitchhiker's Guide to Altruism: Gene-culture Coevolution, and the Internalization of Norms

Date post: 14-Nov-2023
Category:
Upload: umass
View: 0 times
Download: 0 times
Share this document with a friend
12
J. theor. Biol. (2003) 220, 407–418 doi:10.1006/jtbi.2003.3104, available online at http://www.idealibrary.com on The Hitchhiker’s Guide to Altruism: Gene-culture Coevolution, and the Internalization of Norms Herbert Gintis n Santa Fe Institute and Department of Economics, University of Massachusetts, 15 Forbes Avenue, MA 01060, Northampton, U.S.A. (Received on 5 December 2001, Accepted in revised form on 4 June 2002) An internal norm is a pattern of behavior enforced in part by internal sanctions, such as shame, guilt and loss of self-esteem, as opposed to purely external sanctions, such as material rewards and punishment. The ability to internalize norms is widespread among humans, although in some so-called ‘‘sociopaths’’, this capacity is diminished or lacking. Suppose there is one genetic locus that controls the capacity to internalize norms. This model shows that if an internal norm is fitness enhancing, then for plausible patterns of socialization, the allele for internalization of norms is evolutionarily stable. This framework can be used to model Herbert Simon’s (1990) explanation of altruism, showing that altruistic norms can ‘‘hitchhike’’ on the general tendency of internal norms to be personally fitness-enhancing. A multi-level selection, gene-culture coevolution argument then explains why individually fitness-reducing internal norms are likely to be prosocial as opposed to socially harmful. r 2003 Elsevier Science Ltd. All rights reserved. Introduction An internal norm is a pattern of behavior enforced in part by internal sanctions, including shame, guilt and loss of self-esteem, as opposed to purely external sanctions, such as material rewards and punishments. Humans internalize norms through socialization by parents (vertical transmission) and extraparental conspecifics (ob- lique and horizontal transmission). The capacity to internalize norms is widespread among hu- mans, although in some so-called ‘‘sociopaths’’, this capacity is diminished or lacking (Mealey, 1995). Human behavior is commonly modeled assuming agents have objective functions which they maximize subject to constraints. In these terms, the capacity to internalize norms means human agents have socially programmable ob- jective functions. Human behavior thus depends not only on beliefs, which concern constraints on action (taking action X will lead to result Y ), but values, which are the very goals of action. Suppose there is one genetic locus that controls the capacity to internalize norms. I develop models of gene-cultural coevolution to show that if an internal norm is fitness enhan- cing, then the allele for internalization of norms is evolutionarily stable. Moreover, if the fitness payoff to the internalized norm is sufficiently large, or if there is a sufficiently high rate of phenotypic level assortative mating, the allele for internalization is globally stable. Basic sociological theory holds that society’s values are transmitted through the internaliza- tion of norms (Parsons, 1967; Grusec & Kuczynski, 1997). Successful societies tend to foster internal norms that enhance personal n Tel.: +1-413-586-7756; fax: +1-413-586-6014. E-mail address: [email protected] (H. Gintis). URL: http://www-unix.oit.umass.edu/Bgintis. 0022-5193/03/$35.00 r 2003 Elsevier Science Ltd. All rights reserved.
Transcript

J. theor. Biol. (2003) 220, 407–418doi:10.1006/jtbi.2003.3104, available online at http://www.idealibrary.com on

The Hitchhiker’s Guide to Altruism: Gene-culture Coevolution,and the Internalization of Norms

Herbert Gintisn

Santa Fe Institute and Department of Economics, University of Massachusetts, 15 Forbes Avenue,MA 01060, Northampton, U.S.A.

(Received on 5 December 2001, Accepted in revised form on 4 June 2002)

An internal norm is a pattern of behavior enforced in part by internal sanctions, such asshame, guilt and loss of self-esteem, as opposed to purely external sanctions, such as materialrewards and punishment. The ability to internalize norms is widespread among humans,although in some so-called ‘‘sociopaths’’, this capacity is diminished or lacking. Supposethere is one genetic locus that controls the capacity to internalize norms. This model showsthat if an internal norm is fitness enhancing, then for plausible patterns of socialization, theallele for internalization of norms is evolutionarily stable. This framework can be used tomodel Herbert Simon’s (1990) explanation of altruism, showing that altruistic norms can‘‘hitchhike’’ on the general tendency of internal norms to be personally fitness-enhancing. Amulti-level selection, gene-culture coevolution argument then explains why individuallyfitness-reducing internal norms are likely to be prosocial as opposed to socially harmful.

r 2003 Elsevier Science Ltd. All rights reserved.

Introduction

An internal norm is a pattern of behaviorenforced in part by internal sanctions, includingshame, guilt and loss of self-esteem, as opposedto purely external sanctions, such as materialrewards and punishments. Humans internalizenorms through socialization by parents (verticaltransmission) and extraparental conspecifics (ob-

lique and horizontal transmission). The capacityto internalize norms is widespread among hu-mans, although in some so-called ‘‘sociopaths’’,this capacity is diminished or lacking (Mealey,1995). Human behavior is commonly modeledassuming agents have objective functions whichthey maximize subject to constraints. In theseterms, the capacity to internalize norms means

nTel.: +1-413-586-7756; fax: +1-413-586-6014.E-mail address: [email protected] (H. Gintis).URL: http://www-unix.oit.umass.edu/Bgintis.

0022-5193/03/$35.00

human agents have socially programmable ob-jective functions. Human behavior thus dependsnot only on beliefs, which concern constraints onaction (taking action X will lead to result Y ), butvalues, which are the very goals of action.

Suppose there is one genetic locus thatcontrols the capacity to internalize norms. Idevelop models of gene-cultural coevolution toshow that if an internal norm is fitness enhan-cing, then the allele for internalization of normsis evolutionarily stable. Moreover, if the fitnesspayoff to the internalized norm is sufficientlylarge, or if there is a sufficiently high rate ofphenotypic level assortative mating, the allele forinternalization is globally stable.

Basic sociological theory holds that society’svalues are transmitted through the internaliza-tion of norms (Parsons, 1967; Grusec &Kuczynski, 1997). Successful societies tend tofoster internal norms that enhance personal

r 2003 Elsevier Science Ltd. All rights reserved.

wFor reviews of the evidence on the importance ofaltruism in human societies, see Sober & Wilson (1998),Gintis (2000a, b), and Fehr & Gachter (2002).zAssuming u40 is conservative, in that it biases the

model against the global stability of the internalizationallele. However, the contrasting assumption uo0 is alsoplausible. I will point out the implications of uo0 whereappropriate.yFeldman et al. (1985) develop a model similar to ours.

Their model, however, is haploid, assumes uniparentaltransmission, and the phenotypic trait is kin-altruistic.Ours, by contrast, is diploid, assumes biparental transmis-sion, and abstracts from kin-altruism.

H. GINTIS408

fitness, such as future-orientation, good personalhygiene, positive work habits, and control ofemotions, as well as altruistic norms thatsubordinate the individual to group welfare,fostering such behaviors as bravery, honesty,fairness, willingness to cooperate, and empathyfor others (Brown, 1991). People follow internalnorms because they value this behavior for itsown sake, in addition to, or despite, the effectsthe behavior has on personal fitness and/orperceived well-being. For instance, an individualwho has internalized the value of ‘‘speakingtruthfully’’ will do so even in cases where the netpayoff to speaking truthfully would otherwise benegative. It follows that where people internalizea norm, the frequency of its occurrence in thepopulation will be higher than if people followthe norm only instrumentally; i.e. when theyperceive it to be in their interest to do so.

Why does the capacity to internalizenorms have adaptive value? It might be arguedthat if a norm is fitness enhancing, a non-internalizing agent could simply mimic thebehavior of an internalizer. But this assumesthat agents maximize fitness. In general, how-ever, in any species, individuals do not maximizefitness but rather a objective function that hasevolved to reflect biological fitness more or lessaccurately for a given environment. If the Homo

sapiens objective function were perfectlyadapted, internalization would not be fitness-enhancing. But the rapid cultural evolution andhighly variable environments (Richerson et al.,2001) that characterized the period in whichHomo sapiens developed doubtless led to asituation in which the unsocialized humanobjective function deviated strongly from fitnessmaximization. The internalization of norms thuspermitted rapid cultural adaptation towardsfitness-maximization, while a purely geneticadaptive process would have taken orders ofmagnitude longer in time. In short, non-internalizers fail to mimic internalizers notbecause they cannot, but because theydo not want to (sociopaths, for instance, do oftenmimic internalizersFdisplaying empathy andhelpfulness, for exampleFbut only as long asit suits them to do so).

Altruism is the tendency of individuals tobehave prosocially towards unrelated others (e.g.

by helping those in distress and punishing anti-social behavior) at personal cost.w Adding analtruism norm allows us to model HerbertSimon’s (1990) explanation of altruism. Simonsuggested that altruistic norms could ‘‘hitch-hike’’ on the general tendency of internal normsto be personally fitness-enhancing. Of course,internal norms may persist even if they arefitness-reducing both for individuals and thegroup (Boyd & Richerson, 1992; Edgerton,1992). I develop a multi-level gene-culturecoevolutionary model to elucidate the processwhereby altruistic internal norms will tend todrive out norms that are both socially harmfuland individually fitness-reducing.

Socialization and Fitness-EnhancingInternal Norms

Suppose there is a norm C that can beinternalized by a new member of society. NormC confers fitness 1þ t41; while the normlessphenotype, denoted by D, has baseline fitness 1.There is a genetic locus with two alleles, {a} and{b}. Allele {a}, which is dominant, permits theinternalization of norms, whereas {b} does not.We assume that possessing at least one copy of aimposes a fitness cost uAð0; 1Þ; on the grounds thatthere are costly physiological and cognitive pre-requisites for the capacity to internalize norms.zWe assume ð1þ tÞð1� uÞ41; so the cost of theinternalization allele is more than offset by thebenefit of the norm C. There are five phenogeno-types, whose fitnesses are listed in Fig. 1.y

Families are formed by random pairing, andoffspring genotypes obey the laws of Mendeliansegregation. Thus, there are six familialgenotypes, aaaa, aaab, aabb, abab, abbb, andbbbb. We assume also that only the phenotypic

Individual IndividualPhenogenotype Fitness

aaC (1-u)(1+t)aaD (1-u)abC (1-u)(1+t)abD (1-u)bbD 1

Fig. 1. Fitnesses of the five phenogenotypes. Here u isthe fitness cost of possessing the internalization allele, and tis the excess fitness value of possessing the norm C. Notethat bbC cannot occur.

HITCHHIKER’S GUIDE TO ALTRUISM 409

traits of parents, and not which particularparent expresses them, are relevant to thetransmission process. Therefore, there are threefamilial phenotypes, CC, CD, and DD, and 18familial phenogenotypes, of which only 14 canoccur. The frequency of familial phenogeno-types are as shown in Fig. 2, where pðiÞrepresents the frequency of phenogenotype i ¼aaC;y; bbD:

The rules of gene-culture transmission are asfollows. If familial phenogenotype is xyzwXY,where x,y,z,wA{a,b}, X, YA{C,D}, an offspringis equally likely to inherit xz, xw, yz, or yw. Anoffspring whose genotype includes a copy of thea allele is equally likely to inherit X or Y.8 Butan offspring of genotype bb always has thenormless phenotype D. The transition table isshown in Fig. 3.z

The above accounts only for parental trans-mission. In addition, extraparental transmissionis ubiquitous in human society, in the form ofsocial pressure (rumor, shunning, and ostra-cism), rituals (dancing, prayer, marriage, birth,and death), and in modern societies, formalizedinstitutions (schools and churches).** Toaccount for extraparental transmission, let pC

8 Simulations show that the assumption that the a alleleis dominant is not critical. The stability results describedbelow continue to hold, and indeed more strongly, when a

is less than fully dominant.zBiased parental transmission, in which heterogeneous

familial phenotypes are more likely to transmit onephenotype to offspring that the other (Cavalli-Sforza &Feldman, 1981) is discussed below.

nnExtraparental transmission is generally individuallycostly and the benefits accrue to unrelated others. Hence itis a form of altruistic behavior, and ideally should not beintroduced until our analysis of altruism is completed. Weintroduce it now purely for expositional purposes.

be the fraction of the population carrying the Cphenotype, and let gA½0; 1�: We assume afraction gpC of aa-types and a fraction npC ofab-types who have not internalized C throughparental transmission, are influenced by extra-parental transmission to switch to C. bb

types are not affected by extraparental trans-mission.

The resulting system consists of four equa-tions in four unknowns (bbC cannot occur, andone offspring phenogenotype is dropped, sincethe sum of phenogenotypic frequencies equalsunity). It is straightforward to check that thereare three pure equilibria (i.e. equilibria in whichthe whole population bears a single phenogen-otype). These are aaC, in which all agentsinternalize the fitness enhancing norm, aaD, inwhich the internalization allele is present butthe phenotype C is absent, and bbD, in whichneither the internalization allele nor the normis present.

A check of the eigenvalues of the Jacobianmatrix of the dynamical system shows that theaaD equilibrium is unstable. Eigenvalues of thesystem at the aaC equilibrium are given by

0; 1;1� g

2ð1þ tÞ;1� g1þ t

� �:

The unit eigenvalue is semisimple,ww so thelinearization of the equilibrium aaC, in whichthe fitness-enhancing norm is internalized, isstable. However, we cannot conclude that thenonlinear model itself is stable. Extensive simu-lations fail to find a case in which the aaC

equilibrium is unstable.zz Moreover, in the casewhere the a allele is incompletely dominant, theunit root disappears, so stability is assured (thisremark applies as well to all cases in which unitroots appear, some of which are discussedbelow).

wwAn eigenvalue is semisimple if its algebraic andgeometric dimensions are equal. Semisimple unit roots oflinear dynamical systems are stable.zzThe process of coding this and the other models

presented in this paper is tedious and error-prone. Toensure accuracy I wrote the simulations in two completelydifferent languages, one Lisp-like (Mathematica) and theother procedural (C++), and verified that the resultsagreed to six decimal places over thousands of generationsof simulation.

Familial Frequency inPhenogenotype Reproductive PoolaaaaCC p( aaC)2(1 _ u)2(1 + t)2/ paaaaCD 2p( aaC)p (aaD)(1 _ u)2(1 + t)/ paaaaDD p( aaD)2(1 _ u)2/ paaabCC 2p( aaC)p (abC)(1 _ u)2(1 + t)2/ paaabCD 2(p (aaC)p (abD) + p( aaD)p (abC))(1 _ u)2(1 + t)/ paaabDD 2p( aaD)p (abD)(1 _ u)2/ pababCC p( abC)2(1 _ u)2(1 + t)2/ pababCD 2p( abC)p (abD)(1 _ u)2(1 + t)/ pababDD p( abD)2(1 _ u)2/ paabbCD 2p( aaC)p (bbD)(1 _ u)( 1 + t)/ paabbDD 2p( aaD)p (bbD)(1 _ u)/ pabbbCD 2p( abC)p (bbD)(1 _ u)( 1 + t)/ pabbbDD 2p( aaC)p (aaD)(1 _ u)( 1 + t)/ pbbbbDD 2p( bbD)2/ p

Fig. 2. Frequencies of phenogenotypes. Here, %p is chosen so the sum of the frequencies is unity. Note that aabbCC,abbbCC, bbbbCC, and bbbbCD are not listed, since bbC cannot occur.

Familial Offspring Phenogenotypic FrequencyType aaC aaD abC abD bbD

aaaaCC 1aaaaCD 1/2 1/2aaaaDD 1aaabCC 1/2 1/2aaabCD 1/4 1/4 1/4 1/4aaabDD 1/2 1/2aabbCD 1/2 1/2aabbDD 1abbbCD 1/4 1/4 1/2abbbDD 1/2 1/2ababCC 1/4 1/2 1/4ababCD 1/8 1/8 1/4 1/4 1/4ababDD 1/4 1/2 1/4bbbbDD 1

Fig. 3. Phenotypic inheritance is controlled by geno-type. Note that aabbCC, abbbCC, bbbbCC, and bbbbCD

are not listed, since bbC cannot occur.

yyThe above result depends on our assumption ofunbiased parental transmission. Suppose, however, that afraction of offspring who would acquire norm C underunbiased transmission in fact acquire D. In this case,inspection of the eigenvalues of the Jacobian tells us thatthe aaC equilibrium is locally stable provided dot and thebbD equilibrium is stable provided ð1� dÞð1þ tÞð1� uÞo2:Thus, parental transmission biased against the internaliz-able norm C is hostile to internalization.

H. GINTIS410

The eigenvalues of the Jacobian matrix of theunnormed equilibrium bbD are given by

ð0; 0; 1� u; 12ð1þ tÞð1� uÞ

� �:

Therefore this equilibrium, in which no inter-nalization occurs, is locally stable if ð1þ tÞð1�uÞo2; and unstable when the opposite inequalityholds. There may exist equilibria involving morethan one type of behavior, although the system istoo complex to determine whether or not this isthe case. Extensive simulations suggest that ifsuch equilibria exist, they are not stable. I shall

assume this is the case in this paper. It followsthat for t42=ð1� uÞ � 1; aaC is a globally stableequilibrium.yy

There are four plausible conditions that renderthe bbD equilibrium unstable, in which case aaCwill be globally stable. The first is uo0; whichmeans that the apparatus upon which internali-zation depends has net positive (pleiotropic)fitness effects independent from its contributionto the internalization of norms. The second isthat t is sufficiently large that ð1þ tÞð1� uÞ42:Third, if parental transmission is sufficientlybiased in favor of C, the internalization equili-brium is globally stable.

The fourth condition leading to the globalstability of the aaC equilibrium is that there issome assortative mating that overcomes thetendency of the internalization allele to become‘‘diluted’’. Suppose each type mates with anotherof its type with probability z; and with anrandom member of the population withprobability 1� z: Then the eigenvalues of the

HITCHHIKER’S GUIDE TO ALTRUISM 411

bbD equilibrium become

f12zð1� uÞ; 1

2zð1þ tÞð1� uÞ; 1� u;

12zð1þ tÞð1� uÞð1þ zÞg: ð1Þ

Therefore, there is always a degree of assortativemating that renders the bbD equilibrium un-stable. Thus, it is plausible that some combina-tion of assortative mating, parental transmissionbiased towards the internal norm, and highreturns to the internal norm, assures the globalstability of the aaC equilibrium.88

Altruism

We now add a second dichotomous pheno-typic trait with two variants. Internal norm A isaltruistic in the sense that its expression benefitsthe group, but imposes fitness loss sAð0; 1Þ onthose who adopt it. The normless state, B, isneutral, imposing no fitness loss on those whoadopt it, but also no gain or loss to othermembers of the social group.

We assume A has the same cultural transmis-sion rules as C: individuals who have a copy ofallele a inherit their phenotypes from theirparents, while bb individuals always adopt thenormless phenotype BD. In addition, there isextraparental transmission, as before. There arenow three genotypes and four phenotypes,giving rise to nine phenogenotypes that canoccur, which we denote by aaAC, aaAD, aaBC,aaBD, abAC, abAD, abBC, abBD, and bbBD,and three that cannot occur, bbAC, bbAD, andbbBC. We represent the frequency of pheno-genotype i by pðiÞ; for i ¼ aaAC;y;bbBC:

We maintain the assumption that families areformed by random pairing and the offspringgenotype obeys Mendelian segregation. Weassume also that only the phenotypic traitsof parents, and not which particular parentexpresses them, are relevant to the transmissionprocess. Therefore there are nine family pheno-types, which can be written as AACC, AACD,AADD, ABCC, ABCD, ABDD, BBCC, BBCD,and BBDD. It follows that there are 54 familial

88The same results hold for a haploid version of themodel, except that there is no unit root in the aC

equilibrium. Moreover, if a is not completely dominant inthe diploid model, the unit root disappears and theequilibrium is unambiguously stable.

phenogenotypes, which we can write asaaaaAACC,y,bbbbBBDD, only 36 of whichcan occur. We write the frequency of familialphenogenotype j as pð jÞ; and we assume thepopulation is sufficiently large that we can ignorerandom drift. For illustrative purposes, here are afew of the phenogenotypic frequencies:

pðaaaaAACCÞ

¼ pðaaACÞ2ð1� sÞ2ð1þ tÞ2ð1� uÞ2= %p;

pðaaaaAACDÞ

¼ pðaaACÞpðaaADÞð1� sÞ2ð1þ tÞð1� uÞ2= %p;

pðaaaaAACDÞ ¼ 2pðabACÞpðabBDÞ

pðabADÞpðabBCÞÞð1� sÞð1þ tÞð1� uÞ2= %p;

pðbbbbBBDDÞ ¼ pðbbBDÞ2= %p;

and so on, where %p is chosen so the sum of thefrequencies is unity:

%p ¼ pðaaaaAACCÞ þ?þ pðbbbbBBDDÞ:

The rules of cultural transmission are as before.If familial phenogenotype is xyzwXYZW, wherex,y,z,wA{a,b}, X,YA{A,B}, and Z,WA{C,D}, anoffspring is equally likely to inherit xz, xw, yz, oryw. An offspring whose genotype includes a copyof the a allele is equally likely to inherit X or Y,and equally likely to inherit Z orW. Offspring ofbb geneotype always have the normless pheno-type BD, unless they are socialized extraparen-tally.zz The transition table is shown in Fig. 4.

We assume both genotypic and phenotypicfitness, as well as their interactions, aremultiplicative. Thus, the fitness of the ninephenogenotypes that can appear with positivefrequency are as shown in Fig. 5. The resultingsystem consists of eight equations in eight of thenine offspring phenogenotypes. One offspringphenogenotype is dropped, since the sum ofphenogenotype frequencies must be unity.

It is straightforward to check that there arefive pure equilibria. These are aaAC, in which allagents internalize both the altruistic and fitnessenhancing norms, aaAD, in which only thealtruistic norm is internalized, aaBC, in which

zzExtensive simulations show that if a is incompletelydominant, the results described below continue to hold.

Familial Offspring Phenogenotypic Frequencytype aaAC aaAD aaBC aaBD abAC abAD abBC abBD bbBD

aaaaAACC 1aaaaABCC 1/2 1/2aaaaBBCC 1aaaaAACD 1/2 1/2aaaaABCD 1/4 1/4 1/4 1/4aaaaBBCD 1/2 1/2aaaaAADD 1aaaaABDD 1/2 1/2aaaaBBDD 1aaabAACC 1/2 1/2aaabABCC 1/4 1/4 1/4 1/4aaabBBCC 1/2 1/2aaabAACD 1/4 1/4 1/4 1/4aaabABCD 1/8 1/8 1/8 1/8 1/8 1/8 1/8 1/8aaabBBCD 1/4 1/4 1/4 1/4aaabAADD 1/2 1/2aaabABDD 1/4 1/4 1/4 1/4aaabBBDD 1/2 1/2aabbABCD 1/4 1/4 1/4 1/4aabbBBCD 1/2 1/2aabbABDD 1/2 1/2aabbBBDD 1abbbABCD 1/8 1/81/81/8 1/2abbbBBCD 1/4 1/4 1/2abbbABDD 1/4 1/4 1/2abbbBBDD 1/2 1/2ababAACC 1/4 1/2 1/4ababABCC 1/8 1/8 1/4 1/4 1/4ababBBCC 1/4 1/4 1/2ababAACD 1/8 1/8 1/4 1/4 1/4ababABCD 1/16 1/16 1/16 1/16 1/8 1/8 1/8 1/8 1/4ababBBCD 1/8 1/8 1/4 1/4 1/4ababAADD 1/4 1/2 1/4ababABDD 1/8 1/8 1/4 1/4 1/4ababBBDD 1/4 1/2 1/4bbbbBBDD 1

Fig. 4. Cultural and biological transition parameters.

Individual Individual Individual IndividualPhenogenotype Fitness Phenogenotype FitnessaaAC (1-u)(1-s)(1+t) aaAD (1-u)(1-s)aaBC (1-u)(1+t) aaBD (1-u)abAC (1-u)(1-s)(1+t) abAD (1-u)(1-s)abBC (1-u)(1+t) abBD (1-u)bbBD 1

Fig. 5. Payoffs to nine phenogenotypes.

nnnWhen allele a is incompletely dominant, the unit rootdisappears, so the aaAC equilibrium is unambiguouslystable.

H. GINTIS412

only the fitness-enhancing norm is internalized,aaBD, in which agents carry the gene forinternalization of norms, but no norms are infact internalized, and bbBD, in which internali-zation is absent, and neither altruistic nor fitness-enhancing norms are transmitted from parentsto offspring. A check of the eigenvalues of theJacobian matrix shows that the aaAD and theaaBD equilibria are unstable.

The Jacobian of the altruistic internalizationequilibrium aaAC has eigenvalues

0; 1;1

2ð1þ tÞ;

1

ð1þ tÞ;1� g1� s

;1� g

2ð1� sÞð1þ tÞ;

1� n4ð1� sÞð1þ tÞ

;1� n

4ð1� sÞð1þ tÞ

�:

It is easy to check that the linearization of thisequilibrium is stable if sog; since the unit root issemisimple. We cannot conclude that the equili-brium itself is necessarily stable for all para-meters s; t; u; g; and n satisfying the above

inequalities. However, many simulations undervarying parameter sets have failed to turn up aninstance of instability.***

The eigenvalues of the Jacobian for the aaBCequilibrium are

0; 1;1

2ð1þ tÞ;

1

ð1þ tÞ;

1

2ð1þ tÞ2ð1� uÞ2;

1

2ð1� sÞ;

1� s

4ð1þ tÞ; 1þ g� s

�:

wwwThis model also has a semisimple unit root, sostability was checked by extensive simulations. A similarresult holds for the haploid model.zzzThis model also has a semisimple unit root, so

stability was checked by extensive simulations. A similarresult holds for the haploid model, except the relevantinequality for stability of aaBC becomes the much stronger,and hence implausible, inequality a4ðg� sÞ=sð1þ g� sÞ:

HITCHHIKER’S GUIDE TO ALTRUISM 413

Thus, aaBC is stable when gos; and unstablewhen the opposite inequality holds.

Finally, the eigenvalues of the Jacobian for thebbBD equilibrium are

f0; 0; 0; 0; 1� u; 12ð1þ tÞð1� uÞ;

14ð1� sÞð1þ tÞð1� uÞ; 12ð1� sÞð1� uÞg:

As in the single phenotype case, this is unstable ifuo0 or ð1þ tÞð1� uÞ42; and is stable if eitherof the opposite inequalities hold. Moreover, itcan be shown that adding assortative matingleads to the instability of the bbBD equilibriumunder the same conditions as in the singlephenotype case, shown in eqn (1).

In sum, under plausible conditions, one inter-nalization equilibrium is stableFthe altruismequilibrium when g4s and the nonaltruismequilibrium when s4g: Since we expect s to besmall, whereas the ubiquity of extraparentaltransmission favors a high g; the altruismequilibrium appears the more plausible of thetwo. Under not implausible conditions, either ahigh return to the internal norm, assortativemating of agents that internalize, or pleiotropismin the form uo0; the only stable equilibrium ofthe system involves internalization. The dynamics,which we present below, support this conclusion.

Copying High-Fitness Phenotypes:The Replicator Dynamic

The above models of cultural transmissionhave been strongly criticized in the literature forsuggesting that agents adopt norms independentof their perceived payoffs. In fact, people do notalways blindly follow the norms that have beeninculcated in them, but at least at times treatcompliance as a strategic choice (Wrong, 1961;Gintis, 1975). The ‘‘oversocialized’’ model of theindividual developed above may be improved byadding a phenotypic copying process reflectingthe fact that agents shift from lower to higherpayoff strategies. We represent this process asa replicator dynamic (Taylor and Jonker, 1978;Samuelson, 1997; Nowak and Sigmund, 1998;Gintis, 2000b). In the current context, there arefour phenotypes whose relative fitness ranksthem as BC4AC4BD4AD, and only agentswith a copy of the a allele will copy another

phenotype, since only such types are capable ofinternalizing a norm, and non-internalizers willnot desire to mimic internalizers.

We assume an agent with the a allele andphenotype XY meets an agent of type WZ withprobability apWZ; where pWZ is the fractionof the population with phenotype WZ, andswitches to WZ if that type has higher fitnessthan XY. The parameter a is the measure of thestrength of the tendency to shift to high-payoffphenotypes.

It is easy to see that adding a replicatordynamic does not change the single phenogen-otype equilibria. By checking the eigenvalues ofthe Jacobian matrix, we find that the aaAD andaaBD equilibria remain unstable, and the repli-cator dynamic does not affect the conditionsfor stability of the unnormed equilibrium bbBD.The condition g4s for stability of the altruismequilibrium aaAC now becomes

aog� s

1� g; ð2Þ

so a sufficiently strong replicator dynamic canundermine the stability of the aaAC equili-brium.www The condition s4g for stability ofthe non-altruism internalization equilibriumaaBC when the replicator dynamic is includednow becomes

a4g� s

1þ g� s;

and this equilibrium is unstable when the reverseinequality holds. Thus in this case, s4g con-tinues to ensure that aaBC is stable, but there isnow for sufficiently large a; this equilibrium isstable even when g4s:zzz

In sum, adding a replicator dynamic changesthe stability properties of the model in only oneimportant way: a sufficiently strong replicatorprocess can render the non-altruistic yetinternalized equilibrium aaBC, rather than the

H. GINTIS414

altruistic equilibrium, aaAC, stable. Realisti-cally, while the replicator process is key tounderstanding social change, in general weexpect this to be a relatively weak force, andcertainly too weak to undermine altruisticnorms, unless they incur substantial fitness costs.Norms do not come labeled ‘‘altruistic norm’’,‘‘instrumental practice’’. Rather, they are inex-tricably intermingled. It is quite common tobelieve that immoral acts lead to disease, forinstance, just as does poor hygiene. Humanpsychology has to be uncritical absorbers ofhosts of beliefs and values, only a small fractionof which can be seriously questioned by anindividual member of society conditions us.

Why is Altruism Predominantly Prosocial?

Internal norms may be either pro- or anti-social. Indeed, there are many accounts of socialnorms that are severely socially costly, such asthose involving invidious displays of physicalprowess (Edgerton, 1992). The reason for thefeasibility of anti-social norms is that once theinternalization gene has evolved to fixation,there is nothing to prevent group-harmfulphenotypic norms, such as our A, from alsoemerging, provided they are not excessivelycostly in comparison with the strength of thereplicator process. The evolution of thesephenotypes directly reduces the overall fitnessof the population.

Yet as Brown (1991) and others have shown,there is a tendency in virtually all successfulsocieties for cultural institutions to promoteprosocial and eschew anti-social norms. Themost reasonable explanation for the predomi-nance of prosocial norms is gene-culture coevolu-

tionary multi-level selection: societies thatpromote prosocial norms have higher survivalrates than societies that do not (Parsons, 1964;Cavalli-Sforza & Feldman, 1981; Boyd &Richerson, 1985; Boyd & Richerson, 1990; Soltiset al., 1995). Note that the usual argumentsagainst the plausibility of genetic group selectiondo not apply to our model. This is becausealtruism is (a) phenotypic, and (b) ‘‘hitchhikes’’on the fitness-enhancing phenotypic norm. Be-cause altruism is phenotypic, and because a highdegree of cultural uniformity can be maintained

within groups, a high ratio of between-group towithin-group variance on the phenotypic trait iseasily maintained, and hence high-payoff groupsquickly outpace low-payoff groups. Becausealtruism hitchhikes, the mechanism that gener-ally undermines group selection, a high rateof inter-group migration (Maynard Smith, 1976;Boorman & Levitt, 1980) does not undermineinternalization, as long as altruistic individualsadopt the A norms of the groups to which theymigrate.

To test this argument, I created an agent-based model of society with the followingcharacteristics (the specific assumptions madeare not critical, unless otherwise noted). Thesociety consists of 256 groups, each initiallycomprising 100 members, arranged spatially on atorus (a 16� 16 grid with the opposite edgesidentified). Each group was seeded with tenaaAC types, ten aaBC types, 74 bbBD types, andone each of the other possible types. In allgroups, t ¼ 0:3 and u ¼ 0:05: Each group wasthen randomly assigned a value of g between 0and 0.60, a value of a between 0 and 0.5, a valueof s between 0.01 and 0.10, and a value of z (thedegree of assortative mating) between 0 and0.70. Each group was also assigned an A

phenotype with a fitness effect between �1 and1, such that if a group’s A-fitness effect is q; andif a fraction f of the group exhibit the A

phenotype, then each member of the group hasits fitness augmented by fq:

In each round, for each of the 256 groups, Isimulated the model as described in the previoussections, and update the frequencies ofthe various types in each group, according tothe fitness effect of their A phenotype andthe fraction of the group that exhibits thisphenotype. A fraction of each group (typically5%) then migrated to a neighboring group. Ifaltruistic migrants adopt the A norm of theirnew groups, migration never undermines thestability of the altruism equilibria. To beconservative, we assume here that agents taketheir genes with them to a new group, and theytake their C phenotype to this new group (sinceC is the same for all groups), but they abandontheir A phenotype when they migrate (e.g. anaaAC type becomes a aaBC type in the newgroup). This assumption is maximally geared to

888This section is an elaboration on Boyd et al. (2001).

HITCHHIKER’S GUIDE TO ALTRUISM 415

undermine the altruism phenotype since immi-grants never exhibit altruism. We then allow forsome random drift in the individual groupsparameters s; a; g; and z; as well as the payoff ofaltruism to the group, q:

Our final modeling assumption is that whengroup size drops below a minimum (generally, Iset this to zero or ten agents), it is replaced by acopy of a randomly chosen other group.

I ran this model many times with varyingnumbers of rounds, and varying the parametersdescribed above. The system always stabilizedby 100 periods, and the specific assumptionsconcerning the parameters were never critical.The following conclusions hold for thesesimulations:

(a) Groups exhibiting the non-internalizationequilibrium bbBD were quickly driven from thepopulation, except under the joint assumptionsthat altruistic agents become non-altruistic whenthey migrate, and the migration rate is over20%.

(b) The equilibrium fraction of groups forwhich the non-internalization equilibrium wasstable was highly variable and dependent uponthe specifics of the initial distribution of groups.Thus in equilibrium, though all groups were nearthe internalization equilibria (aaAC and aaBC),in some this was globally stable and in others,only locally.

(c) The equilibrium fraction of the populationexhibiting the altruistic phenotype was greaterthan 85% (the mean at the start of eachsimulation was 10%).

(d) All but the highest prosocial A phenotypeswere eliminated from the population, so that themean fitness effect of the altruistic phenotypeswas greater than 0.9 (the maximum possible was1.0, and the mean at the start of the simulationwas approximately zero). In particular, nogroups with anti-social norms ever survived inequilibrium.

(e) The mean level of extraparental socializa-tion, g was also very high, being at least 45% (themean at the start of the simulation was 30%).

(f ) The mean strength of the replicatordynamic, a; was 9% (the mean at the start ofthe simulation was 25%). This shows that whilea higher a helps individuals, because they are

then more likely to move to high-fitness pheno-types, it hurts the groups they are in, and onbalance lowers group fitness.

(g) The altruistic equilibrium was attained aslong as the initial average assortative matingprobability z was at least 12.5%. An increase inthe rate of assortative mating led to moreglobally stable altruism equilibria, but had nomeasurable effect on the equilibrium values ofother variables.

(h) The emergence of the aaAC agents and theelimination of anti-social internal norms wereboth due to population growth alone. Theextinction and replacement of groups by moresuccessful groups accounted only for the changein the frequency of g and a (extinctions began tooccur after 20 rounds, and more than 1500extinctions typically occurred in a 100 roundsimulation).

These simulations thus strongly support thebasic arguments of this paper. In particular, ahigh level of migration does not undermine thealtruistic equilibrium, since most of the effectsoccur on the cultural rather than the geneticlevel. Moreover, plausible patterns of populationgrowth and migration account for the prosoci-ality of the altruism phenotype A. The criticalassumption that drives the model is simply thatthere is a fitness-enhancing effect of the selfish Cphenotypic norm sufficiently strong to ensurethat C can invade a population of D agents. Theability of the altruism phenotype A to ‘‘hitch-hike’’ on C is quite robust.

Altruistic Punishment Can Sustain CooperationWhen Altruistic Participation Cannot

Consider the following ‘‘social dilemma’’.888Each member of a group can either cooperate orshirk. Shirking costs nothing, but adds nothing tothe payoffs of the group members. Cooperatingcosts sn40; but contributes an amount f n4sn

shared equally by the other members. Selfishindividuals will always shirk in this situation, sothe potential gains from cooperating will beforgone. If the situation is repeated sufficientlyfrequently with the same players, and if the group

H. GINTIS416

is sufficiently small, cooperation can be sustainedeven with selfish players (Trivers, 1971; Axelrod& Hamilton, 1981). However, with large groupsand/or infrequent repetition, universal shirking isvirtually inevitable (Boyd & Richerson, 1988), ashas been confirmed repeatedly in experimentswith humans (Ledyard, 1995).

Given the potential gains to society of peopleinternalizing the altruistic norm A¼‘‘alwayscooperate’’ in the above situation, the absenceof this norm in society suggests that the cost sn issimply too high to sustain as an equilibrium ofthe aaAC form. However, experimental results(Fehr & Gachter, 2002) and ethnographic data(Boehm 1993, 2000), not to mention everydayobservation, suggest that the threat of beingpunished by other group members for shirkingmay serve to sustain cooperation where theinternalized value of cooperating does not.

Punishment can succeed where the norm ofcooperation cannot because the expected cost perperiod of punishing shirkers is typically muchsmaller that the cost of cooperating. This isbecause (a) punishment such as shunning andostracizing are inherently low cost, yet effectivewhen directed by large numbers against a fewtransgressors; and (b) the punishment need becarried out only when shirking occurs, which islikely to be infrequent in comparison with thenumber of times cooperation must be carried out.

Nevertheless, altruistic punishment likely hasstrictly positive cost, so a selfish individual stillwill refrain from engaging in this activity. Whilea genetic group selection model can explain theevolutionary stability of altruistic punishment(Gintis, 2000b), such models are sensitive togroup size and migration rates (Eshel, 1972;Rogers, 1990). The gene-culture coevolutionarymodel presented in this paper, by contrast,suffers less from these problems.

To see this, suppose a fraction p of a groupwith n members consists of altruistic punishers.To prevent intentional shirking by selfish agents,each must be prepared to inflict a punishmentsn=pn on a shirker. Suppose a fraction q of thegroup nevertheless shirks (or perhaps is simplyperceived to shirk under conditions of imperfectinformation). Then the total amount of punish-ment per altruistic punishment is s ¼ qsn=p:If p is large (as in our simulations) and q is

small (as is likely to be the case except underextreme conditions, since no one has an incentiveto shirk), then this value of s will be close to zerofor each altruistic punisher. But then thealtruism equilibrium will be stable accordingto eqn (2), even when it would be violated fors ¼ sn:

Since the fitness costs of altruistic punishmentare low, a replicator dynamic is unlikely torender the altruism equilibrium unstable in thiscase. Moreover, there is evidence that altruisticacts serve as costly signals of agent fitness (Gintiset al., 2003), in which case the altruisticphenotype is cannot be undermined by thetendency to shift from lower to higher payoffphenotypes.

Conclusion

We have developed a plausible model ofaltruistic cooperation and punishment that doesnot depend on repeated interaction, reputationeffects, or multi-level selection. The latterobtains because there is no net within-grouppenalty to either the altruistic gene or thealtruistic norm, even though there is a penaltyto individuals carrying the gene and behavingaccording to the norm.

One shortcoming of our model is that payoffsare assumed constant, whereas in many cases, wewould expected payoffs to be frequency depen-dent, as when group members are engaged in anon-cooperative game. For instance, the payoffto being self-interested may increase when agentsare predominantly altruistic. In a related paper(Gintis, 2003), I show that such a situation givesrise to a heterogeneous equilibrium, in whichboth altruists and self-interested types partici-pate. Since the payoffs to the two types are equalin equilibrium, once again we can dispense withmulti-level selection in specifying an equilibriumwith a positive level of altruism.

There are two objections that biologistsnaturally raise to this model of altruism. First,if the C norm is individually fitness enhancingwhile the A is not, why is there not a geneticmutation (for instance at another genetic locus)that allows the individual to distinguish betweenaltruistic and fitness-enhancing behaviors, andhence to eschew the former? The answer is that

HITCHHIKER’S GUIDE TO ALTRUISM 417

A- and C- type behaviors are exhibited only onthe phenotypic level, and hence have no clearinherent characteristics according to which sucha gene could discriminate. Moreover, if such aninherent characteristic does exist for a particularA-type norm, that type would be driven toextinction. But there are so many varieties ofcultural norms that others, unaffected by thismutation, would arise to replace the one towhich people have become ‘‘immune’’. Finally,we should note that generally the degradation ofthe genetic capacity to discriminate is muchmore likely than the emergence of such acapacity, since the latter, being complex in thecase of A-type norms, requires the existence of asequence of one-point mutations that are eachfitness-enhancing, finally leading to the capacityto discriminate. This is implausible in the currentcontext.

A second objection to our model of altruismis that we have assumed rather than providedan explanation of why the internalization ofnormsFhaving a programmable objective func-tionFis individually fitness enhancing. Whywould an agent gain from an altered objectivefunction when he always has the option ofobeying a norm when it is his interest to do so,and violating the norm when it is not? However,agents do not maximize fitness, but rather anobjective function that is itself subject toselection. In a constant environment, this objec-tive function will track fitness closely. In achanging environment, natural selection will betoo slow, and the objective function will nottrack fitness closely. Cultural transmission andthe ensuing increase in social complexity pro-duced such a rapidly changing environmentin human groups. Imitation (the replicatordynamic) will not correct this failure, becauseagents copy objective-function-successful, notfitness-successful, strategies. In this situation,there are large fitness payoffs to the developmentof a non-genetic mechanism for altering theagent’s objective function, together with a genetic

mechanism for rendering the individual susceptibleto such alteration. Internalization of norms,which may be an elaboration upon imprintingand imitation mechanisms in non-human ani-mals, doubtless emerged by virtue of its ability toalter the human objective function in a direction

conducive to higher fitness. There is not to myknowledge a confirmed instance of internaliza-tion in nonhuman animals. This may in part bedue to the fact that the relevant research has notbeen carried out. Yet there are obvious reasonsto doubt that internalization might be impor-tant, because cultural transmission in non-human animals is relatively rudimentary.

There is not to my knowledge a confirmedinstance of internalization in non-human ani-mals. This may in part be due to the fact that therelevant research has not been carried out. Yetthere are obvious reasons to doubt that inter-nalization might be important except perhaps inthe case of animals with a long history ofdomestication by humans (dogs come to mind).This is because cultural transmission in animalsis relatively rudimentary. The proximate cause ofcomplex behaviors in animals lies in geneticallyexpressed mechanisms, the ultimate cause beingthe contribution of these mechanisms to thefitness of the individuals who express them. Sinceculture is very important in the fitness ofhumans, internal norms become the proximatecause of complex behaviors in humans, but theultimate cause of the capacity to internalize, andthe content of internal norms themselves, in thefirst instance remains the same: the capacityto enhance the fitness of the individuals whoexpress them. However, we have seen that thereis a second instance in this case: altruistic normscan hitchhike on personally fitness-enhancingnorms. Were this not the case, human society aswe know it would not exist.

It would be a serious mistake to conclude thatthe socialization process in humans is sufficientlypowerful to permit any pattern of norms to bepromulgated by internalization. For instance,many have suggested that it would be betterif people acted on the principle of contributing tosociety according to one’s ability, and takingfrom society according to one’s needs. Whateverthe moral standing of such a principle, no societyhas lasted long when its incentives have beenbased on it. Our model suggests one reason whysuch a principle might fail: the operation of thereplicator dynamic. In this case, the payoffto defectors from the norm is simply too highto prevent its erosion. There may be othercriteria determining what types of altruistic

H. GINTIS418

norms are likely to emerge from the gene-culturecoevolutionary process described in this paper.For instance, behaviors that are altruistic, butvery similar to ones that are personally fitnessenhancing may be relatively easy to internalize;e.g. since it is generally fitness enhancing tospeak truthfully, it may be relatively easy tomove the decision to speak truthfully from therealm of instrumental calculation to that of themoral realm of right and wrong. Similarly,altruistic punishment may be widespread be-cause it is generally prudent to develop areputation for punishing those who hurt us,and it is a short step to turning this prudenceinto a moral principle.

I would like to thank Robert Boyd, MarcusFeldman, Samuel Bowles, Marci Gintis, Eric AldenSmith, John E. Stewart, and Claus Wedekind forhelpful comments, and the John D. and Catherine T.MacArthur Foundation for financial support.

REFERENCES

Axelrod, R. &Hamilton, W. D. (1981). The evolution ofcooperation. Science 211, 1390–1396.

Boehm, C. (1993). Egalitarian behavior and reversedominance hierarchy. Curr. Anthropo. 34, 227–254.

Boehm, C. (2000). Hierarchy in the Forest: The Evolution ofEgalitarian Behavior. Cambridge, MA: Harvard Univer-sity Press.

Boorman, S. A. & Levitt, P. (1980). The Genetics ofAltruism. New York: Academic Press.

Boyd, R. & Richerson, P. J. (1985). Culture and theEvolutionary Process. Chicago: University of ChicagoPress.

Boyd, R. & Richerson, P. J. (1988). The evolution ofcooperation. J. theor. Biol. 132, 337–356.

Boyd, R. & Richerson, P. J. (1990). Group selectionamong alternative evolutionarily stable strategies.J. theor. Biol. 145, 331–342.

Boyd, R. & Richerson, P. J. (1992). Punishment allowsthe evolution of cooperation (or anything else) in sizeablegroups. Ethol. Sociobiol. 113, 171–195.

Boyd, R., Gintis, H., Bowles, S. & Richerson, P. J.(2001). Altruistic punishment in large groups evolves byinterdemic group selection. Working Paper.

Brown, D. E. (1991). Human Universals. New York:McGraw-Hill.

Cavalli-Sforza, L. L. & Feldman, M. W. (1981).Cultural Transmission and Evolution. Princeton, NJ:Princeton University Press.

Edgerton, R. B. (1992). Sick Societies: Challenging theMyth of Primitive Harmony. New York: The Free Press.

Eshel, I. (1972). On the neighbor effect and the evolutionof altruistic traits. Theor. Popul. Biol. 3, 258–277.

Fehr, E., & Gaº chter, S. (2002). Altruistic punishment in

humans. Nature 415, 137–140.Feldman, M. W., Cavalli-Sforza, L. L. & Peck, J. R.

(1985). Gene-culture coevolution: models for the evolu-

tion of altruism with cultural transmission. Proc. Natl

Acad. Sci. 82, 5814–5818.Gintis, H. (1975). Welfare economics and individual

development: a reply to talcott parsons. Quart J. Econ.

89, 291–302.Gintis, H. (2000a). Game Theory Evolving. Princeton, NJ:

Princeton University Press.Gintis, H. (2000b). Strong reciprocity and human

sociality. J. theor. Biol. 206, 169–179.Gintis, H. (2003). Solving the puzzle of human pro-

sociality. Rationality and Society 15, forthcoming.Gintis, H., Smith, E. A. & Bowles, S. (2001). Costly

signaling and cooperation. J. Theor. Biol. 213, 103–119.Grusec, J. E. & Kuczynski, L. (1997). Parenting

and Children’s Internalization of Values: A Handbook

of Contemporary Theory. New York: John Wiley

& Sons.Ledyard, J. O. (1995). Public goods: a survey of

experimental research. In The Handbook of Experimental

Economics, (Kagel, J. H. & Roth, A. E., eds), (pp.

111–194). Princeton, NJ: Princeton University Press.Maynard, S. J. (1976). Group selection. Quar. Rev. Biol.

51, 277–283.Mealey, L. (1995). The sociobiology of sociopathy. Behav.

Brain Sci. 18, 523–541.Nowak, M. A. & Sigmund, K., (1998). Evolution

of indirect reciprocity by image scoring. Nature 393,

573–577.Parsons, T. (1964). Evolutionary universals in society. Am.

Sociol. Rev. 29, 339–357.Parsons, T. (1967). Sociological Theory and Modern

Society, New York: Free Press.Richerson, P. J., Boyd, R. & Bettinger, R. L. (2001).

Was agriculture impossible during the pleistocene butmandatory during the holocene? A climate changehypothesis. American Antiguity 66, 387–411.

Rogers, A. R. (1990). Group selection by selectiveemigration: the effects of migration and kin structure.Am. Nat. 135, 398–413.

Samuelson, L. (1997). Evolutionary Games and EquilibriumSelection, Cambridge, MA: MIT Press.

Simon, H. (1990). A mechanism for social selection andsuccessful altruism. Science 250, 1665–1668.

Sober, E. & Wilson, D. S. (1998). Unto Others:The Evolution and Psychology of Unselfish Behavior,Cambridge, MA: Harvard University Press.

Soltis, J., Boyd, R. & Richerson, P. (1995). Can group-functional behaviors evolve by cultural group selection:an empirical test. Curr. Anthropo. 36, 473–483.

Taylor, P. & Jonker, L. (1978). Evolutionarilystable strategies and game dynamics. Math. Biosci. 40,145–156.

Trivers, R. L. (1971). The evolution of reciprocal altruism.Quar. Rev. Biol. 46, 35–57.

Wrong, D. H. (1961). The oversocialized conception ofman in modern sociology. Am. Sociol. Rev. 26, 183–193.


Recommended