1 Responsibility, Accountability, and Liability: Studies in the Theory of Responsibility for Engineering Ethics and Engineering Accountability “A man’s ethical behavior should be based effectively on sympathy, education and social ties; no religious basis is necessary. Man would be in a poor way if he had to be restrained by fear of punishment and hope of reward after death” -- Albert Einstein (1879-1955) “We live as if the world were as it should be, to show what it can be”
Transcript
1
Responsibility, Accountability, and Liability:
Studies in the Theory of Responsibility for Engineering Ethics
and Engineering Accountability
“A man’s ethical behavior should be based effectively on sympathy, education and social ties; no religious basis is necessary. Man would be in a poor way if he had to be restrained by fear of punishment and hope of reward after death”
-- Albert Einstein (1879-1955)
“We live as if the world were as it should be, to show what it can be”
2
The Importance of Understanding the Concept of Responsibility
• My hypothesis is that technological risks, vulnerabilities and failures often occur because responsibilities are inappropriately assigned. If we can construct some model for the responsibility of actions taken or tasks performed, that, for example may have lead to a technological disaster, we are poised to make better decisions about the ascription of moral responsibility and accountability
• This will reduce vulnerabilities and subsequent failures and disasters
• To usefully reason about responsibilities in a complex socio-technical system, we must have some way of modeling the responsibility itself (in addition to, and distinct from, the important task of modeling the assignment of responsibilities)
3
The Concept of Responsibility
Four-Fold Definition of Responsibility
• Causal Responsibility
• Liability-Responsibility
• Role-Responsibility
• Moral-Responsibility
4
Causal Responsibility
• A purely descriptive sense of responsibility• “The heavy rain is responsible for the flooding”• “The operator was responsible for turning off the
control switch” • The “But-For” conception of being causally
responsible:• X was causally responsible for Y =• But for the occurrence of X, Y would not have
happened• For Example: But for the operator turning the
switch, the control would not have went off”
5
The Concept of Liability
• Liability for one’s actions means that one can rightly be made to pay for the adverse effects of ones actions on others
• Automobile liability insurance is intended to cover the costs of damage to other persons or property
• We are usually liable for such payments as long as we are causally responsible, even if our actions were unintentional
• Liability, does not necessarily involve moral responsibility for the action
6
Strict Liability
• It means that no excusing conditions are applicable or accepted
• Responsibility without fault
• Strict Products Liability
• Part of the debate about legal liability concerns where the line should be drawn when assigning strict liability
7
Strict Products Liability • Charges of strict liability in torts (harms) are generally assigned to
manufacturers for products that are in “ a defective condition or unreasonably dangerous.”
• That liability can be assigned regardless of whether the defendant has been negligent or has been careful (applying accepted standards of care for the product, its design, its manufacture, its assembly and associated warnings).
• In order to prove strict liability, the plaintiff need not prove that the defendant's action fell below society's expectation for reasonable behavior. Instead, the plaintiff must prove that the product per se was in a defective condition unreasonably dangerous. It is certainly true that negligent behavior can result in a product in a defective condition unreasonably dangerous. The plaintiff may, of course pursue both theories of liability at the same time
8
Strict Products Liability
• In order to apply strict liability for products, courts have required the following: 1. The 'product' was in a 'defective condition [resulting in a product
that is ] unreasonably dangerous'. Defects can be created by manufacture, assembly, design, warning labels, marketing, etc.
2. The defendant was in the 'stream of commerce' that produces the product and/or delivers the product to the customer (manufacturer, subcontractor, wholesaler, distributor, retailer, etc.).
3. The product was defective when it left the defendant's hands. 4. The product was intended to reach the plaintiff without
substantial change. 5. The defect caused in fact) physical harm to the plaintiff. (Strict
liability in torts may relieve the plaintiff of responsibility for unforeseeable misuse, abuse, alterations and other defenses
9
Strict Products Liability • The rationale used by courts for imposing strict liability in tort
includes three principles: 1) deterrence, 2) loss spreading, and 3) responsibility
• Deterrence: courts have stated that strict liability in torts encourage manufacturers (and others in the 'stream of commerce') to make products safer. This increased liability may make products more expensive, but courts argue that the increased price more accurately reflects the true social costs of the products.
• Loss spreading: courts have stated that strict liability spreads losses that would be a hardship upon individuals, but the manufacturer (and others in the 'stream of commerce') can offset the increased risk by purchasing insurance. The ethical basis of this principle is utilitarianism
• In addition to deterrence and loss spreading, courts have also argued that applying strict liability places responsibility (liability) on the same entities and individuals that control the design, specifications, manufacturing tolerances, material specifications, and condition of the final product as it is delivered to the ultimate customer.
10
Role Responsibility
• Role-Responsibility: “Whenever a person occupies a distinctive place or office in a Social organization, to which specific duties are attached…he or she is properly said to be responsible for the performance of these duties, or for doing what is necessary to fulfill them.
• Such duties are a person’s (role) responsibilities.”
11
The Concept of Role Responsibility
“Whenever a person occupies a distinctive place or office in a Social organization, to which specific duties are attached…he or she is properly said to be responsible for the performance of these duties, or for doing what is necessary to fulfill them. Such duties are a person’s (role) responsibilities.”
•The term "role” includes tasks assigned to people by agreement or otherwise.
•The term role-responsibility generally refers to a situation where a certain person occupies a distinct place or office in a social organization, and particular duties are attached to this role in order to provide for the welfare of others or to advance in some specific way the objectives or functions of the concerned organization.
•It is necessary to differentiate
• “Internal” role responsibility – responsibility for the role one plays as a member of an organization or profession•“External” role responsibility – responsibility for the role one plays in the larger society and culture
12
Role Responsibilities in Professional Engineering
• Role responsibilities are often recognized as professional responsibilities, and one of the key issues of engineering ethics is to formulate the relevant sets of responsibilities that can be attached to the roles of the members of the engineering community
• What are the duties and obligations that are attached to an individual or group of engineers with respect to their role in a professional organization, corporation, or society?
• How best can engineers fulfill their role responsibilities, duties and obligations?
13
Role Mapping• Without a way to effectively connect the various responsibilities that
people in organizations have with their roles in the organization, accountability may not be able to be established and this can allow people to avoid responsibility for their decisions and/or their actions
• Role Mapping techniques are essential in order to ensure appropriate matching of roles and responsibilities across the organization.
• Role Mapping-who does what in terms of roles; what is each person’s commitment/promise of performance and how does it contribute to overall organizational results.
• This includes:
– Clarify organizational goals and objectives– Identify every employee’s personal accountability for both results and
values– Measure performance of both the organization and every employee
14
Role Mapping
• Clarifies unexpected complexity, problem areas, redundancy, unnecessary loops, and tasks where simplification and standardization may be possible;
• Helps identifies roles and responsibilities, thus supporting more effective allocation of staff resources and more effective stakeholder partnerships
15
Moral Responsibility• To say a person is responsible in this sense is to say that the person is
deserving of blame. • This sense of "responsible" seems to imply fault. • That is, when we say people are responsible in this sense we are
evaluating their behavior relative to some principle or standard. • Those responsible in this evaluative sense may also be responsible in
one of the other senses of the term • an assessment of responsibility in one of the first three senses is often the basis for
attributing responsibility in this fourth sense
• Moral Responsibility: Accountability for the actions one performs and the consequences they bring about, for which a moral agent could be justly punished or rewarded. It is commonly held to require the agent's freedom to have done otherwise (autonomy).
• Moral responsibility is a normative notion—it involves an evaluation• Connected to other concepts such as duty, obligation, knowledge,
freedom, choice, accountability, agency, praise, blame, intention, pride, guilt, shame, conscience, and character
16
Two Types of Moral Responsibility
• The assignment of moral responsibility based on the attribution of accountability to a moral agent, where the moral agent acted freely and possessed the capacity for rational choice and the agent has acted voluntarily
• Moral Responsibility in the second sense reflects a positive judgment about the manner in which the moral agent has deliberated and the particular way they choose to act
17
Types of Moral Responsibility Attribution
• Depending on the kind of responsibility, there are different mechanisms for attributing responsibility
• Responsibility can be attributed:– Ex Ante (Before something happens) as in “I take full
responsibility that nothing will go wrong”– Ex Post (After something happens) as in “I take full
responsibility for everything that went wrong”
• Assignment of responsibility is not an all or nothing affair – individuals can be assigned various degrees of responsibility based on a variety of influencing factors
18
To hold someone morally responsible for their actions or omissions,At least five conditions need to be met:
(1) That the subject had some role to play in the particular chain of events(2) That the person was competent to understand their role in the chain of
events, and that their competency is relevant to the issue at hand (3) That the person act voluntarily, and if not, what precluded or diminished
their capacity to act voluntarily?(4) That the person was able to influence the chain of events, and if not, what
precluded or diminished their capacity to influence the chain of events?(5) That the person was aware of the effects of their actions and knew about the
results and their own power of influence or lack of power(6) Related concepts: Rationality, Freedom, Intentionality, Autonomy
Ascriptions of Individual Moral Responsibility
19
A Reasonable Care Model of Professional Responsibility
– (1) As a member of a profession taking on a specific role in a large organization (corporation, government), E has a duty to conform to the standard operating procedures of his or her profession as well as fulfilling all of the responsibilities which are attached to that particular role within the organization.
– At time t, decision or action (X) conforms to the standard of reasonable care and of role responsibility as defined in (1)
– E omits to execute decision or action (X) at time t (culpable ignorance may be relevant here)
– Harm (H) is caused to some person or group of persons (P) as a result of E’s failure (f) to decide or do X (HP = EfX)
20
Moral Responsibility and Role Responsibility
• Questions of accountability are often raised when an individual or group is thought to be responsible for a failed technology.
• For example, the breaking of a dam may be the result of such factors as honest mistakes in statics or dynamics analyses; careless, negligent, or even criminal misconduct; incompetence; and the use of substandard materials
21
Moral Responsibility and Free Will
• Instances of coercion and constraint may exempt agents from judgments of moral responsibility
• Coercion and constraint mean the imposition of some external force that compels or precludes a particular choice or a particular action itself
• Consideration of the form and degree of external force imposed can affect the extent to which one considers an action to have been less than voluntary or non-voluntary
• Principle: the greater the threat imposed by some external source, the more it eliminates freedom of choice
• The more freedom of choice is eliminated, the less voluntary actions become
• Some threats reduce the voluntariness of an actions by making any other choice extremely difficulty for an individual to make in the face of the relevant threat
• The greater the coercion or constraint, the less likely we will consider the action voluntary and the less moral responsibility we will assign to the agent
• One often can be “excused” from being held responsible for an action if the moral agent was coerced or forced to perform the action contrary or against the agent’s free will
22
Comparing Liability and Moral Responsibility
Liability
Particular: Derives from legislation in force in a certain time and place
Limited: Applies only to specificPeople at specific times or places
Divisible: It can be delegatedor distributed
It can be waived: Sometimes not applicable, implemented or enforced
Punishable
Moral Responsibility
Universal: Ethical principles aspire to universality in that they are not limited to particular people or particular groups or societies
Unlimited: It applies to any person in the same situation
Indivisible: It cannot be delegated nor distributed
It cannot be waived: it always applies
Not based on punishment except social shame or guilty conscience
23
Legal Liability vs. Moral Responsibility
• The essential characteristics of liability responsibility demonstrate its limitations as a legitimate response in many areas of engineering, technology, and science
• Responsibility and blameworthiness are only a part of what is covered when we apply the robust and intuitive notion of accountability
• When we say someone is accountable for a harm, we may also mean that he or she is liable to punishment (e.g., must pay a fine, be censured by a professional organization, go to jail), or is liable to compensate a victim (usually by paying damages).
• In most actual cases these different strands of responsibility, censure, and compensation converge because those who are to blame for harms are usually those who must “pay” in some way or other for them.
25
3 Motivations for Accountability
• Accountability as a virtue that is desirable in its own right
• Accountability as a guideline for answerability which motivates precautionary behavior that, in turn, caters to social welfare
• Accountability as a tracing too that allows us, a posteriori, to identify the people involved in accidents and damage-inducing errors, punish the responsible if necessary and compensate the victims if possible
26
individuality, personhood
conceptual foundations of accountability
responsibility, fault, guilt
accountability
theo
ry o
f
caus
atio
n
27
A Typology of Moral Accountability
Human Actions/Behavior
Recklessness
Negligence
Incompetence
Supererogatory
Dutiful
Due Diligence
Malice
Competence
Blameworthy
Praiseworthy
28
A Typology of Moral Accountability
• Malice: to set out on a course of action with the deliberate aim of imposing harm or risks to people
• Recklessness: to act knowing that it will cause harm or risk, but not taking this properly into account
• Negligence: the failure to exercise in the given circumstances that degree of care for the safety of others which a reasonable person would exercise under the same or similar circumstances
• Incompetence: not qualified or suited for a purpose; showing lack of skill or aptitude; "a bungling workman"; "did a clumsy job"; "his fumbling attempt to put up a shelf"
• Competence: qualified or suited for a purpose; showing appropriate skill or aptitude• Due Diligence: the exercise in the given circumstances that degree of care for the
safety of others which a reasonable person would exercise under the same or similar circumstances
• Dutiful: to know what the right thing to do is and to do it regardless of how it effects you
• Supererogatory behavior: going above and beyond the call of duty
29
A Typology of Moral Accountability
• What is the difference between ignorance and incompetence?
• Ignorance is when you do something wrong because you do not know any better
• Incompetence is when you do something wrong even though you do or should know better
30
Responsibility vs. AccountabilityAccountability
• Implies imminence of retribution for unfulfilled trust or violated obligations
• The focus is more upon what others expect from the person who is accountable
• “Other-Centered”
• Includes judgment and the extent of judgment for the success or failure to do, complete, or protect that for which a person is held accountable
• Accountability always assumes a prior responsibility for we always lay out what we expect before we can lay out what the consequences will be for failure to meet the expectations
Responsibility• Implies holding a specific office,
duty, or trust
• The focus is on what can and should do; an individual’s personal integrity with respect to a specific task
• “I-Centered”
• One has a clear duty to perform an action and take care to carry it out or bring something to fruition
• While being responsible always has other persons in mind, the focus of meaning is upon the individual’s effort, duty, and obligation
31
Responsibility vs. Accountability
Accountability• Liable to be called to account; answerable• Refers to how the individual will be judged
and thus either rewarded or punished• A person is accountable only when we
know they have to answer to being punished
• If someone is accountable, it is assumed a responsible party be able to meet the demands of the higher authority to whom they will give their accounting
• Accountability focuses for the most part upon all of the elements of duty after the decision is made
• When accountable one is duty bound externally or one imposes a much stronger duty upon themselves to answer to any actions which may cause harm or damage to those they are accountable for
Responsibility• We call someone responsible when
we judge the person’s motives, intentions, and carefulness with respect to the task
• We can be responsible without being held accountable to anyone in particular
• Responsibility focuses for the most part upon all the elements of duty up to the point of decision
• The major difference is the certainty or strength of implied/suggested duty
• When responsible one may be asked or take it upon themselves to be morally responsible for the actions they take, for themselves, or others
32
Responsibility vs. Accountability
Accountability• Accountability: "I’ll pay a price if I
don’t do it right."
• Required to explain or justify all of the reasons for one’s actions
• Accepting personal liability for one’s actions, accepting one’s actions and the consequences
• When we know that we must answer with respect to how well we accomplished the task and what reward or punishment was meted out for failing at the task
Responsibility• Responsibility: "I’ll do it.“
• A sense of obligation, commitment, etc.
• Includes exercising one’s judgments with regard to the powers and authority of discretion one has
33
The Social Nature of Responsibility • Moral responsibility is assigned with the understanding that the
moral agent who has voluntarily chosen and acted is the product of numerous social institutions (family, community, professional society, etc.) and all the subsequent societal influences that mold and individual into what they will become
• The assignment of moral responsibility can be understood as a social practice that serves the crucial function of calling the agent’s attention to her or his effects on the world as well as the individual’s relationships and obligations to other persons in the world
• The assignment of responsibility is related to the development of an attitude of care and concern for one’s effects, relationships and duties
• The assignment of responsibility and the processes of being held accountable for your (voluntary) actions is part of a ingenious practice of social control by which the community furthers its common ends and interests (Smiley, 1992: 238-254)
• Smiley, Marion (1992) Moral Responsibility and the Boundaries of Community Chicago: University of Chicago Press
34
The Social Nature of Responsibility
• Moral responsibility is the basis for praise or blame, reward or punishment, fame or infamy
• These mechanisms are essential ways in which communities may effect personal change in their members toward behavior that is more in line with collective (social, cultural) ends and values
• Example: Judgments of praise and blame, when internalized, create social emotions such as guilt, shame, regret, remorse, pride, etc—in our response to how we interacted with and treated others—that contribute to the development of conscience (Gaylin and Jennings, 1996: 137-49)
• Praise and Blame form part of the organization of social adaptation which operates through the assignment of responsibility and of holding people accountable for their actions and attitudes
• Moral Responsibility becomes an aspect of our “social practice of blaming” (Smiley: 252)
35
The Social Nature of Responsibility• This works in the pluralistic liberal democracy of the USA
by a tacit agreement between members of this large community (country)
• It is “agreed” that individuals are free to choose a way of life free of coercion or constraints provided that individuals realize that these free choices are subject to judgment and criticism by others in the community if an individual is judged to have crossed the line
• Principle of Liberalism – I am free to do whatever I please as long as in pursuing my ends I do not inhibit another person’s right to do whatever they please
• Moral Responsibility in this first sense is mainly an assignment of accountability by the communal “will” (an external judgment) which, in turn, reserves the right to constrain another’s actions so that they are in accord with the values of the community
36
Moral Responsibility as a Virtue
• Moral responsibility in the second sense – it is a virtue• Moral responsibility as a virtue requires
– the acceptance and internalization of moral accountability (responsibility in the first sense)
– with the addition of care and concern for oneself and for other people
• The disposition to deliberate, decide, and then take action in ways that one’s respected community can judge to be morally worthy of rightness and praise – Acting in this way one really embodies the virtue of moral responsibility
• The Virtue of Moral Responsibility • A cognitive element – the process of rational deliberation about
what to do in connection with all the relationships and obligations which arise is a social network
• An affective (emotional) element – expressed in the genuine care and concern for how an individual responds to their world, in their thoughts and actions and their effects on others, as well as towards the community as a whole
37
Moral Responsibility as a Virtue• The Virtue of Moral Responsibility
– Starts to develop once a person has internalized the acceptance to be held accountable for his or her own free and autonomous choices and actions
– One then develops a genuine concern about the consequences of one’s actions and how one’s actions will or will not measure up to the societal norms tacitly agreed upon by the individual when they entered the community they belong to
– A person could, for instance, take complete responsibility for their actions but yet not care in what manner their actions impacted other people nor the social relationships and bonds they form with them
– Moral Responsibility as a Virtue includes the element of altruism or the genuine care and concern for the well-being of others and a strong commitment to deliberate and make moral choices consistent with the social ethic, acting only on the internalized norms of the moral community in which one lives and partakes (Nussbaum, 19XX; Card, 1996). Claudia Card
– Social Responsibility is also considered a Virtue by some researchers (Etzioni, 1993: 11; May 1992)
38
CULPABLE IGNORANCE
• Culpable ignorance – when one fails to know something that they should have known
• Culpable ignorance – an individual rejects or avoids knowledge they should be aware of. This can result from laziness, incompetence, or intention
• Culpable Ignorance can be either direct or indirect– Direct voluntary ignorance is when one decides to not
know – it is done deliberately – Indirect voluntary ignorance is when one could/should
have known but remained in ignorance – it was done without due diligence
• Due diligence – taking care to make sure you learn something that you should know
39
CULPABLE IGNORANCE
• Culpable ignorance is a case where ignorance of the facts surrounding a situation does not diminish the responsibility of the moral agent for unwanted or immoral outcomes of an action.
• This is usually because some degree of due diligence or reasonable care has not been taken by the agent in question.
• Due diligence means that the agent in question failed to do know something that they could be reasonably expected to know and this led to the performance of the immoral act. – For example, a doctor kills a patient by administering penicillin to
a patient that is allergic. The doctor was unaware of the allergy because they had failed to investigate the patient’s history.
• Epistemic responsibility • Epistemic conditions on moral responsibility
40
CULPABLE IGNORANCE• Culpable ignorance is a case where ignorance of the
facts surrounding a situation does not diminish the responsibility of the moral agent for unwanted or immoral outcomes of an action.
• Even though the agent acted in good faith at the time, we say that ‘they should have known better’ or ‘they should have realised what they were doing’ and so they are still blameworthy for the immoral outcomes of their action, even though these outcomes were not intended.
• It is culpable ignorance because it could be cleared up if the person used sufficient diligence.
• You were capable of knowing something, and you should have taken pains to come to know it.
• One is said to be culpably ignorant if one fails to make enough effort to learn what should be known; guilt then depends on one's lack of effort to clear up the ignorance
41
Culpable Ignorance • What is the difference between culpable and non-
culpable ignorance?• The criterion for determining culpable ignorance, is if
harm is likely to result and the agent could have found out about the likely circumstances of the action
• We should be expected to know in general what kinds of effects will result from familiar types of actions, even if we can’t predict the exact details
• For example, there is an historical record of human-made disasters, and the causes of them can be determined and understood by identifying general categories of belief and action, as well as design and technical breakdown of engineered systems
• The SHOT model is an example of this
42
Culpable Ignorance
• Some things are unpredictable in detail, but are familiar enough that one would be culpable not to expect them if they fit into our categorical scheme – SHOT
• Those who perform actions that have potentially disastrous consequences can be morally culpable even if they cannot foresee the specific consequences
• They are culpable because experience has shown that one should expect certain kinds of events
• In general, we have an ethical duty to find out what the likely effects of our actions are
43
Culpable Ignorance
• In considering culpable ignorance, typically one is concerned with ignorance of fact.
• But there is also another type of culpable ignorance called ignorance of moral principle. One can fail to know what one ought to do in a particular case. – One can fail to know some general moral rule. – One can fail to know that people have certain rights;
or that one has certain responsibilities
• An omission may be culpable on account of some special position of role or other responsibility held by the agent
44
Moral Accountability and “Excusing Conditions” When someone is the “cause” of some wrongdoing, they are not
automatically considered responsible and hence accountable. The law and ethics recognizes certain, valid “excusing conditions”
Ignorance Excuse • Is it possible to know? • Could we, or should we have known?
– Would a reasonable person considered the possibility? – If not: excusable ignorance – If impossible for us to know: invincible ignorance
Lack of Freedom Excuse • Four conditions:
– No alternatives: not even lack of action – Lack of control:– External coercion: force – Internal coercion: Illness, passion, uncontrollable psychological
compulsion, etc.
45
Theory of Negligence
• Negligence has come to define the expected standard of conduct replacing, for some people, ideas of honor, propriety, and simple right and wrong
• No case of actionable negligence will arise unless the duty to be careful exists
• A person is considered negligent or careless if they do not exercise the kind of due care that is appropriate to the particular situation in question
• Negligent omission: failing to act when the person has a duty to act
46
Negligence• The law of negligence imposes a duty to think before you act. • The ordinary care standard imposes a social standard which is judged by
members of the community who may or may not agree with your evaluation of your own conduct.
• Therefore, it is important to look at your acts and omissions from the stand point of others in the community who will be judging your conduct.
• If you have negligence concerns, ask: • 1. What would members of the community require me to do under these
circumstances; • 2. What would members of the community forbid me to do under these
circumstances; • 3. What would members of my profession/vocation/calling require of me
under these circumstances;• 4. What would members of my profession/vocation/calling counsel me to
avoid under these circumstances; • 5. What are the risks of my conduct, considering the probability of harm and
the degree of injury or damage that would result if an accident occurred; and • 6. Would ordinary people in the community believe that I am taking
reasonable risks?
47
Proving Negligence
• Negligence is 'conduct which falls below the standard established by law for the protection of others against unreasonable risk of harm' [4].
• In order to establish liability for damage, the courts analyze the following four elements:
• duty • breach • proximate cause • damages.
48
Proving Negligence
Negligence: the injured party (plaintiff) must prove: • a) that the party alleged to be negligent had a duty to the
injured party-specifically to the one injured or to the general public,
• b) that the defendant's action (or failure to act) was negligent-not what a reasonably prudent person would have done because it did not fulfill the “standard of care” typical of how any similar engineer would judge and act in similar situations
• c) that the damages were caused ("proximately caused") by the negligence.
• d) That the damages were "reasonably foreseeable" at the time of the alleged negligence.
49
Standard of Care
• In legal cases, a judge or jury, has to determine what the standard of care is and whether an engineer has failed to achieve that level of performance.
• They do so by hearing expert testimony. • People who are qualified as experts express
opinions as to the standard of care and as to the defendant engineer's performance relative to that standard.
• The testimony from all sides is weighted and then a decision is made what the standard of care was and whether the defendant met it
50
Standard of Care• Jury instructions have been standardized. A Bench
Approved Jury Instruction (BAJI, 1986) reads:• "In performing professional services for a client, a
(structural engineer) has the duty to have that degree of learning and skill ordinarily possessed by reputable (structural engineers), practicing in the same or similar locality and under similar circumstances.
• It is (the structural engineer's) further duty to use the care and skill ordinarily used in like cases by reputable members of the (structural engineering) profession practicing in the same or similar locality under similar circumstances, and to use reasonable diligence and (the structural engineer's) best judgment in the exercise of professional skill and in the application of learning, in an effort to accomplish the purpose for which (the structural engineer) was employed.
• A failure to fulfill any such duty is negligence"
51
Standard of CareThree key items in this instruction bear repeating:
1. ...have learning and skill ordinarily possessed by reputable engineers practicing in the same or similar locality and under similar circumstances.
2. ...use care and skill ordinarily possessed by reputable engineers practicing in the same or similar locality and under similar circumstances.
3. ...use reasonable diligence and best judgment to accomplish the purpose for which the engineer was employed.
• If any one of these conditions is not met, the engineer has failed to meet the standard of care, and is professionally negligent.
Joint Tortfeasors (wrongdoers): two or more persons whose negligence in a single accident or event causes damages to another person.
• In many cases the joint tortfeasors are jointly and severally liable for the damages, meaning that any of them can be responsible to pay the entire amount, no matter how unequal the negligence of each party was.
• Example: Harry Hotrod is doing 90 miles an hour along a two-lane road in the early evening,
• Adele Aimster has stopped her car to study a map with her car sticking out into the lane by six inches.
• Hotrod swings out a couple of feet to miss Aimster's vehicle, never touches the brake, and hits Victor Victim, driving from the other direction, killing him.
• While Hotrod is grossly negligent for the high speed and failure to slow down, Aimster is also negligent for her car's slight intrusion into the lane. As a joint tortfeasor she may have to pay all the damages, particularly if Hotrod has no money or insurance.
• However, comparative negligence rules by statute or case law in most jurisdictions will apportion the liability by percentages of negligence among the tortfeasors and the injured parties.
53
Res Ipsa Loquitur (The Thing Speaks for Itself)
• (rayz ip-sah loh-quit-her) n. Latin for "the thing speaks for itself," • A doctrine of law that one is presumed to be negligent if he/she had
exclusive control of whatever caused the injury even though there is no specific evidence of an act of negligence, and without negligence the accident would not have happened.
• Examples: a) a load of bricks on the roof of a building being constructed by High-rise Construction Co. falls and injures Paul Pedestrian below
• High-rise is liable for Pedestrian's injury even though no one saw the load fall.
• b) While under anesthetic, Isabel Patient's nerve in her arm is damaged although it was not part of the surgical procedure, and she is unaware of which of a dozen medical people in the room caused the damage.
• Under res ipsa loquitur all those connected with the operation are liable for negligence.
• Lawyers often shorten the doctrine to "res ips," and find it a handy shorthand for a complex doctrine.
54
Negligence Per Se
• Negligence due to the violation of a public duty, such as high speed driving.
• In Black’s Law Dictionary negligence ’per se’ is defined as: ’Conduct, whether of action or omission, which may be declared and treated as negligence without any argument or proof as to the particular surrounding circumstances, either because it is in violation of a statute or valid municipal ordinance, or because it is so palpably opposed to the dictates of common prudence that it can be said without hesitation or doubt that no careful person would have been guilty of it.
• As a general rule, the violation of a public duty, enjoined by law for the protection of person or property, so constitutes."
55
Recklessness
• Recklessness: An injury caused by conduct that is more than mere carelessness but less than actual intent to cause harm
• Recklessness: Carelessness in reckless disregard for the safety of the lives of others. It is more than simple inadvertence but it is less than being consciously intent on causing harm
• Gross negligence is another way of saying recklessness• Culpable negligence: a degree of carelessness greater
than simple negligence. It is a negligent act or omission accompanied by a culpable disregard for the foreseeable consequences of that act or omission
56
Intention
• Intend: To fix the mind upon (something to be accomplished); to be intent upon; to mean; to design; to plan; to purpose
• Intend: have in mind as a purpose; to design for a specific purpose.
• Intend: to act with purpose; mean; design; plan; conceive; contemplate.
• Intentionality: expressive of intentions
57
Barriers to Responsibility and Accountability
1. The Social Psychology of Identification of One’s Role in Social Interaction (The Zimbardo Experiment)
2. Obedience to Authority in Social Contexts (The Milgram Experiment)
3. The Problem of Many Hands
4. Diffusion of Responsibility
5. Risky Shift Phenomena
58
Introduction: Zimbardo Experiment• Why do human beings, even seemingly “normal” people, sometimes
commit despicable acts? • One answer points to individual dispositions; another answer
emphasizes situational pressures. • For example, In 2005, Secretary of State Condoleezza Rice
stressed the importance of individual dispositions in describing terrorists as "simply evil people who want to kill."
• The Theory of Situational Context (TSC) rejects this view. • It hypothesizes that horrible acts can be committed by perfectly
normal people.• The TSC view has received strong support from some of the most
famous experiments in social science, conducted by the psychologist Stanley Milgram in the early 1960s. (See below: The Milgram Experiment slides)
• The TSC view has also received strong support from another famous experiment in social psychology: The “Zimbardo Experiment”
59
Role Responsibility and The Zimbardo Experiment
•To study the roles people play in prison situations, Zimbardo converted a basement of the Stanford University psychology building into a mock prison. •They advertised for students to play the roles of prisoners and guards for a two-week period. •Zimbardo selected the 21 applicants who seemed the healthiest, maturest and most 'normal'. At random 11 were assigned the role of 'guards', 10 the role of 'prisoners'. •The guards were given an official-looking uniform; the prisoners something like a prison uniform and toothbrush, towels and bed linen. No personal belongings were allowed in the cells. •Zimbardo and the guards worked out a set of rules which prisoners were expected to memorize and follow. •Prisoners were required to work to earn their $15 per day and were allowed prisoners twice per week•Guards were allowed to give certain rewards for good behavior.
60
•On the first day, the 'count' of the prisoners (carried out three times per day) took ten minutes. •By the second day, the 'count' time had increased as the guards started to use it to harass the prisoners and by the fifth day the 'count' occupied several hours as the guards berated the prisoners for minor infractions of the rules. •The “prisoners” carried out a real insurrection, which was put down quickly by the “guards.”•The guards then proceeded to punish the prisoners for their disobedience and protest•Instead of protesting, some of the prisoners began to act in depressed, dependent ways, just like many real prisoners and inmates of institutions. •They deteriorated into learned helplessness, becoming ever more subdued and depressed, and acting “zombie-like”•The more they acted in that way, the more they were mistreated. •The behavior of the guards was one of growing cruelty, aggression and dehumanization•They stripped the “prisoners” hooded them, chained them, denied them food or bedding privileges, put them into solitary confinement, and made them clean toilet bowels with their bare hands
Role Responsibility and The Zimbardo Experiment
61
Zimbardo Experiment• By the end of the sixth day, the situation had deteriorated to such an
extent, with guards inventing new rules to make the prison regime more punitive, that Zimbardo called a halt to the experiment.
• Zimbardo said in his book that the mock prison had to be shut down because "the ugliest, most base, pathological side of human nature surfaced.“
• The important question for ethics becomes: What caused it to surface?
• Was it simply deep down inside of each individual?• Or, did the particular situation that they were put into “cause” them
to act like they did? • The analysis of the results showed that the subjects simply 'became'
the roles they played. More than a third of the guards behaved in such a hostile manner consistently, that Zimbardo described their behavior as sadistic.
• This was despite the fact that the roles were assigned at random and there was absolutely no prior evidence that any of the subjects was inclined to behave as they did.
62
Zimbardo Experiment• In his book The Lucifer Effect: Understanding How Good People
Turn Evil, Zimbardo explains the full meaning of Stanford Prison Experiment.
• Generalizing from original results of the experiment, he suggests that dispositionism (i.e., that the propensity to do good or evil resides in our personal dispositions or characters or temperament) is a serious error, that good and evil are largely a function of our contexts and our roles, and that almost all of us are capable of real evil, given the proper situation. The theory is called situationism.
• Zimbardo uses his experiment to cast light on diverse problems, including;
– the conduct of American soldiers at Abu Ghraib,– airplane accidents, – human inaction in the face of evident cruelty, – the mistreatment of patients in hospitals, and – the behavior of suicide bombers and terrorists in general
• Watch a documentary of the experiment, explained by Dr. ZImbardo at: www.prisonexp.org
63
Obedience to Authority A Barrier to Responsibility
Milgram experiment (late 1960s Yale University)
•In the experiment ordinary men and women were brought in to participate in what they were told was a study of memory.• When they arrived at the laboratory they were told that they were to play the role of teacher. •They had to read a series of word pairs to another person on the other side of a partition. •In the experiment, so-called "teachers" (who were actually theunknowing subjects of the experiment) were recruited by Milgram. •They were asked administer an electric shock of increasing intensity to a"learner" for each mistake he made during the experiment. •The fictitious story given to these "teachers" was that the experiment was exploring effects of punishment (for incorrect responses) on learning behavior. •The "teacher" was not aware that the "learner" in the study was actually a compatriot of Milgram’s - - merely feigning discomfort as the "teacher" increased the electric shocks.
64
Milgram experiment• When the "teacher" asked whether increased shocks
should be given he/she was verbally encouraged to continue.
• Sixty percent of the "teachers" obeyed orders to punish the learner to the very end of the 450-volt scale!
• No subject stopped before reaching 300 volts! • At times, the worried "teachers" questioned the
experimenter, asking who was responsible for any harmful effects resulting
from shocking the earner at such a high level. • Upon receiving the answer that the experimenter
assumed full responsibility, teachers seemed to accept the response and continue shocking, even though some were obviously extremely uncomfortable in doing so.
• The study raised many questions about how the subjects could bring themselves to administer such heavy shocks.
65
Milgram experiment• The apparent shocks were delivered by a simulated shock
generator, offering thirty clearly delineated voltage levels, ranging from 15 to 450 volts, accompanied by verbal descriptions ranging from "Slight Shock" to "XXX."
• As the experiment unfolded, the subject was asked to administer increasingly severe shocks for incorrect answers, well past the "Danger, Severe Shock" level, which began at 375 volts.
• The mechanism for administering the shocks had 30 levels or settings raging from 15 to 450 volts, so that the maximum number of shocks that could be given was 30. Milgram devised a set of four “prods” that the experimenter gave to subjects who asked whether they should continue to administer shocks (Milgram, 1974:21): 1. “please continue,”2. “the experiment requires you to continue,” 3. “it is absolutely essential that you continue,” and4. “you have no other choice, you must go on.”
• These prods were made in sequence and if the subject refused to obey after prod 4, the experiment was terminated.
66
67
Milgram experiment
• The expected break-off point is the "Very Strong Shock" of 195 volts. In Milgram's experiment, however, every one of the forty subjects went beyond 300 volts.
• A large majority--twenty-six of the forty subjects, or 65 percent--went to the full 450-volt shock, five steps beyond "Danger, Severe Shock."
• Replications of Milgram's experiments, with thousands of diverse people in numerous countries, show essentially the same behavior.
• And women do not behave differently from men.• Milgram concluded that ordinary people will follow orders
even if the result is to produce great suffering in innocent others.
68
The Surveillance Effect
•There is clear evidence from Milgram's study that the presence of the experimenter helped to increase obedience. When he left the room, obedience dropped from 65% to 21%. The same thing happens in classrooms, offices and factory floors as well.
The Buffer Effect
•The buffer in the Milgram experiment was the wall between teacher and learner. Milgram showed that if the teacher was personally required to hold the learners hand on the shock plate, then obedience dropped from 65% to 40%. •It seemed that the more direct the interaction between the teacher and the learner, the lower the obedience would be. •Milgram tested this theory in reverse by conducting an experiment where the teacher was required to pull a lever which would cause another person to administer the shocks. •In this case the obedience level went up from 65% to 93%.
Obedience to Authority Studies
Milgram experiment
69
Milgram and Zimbardo Experiments and Ethics
• Both the Zimbardo and Milgram experiments shed light on the situational affects on a human psyche
• It sheds light on the philosophical debate over the nature of responsibility and accountability:
• Are only individuals totally responsible for their actions or could their environment or situation be implicated in causing their behavior?
• "How do average even admirable people become dehumanized by the critical circumstances pressing in on them?" asked the famous philosopher Hannah Arendt in her book about the Nuremburg Trials, The Banality of Evil. This is what she called the phenomena because the German officers who committed atrocious acts were no more “evil” than any other person in their inner character
• What, is blind obedience?
70
Milgram and Zimbardo Experiments and Ethics
• . In his book, Obedience to Authority, Milgram concludes that "A substantial proportion of people do what they are told to do, irrespective of the content of the act and without limitations of conscience, so long as they perceive that the command comes from a legitimate authority."
• But, what constitutes “legitimate authority” is the crucial question.
• "What encourages obedience? – Is it fear of punishment or negative repercussions? – A desire to please?– A need to go along with the group? – A blind faith in authority?"
71
Situationism and Professional Contexts
• In a typical workplace, employers are the authority figures and employees are subordinates to them
• This relationship of superior to subordinate can lead to abuses.
• For one thing the employer holds your job, your paycheck, and your “livelihood” over your head, so to speak
• If they threaten to deprive you of any or all of these if you fail to do as you are told, what should you do?
• Can you be excused from being responsible or accountable for your actions because your boss “coerced” you to do things you thought were wrong?
72
The “Problem of Many Hands”A Barrier to Accountability
• Because so many people contribute in so many different ways, it is very difficult to determine who is accountable for organizational behavior.
• It can often be extremely difficult to determine an individual's contribution to failures in large organizations or large engineering projects where many people participate and add their particular skills or expertise (in fact, the same goes for “successes”).
• The case studies you will analyze in this course begin to suggest some of the ethical implications that ensue from the diffusion of responsibility in engineering ethics contexts, particularly in the design and operation of complex technological artifacts and systems.
73
The “Problem of Many Hands”A Barrier to Accountability
• One philosopher notes that "With respect to complex organizations, the problem of many hands often turns the quest for responsibility into a quest for the Holy Grail“
• Bovins, B. (1998). The quest for Responsibility: Accountability and Citizenship in Complex Organizations. Cambridge: Cambridge University Press.
• In many cases we simply cannot isolate individual contributions to organizational action. This suggests not only that we lack some of the basic incentives that could be used to increase individual effort in pursuit of quality, but that the ability to achieve justice in organizations is compromised.
• Research on decision making shows that some layers of the organizational hierarchy are responsible for decisions that are more visible, concrete, limited in time, and identifiable with specific individuals than are others
74
The “Problem of Many Hands”
• It is quite natural to assume that when mistakes are committed, we can associate it with the particular decision behind it
• If this decision leads to adverse consequences, it is assumed that the decision maker is at fault
• This is an unwarranted assumption, as legal scholars well know. • People at the top and bottom of organizations tend not to be blamed when accidents
happen– Braithewaite, J. (1998) “The allocation of responsibility for corporate crime: Individualism,
collectivism, and accountability,” Sidney Law Review 12: 468-513.• The focus is usually on the managers in the middle because, although they exhibit
enough seniority to make important and visible decisions, they are not senior enough to be able to hide behind the diffusion of responsibility that provides top management cover
• Empirical research confirms this: Decision making at the operational level tends to be highly visible and are marked by clearly defined beginning, middle, and end states
• Top management decisions are more fluid, evolutionary, consensual, and temporal, where negations are carried on with numerous individuals and groups over a period of time
75
The Doctrine of “Many Eyes”• “Many eyes” as a solution to the problem of many hands• Given enough morally responsible individuals (“many eyes”) the network of
accountability can be managed • In a responsible organization, the many eyes that watch the many hands are a
watchdog that could prevent risk and harm• A culture of responsibility can develop if only because one can fix the errors of
another• Engineers have a responsibility to address the errors of their co-worker engineers
working on the same project• The motivation and ability to prevent risk and harm is increased, not by the fear of
punishment but rather by the desire to maintain a respectful standing within the profession or social group
• By ensuring that engineering projects be free of risk by guaranteeing that enough eyes watch the many hands that work or operate technologies, and by accommodating new modes of collaborative moral accountability through social—not legal– mechanisms, one can hope that the barriers to accountability will diminish
• A framework for moral and ethical debates needs to be developed that can accommodate meaningful discussions about exercising due care in engineering design and practice when working on large projects and/or in large organizations
• Related to the concept of “social responsibility”
76
The Diffusion of Responsibility Phenomenon
The Genovese Effect
•Kitty Genovese Murder NY, NY 1964•Fought off murderer he returned again and again •Rape and murder took full half hour•No one came to her assistance•Police determined that at least 38 neighbors were aware of the attack
•“Unresponsive Bystander Effect”
77
Diffusion of Responsibility• The tendency for persons in a group situation to fail to take action
because others are present, thus diffusing the responsibility for acting. A major factor in inhibiting bystanders from intervening in emergencies
• People are much more likely to intervene if they are alone rather than in the presence of others, especially if the other is a stranger
• Experiment: Reasoned that the presence of a stranger weakened individual responses by diffusing their sense of responsibility
• Finding: If individuals have their efforts identified when they are part of a cohesive highly moral group, they will exert even more effort than they would if they were only working for their own personal benefit
• Finding: If the roles and responsibilities of team members are not clearly identified, individuals will tend to “loaf” and they will not produce their best effort
78
•The greater the number of observers of wrongdoing, the less likely that any of them will feel morally responsible for acting to prevent the wrongdoingWhen individuals know that many others are present, then they as individuals will not bear the full burden of responsibility•Hypothesis: humans are certainly capable of self-interested behavior (in this case, lying back and let others do the work, but we are also capable of showing “team spirit” depending on slight but important modification to the situational conditions•Crucial variable: the extent to which our contribution is visible to the rest of the team•Moral Approbation plays a role here
The Diffusion of Responsibility Phenomenon
79
The Diffusion of Responsibility Phenomenon
• Tested Hypotheses– Predicted that group members would feel less
personally responsible than individuals and that the decrease in felt responsibility would be proportional to the size of the group
– Predicted that the valence of the group’s outcome would influence diffusion, such that members of successful groups would be less likely to diffuse responsibility in comparison to members of failure groups
– Both hypotheses were confirmed in major research
80
The Diffusion of Responsibility Phenomenon
• Individuals may fail to act as whistleblowers because they fear the consequences but also because they may expect someone else will take the appropriate steps to prevent a major problem
• The idea that “If I don’t do it somebody else will” shifts responsibility from the individual onto some unspecified “Other”
• The sociological literature suggests that the more one’s actions becomes disentangled from the organization in which they are embedded, the better able one to encourage their maximum contribution to the group, including alerting others when things are starting to go wrong
81
Groups reach much bolder, more adventurous decisions than any single member of the group would take
•No single individual is fully responsible for the decision taken so they are free to take risky decisions without bearing the burden of responsibility for them because the shared risk makes the individual risk less
•It is related to diffusion of responsibility and sometimes results from it
•Risky shift occurs when a group collectively agrees on a course of action that is likewise more extreme than they would have made if asked individually.
•Example: Ford Pinto
•What are the possible ethical implications of this phenomena? Why a Shift Toward Risk?
Explanation for it: The Risk As Value Hypothesis: Moderate risk is valued in our culture. Therefore, people want to shift toward risky decisions to gain status and approval
The Phenomenon of Risky Shift
82
Barriers to Individual Accountability
1. Self-Interest
2. Fear
3. Self-Deception
4. Ignorance
5. Egocentrism
6. Narrowness of Vision
7. Uncritical Acceptance of Authority
8. Groupthink
83
The Mens Rea Doctrine
• To seek individual liability in natural persons• The theory of fault based on the notion of free will, focused on human
psychological processes• Punishment: based on certain conscious, intentional psychologically
instantiated blameworthy choices made by a guilty mind • Centrality of Mens Rea Doctrine. Take for example:
Jones is presently depraved. She lacks the capacity for moral reflection and self-control. In a fit of rage she assaults a stranger for no good reason. Now consider two versions of the case.
(a) Jones is blamelessly depraved due to an acute psychosis that it was never in her power to predict or prevent.
(b) Jones has deliberately ingested a depravity-inducing drug because she’s a blocked novelist who wants to know how it feels to lose control.
Jones is culpable for the assault in the second case but not the first.
84
Responsibility
Indirect (Causal) Fault
SuretyshipMastershipHierarchyAuthorization
Individual
Vicarious Liability
Whole Groups
CollectiveAnd
Non Distributive
Collective And
Distributive
Liability withContributory
Guilt
Liability withNon-Contributory
Guilt
Collective
Strict Liability
Direct(Causal) Fault
CriminalIn TortContracts(ContractsWarranties
(UltraHazardousMaterials)
(Public WelfareMilk, Water)
(Bond AgencyMessenger Service)
(CorporationsMilitarySchools)
(Employer-Employee)
(Bonded Employee)
(Athletic teamsNazis-JewsWhites-Blacks)
(ConspiracyComplicity) Train Robbery
(Good Samaritan)
85
Accountability for Decision Making in Organizational Contexts
• Mistakes made in accident or disaster settings are often associated with a particular decision—for example, the decision to launch the Shuttle Challenger in below normal temperatures, the decision to change a box-beam design for a suspended walkway in the Hyatt-Regency case, or the decision to shoot down an Iranian Airbus aboard the USS Vincennes.
• If these decisions are followed by adverse events it usually seems that the decision maker was "at fault“
• However, greater understanding about the nature of decision making in complex organizations undermines such simple thinking.
86
Collective Responsibility• Collective responsibility, like personal responsibility, refers to
both the causal responsibility of moral agents for causing harm
• And the blameworthiness that we ascribe to them for having caused such harm
• It is always a notion of moral rather than purely causal responsibility
• It does not locate associate either causal responsibility or blameworthiness with discrete individuals or locate the source of moral responsibility in the free will of the individual moral agents
• Instead, it associates both causal and moral responsibility with groups and locates the source of moral responsibility in the collective actions taken by these groups understood as collectives
• Related to notions of group intentions, collective actions, and group blameworthiness
87
Collective Responsibility
Ascriptions of Responsibility for wrong doing to groups or large organizations
Arguments Against:
1. Organizations are not persons—unlike individuals, they cannot form intentions and hence cannot be understood to act or to cause harm as a group.
2. Organizations as distinct from their individual members, cannot be understood as morally blameworthy in the sense required by moral responsibility.
88
Theories of Collective Responsibility Arguments For:1. Organizational Decision Procedure2. Organizational Climate/Organizational Culture3. A collective action is caused by the
beliefs and desires (wants) of the collective itself, whether or not such beliefs and desires can be accounted for or explained in individualistic terms
4. Group intentions exist when two or more persons constitute the plural subject of an intention to carry out a particular action, or, in other words, when they are jointly committed to intending as a body to pursue a collectively-held goal or objective
1. A team is jointly committed and jointly intends to carry out a play
89
Collective Intentionality1. Collective intentionality -- Groups exhibit joint commitments to act
as a single body 1. Example: codes of professional ethics
2. “mind-sets” or “cognitive schema” unique to a group or organization E.g., engineers think differently than managers
3. We have a practice of attributing responsibility to organizations (consider, for instance, current tobacco lawsuits) and this seems to presuppose that organizations literally have intentional states. For we could not hold them legally and morally responsible for an action unless they intended to commit the act.
4. Groupthink as a possible example 5. Risky shift as a possible example
90
Theories of Collective Responsibility
Two conditions of Collective Moral Agency:
• Free Action: We must be able to claim that a collective performed an action such that the action cited cannot be reduced to the action of any given individual within the collective. (e.g., representational government)
• Knowledge: We must be able to say in some sense that the collective knew the consequences of the action, or should have known of these consequences, so as to support the claim that its action is intentional.
91
Theories of Collective Moral Responsibility
III. Determination of collective responsibility – determining the conditions under which the group is responsible to a third party for the actions of one of its members (Is a university collectively responsible for the sexual harassment of a student by a university employee?)
• The Causal Theory: the causal role that organizations play in shaping the behavior of their agents
92
Theory of Collective ResponsibilityCollective Responsibility can be located in the CID (Corporate
References• Bovins, B. (1998) The Quest for Responsibility, Accountability and
Citizenship in Large Organizations. Cambridge: Cambridge University Press• Gaylin, Willard and Bruce Jennings (1996) The Perversion of Autonomy:
The Proper Uses of Coercion and Constraints in a Liberal Society New York: The Free Press
• Latane, B. and R. Rodin (1969) “A Lady in Distress: Inhibiting Effects of Friends and Strangers on Bystander Intervention,” Journal of Experimental Psychology 5: 189-202
• Latane , B and K. Williams and S. Harkins (1979) “Many hands make light work: the causes and consequences of social loafing,” Journal of Personality and Social Psychology 37: 822-32.
• Smiley, Marion (1992) Moral Responsibility and the Boundaries of Community: Power and Accountability From a Pragmatic Point of View. University of Chicago Press
• Hacking, Ian “Culpable Ignorance and Interference Effects” in Values at Risk
