Published: March 18, 2011

r 2011 American Chemical Society 3848 dx.doi.org/10.1021/es1031294 | Environ. Sci. Technol. 2011, 45, 3848–3857

POLICY ANALYSIS

pubs.acs.org/est

Multiple-Criteria Decision Analysis Reveals High StakeholderPreference to Remove Pharmaceuticals from Hospital WastewaterJudit Lienert,* Mirjam Koller, Jonas Konrad, Christa S. McArdell, and Nele Schuwirth

Eawag: Swiss Federal Institute of Aquatic Science and Technology, Ueberlandstrasse 133, P.O. Box 611, 8600 Duebendorf, Switzerland

bS Supporting Information

ABSTRACT: Point-source measures have been suggested to decrease pharmaceuticals in water bodies. We analyzed 68 and 50alternatives, respectively, for a typical Swiss general and psychiatric hospital to decrease pharmaceutical discharge. Technicalalternatives included reverse osmosis, ozonation, and activated carbon; organizational alternatives included urine separation. Tohandle this complex decision, we used Multiple-Criteria Decision Analysis (MCDA) and combined expert predictions (e.g., costs,pharmaceutical mass flows, ecotoxicological risk, pathogen removal) with subjective preference-valuations from 26 stakeholders(authorities, hospital-internal actors, experts). The general hospital contributed ca. 38% to the total pharmaceutical load at thewastewater treatment plant, the psychiatry contributed 5%. For the general hospital, alternatives removing all pharmaceuticals(especially reverse osmosis, or vacuum-toilets and incineration), performed systematically better than the status quo or urineseparation, despite higher costs. They now require closer scrutiny. To remove X-ray contrast agents, introducing roadbags ispromising. For the psychiatry with a lower pharmaceutical load, costs were more critical. Stakeholder feedback concerning MCDAwas very positive, especially because the results were robust across different stakeholder-types. Our MCDA results provide insightinto an important water protection issue: implementing measures to decrease pharmaceuticals will likely meet acceptance. Hospitalpoint-sources merit consideration if the trade-off between costs and pharmaceutical removal is reasonable.

’ INTRODUCTION

Pharmaceuticals are only partially removed in wastewatertreatment plants (WWTPs) and have been detected in waterbodies. For some, negative consequences on aquatic ecosystemshave been established, but often cause�effect relationships arelacking.1 X-ray contrast agents give rise to concern because oftheir persistence and usage in highest quantities.2 Additionally,aquatic ecosystems must cope with mixtures of different com-pounds that interact or show concentration additivity.3 Manycountries are discussing precautionary measures; Swiss autho-rities propose upgrading the 100 largest WWTPs4 by ozonationor activated carbon.5,6

Upgrading WWTPs to deal with water pollution follows theruling paradigm of centralized wastewater treatment, which isincreasingly challenged;7 source control is an alternative.8 Thereare several source control options to decrease water pollutionfrom pharmaceuticals. The most radical, strict prohibition, ishardly conceivable in the case of pharmaceuticals. Designingbiodegradable pharmaceuticals and labeling of environmentallyfriendly medicaments are options.9 Partial source control mea-sures capture some micropollutants. Urine separation (NoMix-technology;8) is one such possibility and could decrease 60�70%of the pharmaceutical load,10 and 50% of the ecotoxicologicalrisk.11 Another possibility, separate treatment of polluted waste-water known from industry is being considered for pharmaceu-tical point sources, especially hospitals (e.g., refs 12�15; www.pills-project.eu). In hospitals, additionally to pharmaceuticals,pathogens and multiantibiotic resistant bacteria give rise toconcern.16 To determine whether the hospital point source isrelevant, several projects have quantified the pharmaceuticalfraction from hospitals at WWTPs. Most found the hospitals’

contribution low, ranging from 10 to 18%,17�19 because phar-maceuticals are widely used throughout the population. More-over, although cytostatics or X-ray contrast agents areadministered in hospitals, only ca. 50% of the latter are foundin hospital sewers, because half the patients go home after X-ray-treatment.18

Although the above studies indicate that hospitals are notalways major sources of pharmaceuticals, it is premature to rejectseparate treatment of hospital wastewater. In a related study wefound that the hospitals’ contribution to the total load stronglydepends on the local situation, the hospitals’ specialization, andpharmaceutical type.20 Moreover, we identified pharmaceuticalsfrom hospitals with a high ecotoxicological risk that were rarelyinvestigated before (amiodarone, ritonavir, and clotrimazole;20),because pharmaceuticals with high ecotoxic potential are notnecessarily those administered in large amounts.1 Hence, furtherresearch to determine the relevance of hospitals as point-sourcesis needed, and this study intends to contribute.

In complex decision situations with multiple, conflictingobjectives and large uncertainty, Multiple-Criteria DecisionAnalysis (MCDA) offers support (e.g., refs 21,22). MCDAmethods structure decision processes and acknowledge thatdecisions with conflicting objectives inevitably include subjectivejudgments. MCDA clearly discriminates between “scientific”expert predictions of consequences of decision alternatives and(subjective) preferences elicited from decision makers. MCDA

Received: September 14, 2010Accepted: March 1, 2011Revised: January 12, 2011

3849 dx.doi.org/10.1021/es1031294 |Environ. Sci. Technol. 2011, 45, 3848–3857

Environmental Science & Technology POLICY ANALYSIS

increases transparency and helps finding consensus solutions formultiple stakeholders (e.g., ref 23). Basically, MCDA decom-poses complex problems into manageable parts. It transformsoutcomes for each alternative into a common unit with a multi-attribute value (utility) function using weights that reflect thedecision makers’ preferences.

We report on an MCDA involving the 26 most-importantstakeholders in two Swiss cases; a general and a psychiatrichospital. Details of the MCDA methodology are given in ref 24and of the ecotoxicological risk assessment in ref 20. Here, themain aim is to evaluate different alternatives that decreasepharmaceuticals in the hospitals’ wastewater, based on two casestudies. An additional aim is to receive stakeholder feedbackconcerning the MCDA approach in this complex real-worldsetting. We combine stakeholder preferences with expert knowl-edge from engineers, ecotoxicologists, chemists, health andwastewater authorities, and hospital actors. It is an exemplaryanalysis within strictly defined hospital boundaries, excludingend-of-pipe alternatives at WWTPs. We hope to provide insightinto this complex decision problem. We discuss implications togive guidance on dealing with pharmaceuticals, also in othercontexts.

’METHODS

Case Studies. The general hospital in Baden is a typicalregionally important Swiss hospital with 338 effectively usedbeds, offering the whole range of medical services (see Support-ing Information (SI) p SI-2; Table SI-2). It participates in a pilot-plant project to remove pharmaceuticals from wastewater.25 Thetypical psychiatric clinic Hard (Embrach) with 211 effectivelyused beds (“psychiatric hospital”) has a very different medicalmandate, reflected in different prescribed pharmaceuticals(Table SI-3). It is understood that alternatives entailing largereconstruction (e.g., NoMix-toilets) can only be implemented incase of renovations.Multiple-Criteria Decision Analysis (MCDA). MCDA con-

sists of following steps:21�24 (1) Framing and stakeholderanalysis, define decision problem; (2) identify objectives andattributes; (3) identify alternatives (measures); (4) predict out-comes for each alternative objectively; (5) elicit and quantify(subjective) stakeholder preferences for outcomes; (6) integratesteps 4 and 5 and calculate the value of each alternative for eachstakeholder, rank alternatives, sensitivity analyses; and (7) sta-keholder feedback. The MCDA produces and synthesizes a largeamount of data and information, which are accessible via theSupporting Information. The study map (Table 1) helps tonavigate through these data. It summarizes the main study-specific results in each step, and references the detailed sources.An overview is given below.Steps 1�3: Problem Formulation. We carried out two sets

of interviews. One aim of the first set (p SI-2) was a stakeholderanalysis (stakeholders presented in Figure 2 and Table SI-1).Furthermore, we discussed objectives one wants to achieveby introducing alternatives to decrease pharmaceuticals, andcreated and discussed alternatives. We also discussed attributesthat quantify the degree of fulfillment of each objective (seeTable 1).Step 4: Predict Outcomes for Each Alternative. Details are

given in the Supporting Information (pp SI-3�SI-4; also seeTable 1). Annual costs of each alternative were calculated byHunziker Betatech AG (www.hunziker-betatech.ch; annual

interest rate: 4%, investment period: 15 yrs.). The estimationsof the ecotoxicological risk potential and load of pharmaceuticalsare part of this study; the main results are summarized in TableSI-3. Details are published separately: the method to estimate theecotoxicological risk potential and load of pharmaceuticals isdescribed in Escher et al.20 Pharmaceutical load (kg/year) wascalculated with the hospitals’ pharmaceutical lists from 2007 andexcretion rates for 100 pharmaceuticals excreted in highestamounts (“Top-100”20). To estimate the decreased load aftertreatment, we compiled literature data for each pharmaceutical,including measurements in a pilot plant.25 For each alternative,total load was summed from loads of individual pharmaceuticals.The ecotoxicological “Risk Quotient” (RQ) was estimated witheach pharmaceutical’s toxicity toward water organisms andsummed for themixture of the Top-100 pharmaceuticals.20 Loadreduction of pathogens and multiantibiotic resistant bacteria wasestimated only for the general hospital. We estimated feasibilitywith additional time of nurses (instruct patients on usage ofNoMix-toilets/roadbags, deal with complaints) and the percen-tage of patients unhappy with the measure. We used neutral,positive and negative articles in mass media, based on mediaexpert-judgments. Since authority representatives were alsostakeholders, we used their MDCA results (Table SI-6) asestimate for “acceptance by authorities” for all other stakeholders(attribute excluded for authority-representatives).Step 5: Elicit Stakeholder Preferences. The advantage of

MCDA is that it focuses on values of stakeholders rather thanalternatives.21 In the second interviews, we presented the generalproblem to each stakeholder (Figure SI-1) and discussed pre-ferences for outcomes without asking for direct judgments of thealternatives. For instance, stakeholders knew that we includedNoMix toilets, but were not asked whether they liked them.Rather, we elicited how important achieving each objective(Figure 1) was with weighting factors w which sum up to 1. Tocompare attributes with different units (e.g., costs with %unhappy patients), we elicited single-attribute value functions v.These have a continuous scale between 0�1 at the y-axis andthe considered range of the attribute in its original unit on thex-axis: 0 means worst and 1 best fulfillment of the objective. Wethen used the weighted average of the values of each attributeto calculate the overall value of each alternative according to eq 1(e.g., refs 21,22):

vðaÞ ¼ ∑m

i¼ 1wi 3 viðaiÞ ð1Þ

where

v(a) = total value of alternative aai = attribute level of alternative a for attribute ivi(ai) = value for attribute i of alternative awi = weighting factor of attribute i; ∑wi = 1The attribute levels ai (“predictions”; e.g., costs) were esti-

mated (see Results and Table SI-4). The “subjective valuations”(single-attribute value functions vi, weights wi; Table SI-8) wereelicited from each stakeholder to calculate the total value of eachalternative, v(a) (Table SI-6).Elicitation is challenging and it is important to avoid introducing

biases.21 We found that standard methods can be somewhatimpracticable.Our procedure 24 compromises between timedemandand elaborateness, but remains methodologically satisfactory: (1)framing (Figure SI-1); (2) instead of the usual “Swing-method”22

to elicit weights we introduced a “Reversed-Swing-method” to

3850 dx.doi.org/10.1021/es1031294 |Environ. Sci. Technol. 2011, 45, 3848–3857

Environmental Science & Technology POLICY ANALYSIS

rank objectives; (3) elicit single-attribute value functions with“Midvalue-Splitting-method”22 for most-important objective; (4)elicit weights with “Reversed-Swing-method”; (5) check for additiveaggregation (i.e., weighted average, eq 1) and consistency with“Trade-Off-method” for two most-important objectives.TheMCDAprocedure and results are synthesized inTable 2 for

representative alternatives and five stakeholders from the generalhospital. As example, alternative no. 44 (collect 83% of all urinewith NoMix toilets, etc.; treat urine with SBR þ ozonation) isestimated to cost CHF 96,972 year�1, but only decreases thepharmaceutical load to 438 kg year�1, etc. (Table 2A). Stakeholder

3 (wastewater authorities) gave 8% of the weight to “low costs”,77% to “good wastewater quality”, 12% to “good feasibility”, and4% to “good public perception” (Table 2B). These weights wi areused, together with single-attribute values vi(ai) (not shown)to calculate the total value score for each alternative v(a) (eq 1).For stakeholder 3 (authorities), the NoMix-alternative 44 receiveda total value score of 0.622 (Table 2C), which is better than thestatus quo (0.562; Figure 2A). Note: the stakeholder did not judgealternative 44 directly. The value of alternative 44 is exclusivelycaused by the elicited weights, the single-attribute value functions,and the predictions of the attributes.

Table 1. Study Map of the Multi-Criteria Decision Analysis (MCDA) Showing Steps of the MCDA, the Main Study-SpecificResults, and Where More Details Can Be Found in the Paper or the Supporting Informationa

aWWTP = communal waste water treatment plant, SH = stakeholder. b Interdisciplinary Eawag group: chemists, ecotoxicologists, engineers, andMCDA-experts. c Single-attribute value functions are elicited from each stakeholder. They transform attribute levels (e.g., costs in CHF, orpharmaceutical load in kg/y) into a continuous, dimensionless scale between [0, 1], where 0 = worst level (e.g., highest possible costs) and 1 = bestlevel (no costs). This allows comparing different attributes (e.g., costs with pharmaceutical load) and is required to form the weighted multiattributevalue function that calculates the total value scores of each alternative. dThe weighting factors are elicited from each stakeholder. Weights reflect thepersonal importance of the specific attribute range of each objective relative to those of the other objectives (e.g., how important are 0�1.5 Mio CHFrelative to 0�6 positive media articles).

3851 dx.doi.org/10.1021/es1031294 |Environ. Sci. Technol. 2011, 45, 3848–3857

Environmental Science & Technology POLICY ANALYSIS

Step 6: Uncertainty and Sensitivity Analyses. We ac-counted for data variability and uncertainty of predictions bydefining probability distributions for each attribute (Table SI-4)and propagating this input uncertainty to the results with MonteCarlo simulations. Because detailed uncertainty estimation forpharmaceutical loads and RQs was not available, we assumedrather high standard deviations (load: 15%; RQ: 20% becauseof additional assumptions for ecotoxicity estimates). Utilitytheory accounts for uncertainty in decisions, but requires trickyelicitation of the decision makers’ risk attitude with lotteries.22

Here, we assumed risk neutrality (value function = utilityfunction) but checked the results’ robustness to this and otherassumptions by sensitivity analyses, including intrinsic risk-aver-sion, doubled standard deviations for pharmaceutical load andecotoxicological risk, assuming linearity/nonlinearity of single-attribute value functions, and 25% increased weight of objectives“low costs” and “good wastewater quality” (see ref 24).We used the statistics and graphics software R (http://www.

r-project.org) and developed an R-package for utility calculations(Reichert, P., Schuwirth, N., in preparation). We discussed resultswith stakeholders and received feedback concerningMCDA(p SI-6).

’RESULTS

Objectives and Attributes. Four fundamental main objectivesand nine fundamental subobjectives were identified (Figure 1). Thestakeholders mentioned more, including “high safety”, which wasomitted (p SI-9). Some interviewees suggested classifying removalof X-ray contrast agents as separate objective. Because they con-tribute substantially to the pharmaceutical load, we decided thattheir removal is sufficiently covered by the load-objective (seeDiscussion). The suggestion to calculate CO2-emissions was be-yond our scope; we propose ex-post Life Cycle Assessment (LCA)only for the most promising alternatives (see Discussion).Alternatives to Decrease Pharmaceuticals. After stake-

holder discussions, we included 68 viable alternatives for thegeneral hospital (Tables 2A and SI-4A) and 50 for the psychiatrichospital (Table SI-4B) by combining collection pathways andtreatment methods (Figure SI-2; pp SI-9�SI-13). The “status

quo” (collect nothing) was included as the first alternative, sinceit is interesting whether any measure should be introduced.Collection Pathways. Excreted pharmaceuticals can be col-

lected from the following: “total (mixed) wastewater from entirehospital” containing excreta and gray water; “mixed wastewaterfrom certain buildings”; or “certain wards” as radiology oroncology (this proved to be unrealistic); “toilets only”, i.e.,excreta preferably collected with vacuum toilets; and collectionof “urine” because ca. 65�75% of the pharmaceutical load is inurine, albeit only 12�25% of the ecotoxicological risk.20 How-ever, urine-collection with NoMix-toilets, urinals, bedpans, urinebottles, and catheters is easy and cheap (Tables 2A and SI-4).Therefore, we included the following: “all urine”; “urine fromwhere it is collected anyway”, equaling 1/3 of urine in generalhospital (not for psychiatry; Table SI-2A); “waterfree urinals”because men already use urinals today in the psychiatry; and“roadbags” (www.roadbag.net) to collect X-ray contrast agents.Collecting X-ray urine would be extremely advantageous, sincecontrast agents are excreted 90�94% with urine (Table SI-3B;ref 11). Around 2/3 of X-rays are with outpatients (Table SI-2C); we included “only stationary patients”; and “all patients”receiving X-rays. “Source control” (replacement, reduction,prohibition of pharmaceuticals) was excluded (p SI-10).Treatment Methods.Collected wastewater can be transported

(by truck) or processed in the hospital. There exist manytreatment alternatives in different development stages. Weconsidered (Figure SI-2; pp SI-11�SI-13) the following: “trans-port and incineration” to solid waste incineration plant; “me-chanical pretreatment” (to remove solids; unnecessary forurine); and “biological pretreatment” (decrease nutrients andorganic matter). Options are biofilm or membrane bio-reactors(MBR; usually chosen in hospital pilot projects). Urine can bepost-treated with or without biological pretreatment; we chosesequencing batch reactor (SBR), based on earlier experience.26

“Post-treatment to eliminate pharmaceuticals”: many technol-ogies are being developed; some have matured to pilot plant stageto treat wastewater from hospitals or at WWTPs (pp SI-12�SI-13). Most frequently discussed are “ozonation” (O3) or “pow-dered activated carbon” (PAC). Because removal of X-ray contrastagents was often unsatisfactory, we included “ozonation combined

Figure 1. Objectives and attributes in a decision to decrease pharmaceuticals in hospital wastewater. In the first set of stakeholder interviews four mainobjectives (light blue) and nine subobjectives (yellow) were identified. The degree of fulfillment of each objective is measured with nine attributes.

3852 dx.doi.org/10.1021/es1031294 |Environ. Sci. Technol. 2011, 45, 3848–3857

Environmental Science & Technology POLICY ANALYSIS

with granulated activated carbon” (GAC).We considered low andhigh doses for ozonation and activated carbon. We included“dense membranes” that fully remove pharmaceuticals and patho-gens; but doubled uncertainty to 25% (costs; Tables 2A and ; SI-4)and did not distinguish betweennanofiltration and reverse osmosis(RO; below nanofiltration is included in RO). All alternativesapply to “post-treatment of urine”, but for urine many others are

possible; e.g., phosphate recycling with struvite precipitation(review in ref 26). We dismissed further alternatives.Pharmaceutical Load and Ecotoxicological Risk Quotient

(RQ). The Top-100 lists of the most-used pharmaceuticals in2007 account for nearly the total load. In the general hospital,1154 kg pharmaceuticals was consumed and 779 kg was excreted(Top-100 list: 1137 and 777 kg excreted; Table SI-3A; p SI-13;ref 20). In the psychiatric hospital, 52 kg was consumed and 17 kgwas excreted (Top-100 list: 50 and 17 kg excreted; Table SI-3C).The general hospital contributes around 38% to the total load attheWTTP and the psychiatric hospital contributes 5%, assumingthat all patients treated at the hospital are also excreting into thehospital sewer.Raw wastewater from the hospital has a very high total RQ,

which is decreased by dilution in sewers; we used the latter toevaluate pharmaceutical concentrations arriving at the WWTPand originating from the hospital. In the general hospital, the RQof raw wastewater today (“status quo”) was 239 (Table SI-3A),and 3.2 after dilution (Table 2A; p SI-22; ref 20). RQ > 1indicates a risk for the aquatic environment. For the psychiatrichospital todays RQ = 114 (Table SI-3C), and 1.5 after dilution.Main Result: High Rank for Alternatives Removing All

Pharmaceuticals. The predicted performance of each alterna-tive regarding each attribute (Tables 2A and SI-4) was integratedwith the “subjective” stakeholder preferences. For an overviewof the value score calculations for selected alternatives andstakeholders see Table 2. Resulting values (eq 1) for the 68(general hospital)/50 alternatives (psychiatric hospital) for all26 stakeholders are shown in Figure 2 (and Tables 2C and SI-6).Additionally, we ranked alternatives from best to worst (Table SI-7;Figure SI-5). Because some alternatives performed very similarly,we present a representative selection in Figure 2 (p SI-28). Thesegeneral results were very robust in the sensitivity analysis.24

For most stakeholders in the general hospital, at-source treat-ment (or incineration) of total mixed wastewater (T) was clearlypreferable to the status quo or collecting urine (Ur) fromwhere itis collected anyway (Figure 2A). This was followed by collectingX-ray urine with roadbags (RdBg). NoMix alternatives (NMX)received lowest scores. The pattern was similar, but less distinctfor the psychiatric hospital (Figure 2B).General Hospital. Treating the total wastewater from the

main building with reverse osmosis or nanofiltration (RO) wasthe best-ranked alternative, based on the median of 13 stake-holders (Table SI-7A, alternatives 25 and 33). RO is the most-expensive alternative, but has highest removal of pharmaceuticalsand pathogens (Tables 2A and SI-4). RO-alternatives treatingwastewater from all buildings,17,9 had much higher standarddeviations (SD) because costs were nearly doubled. Vacuumtoilets and incineration (alternative 34) ranked third; anotherexpensive, but highly efficient alternative. However, the SD of therank was high (11.7), ranking first for two authority representa-tives, but ranking somewhat lower (28�33) for three otherstakeholders. Generally, the value differences between RO andvacuum toilets compared with other top-ranked alternatives(treating total wastewater with ozonation or activated carbon)were extremely small (Tables 2C and SI-6A). The status quoranked 50, averaged over all stakeholders (SD 7.4; Table SI-7A).Differences were far less pronounced for urine-alternatives. Thebest averaged rank was 33 (urine collected anyway and incinera-tion; alternative 66).Psychiatric Hospital. The pattern for the psychiatric hospital

was very similar (Table SI-7B). Vacuum toilets and incineration

Figure 2. Result of MCDA; average values for selected alternatives todecrease pharmaceuticals. We show values for 13 stakeholders in eachhospital on a scale from 0 (objectives not at all achieved) to 1 (allobjectives fully achieved). The numbers in the legend refer to thealternative (Table SI-4). Legend: Status Quo (black). T = Total mixedwastewater (general hospital: main building, psychiatry: all buildings;blue) with MBR pretreatment and post-treatments: none/O3 = Ozona-tion/PAC = Powdered Activated Carbon/O3GAC = Ozonation þGranulated Activated Carbon, RO = Reverse Osmosis (includingnanofiltration). Vac Inc. = Vacuum toilets, transport, incineration.NMX = total urine collected with NoMix-toilets, waterfree urinals,and urine where collected anyway (red); with Sequencing Batch Reactor(SBR) pretreatment and post-treatment (none/O3/PAC/O3GAC/RO), or incineration (Inc.). Ur is identical to NMX, but collects onlyurine which is collected anyway in general hospital (urinals, bedpans,catheters), andmen using urinals in psychiatry (yellow). RdBgH = threeroadbags given to patients staying in hospital after X-rays (H) or also toout-patients (HH; only general hospital; green).

3853 dx.doi.org/10.1021/es1031294 |Environ. Sci. Technol. 2011, 45, 3848–3857

Environmental Science & Technology POLICY ANALYSIS

(alternative 18) ranked highest in themedian for all 13 stakeholders(rank 1�3), except for two authorities (Federal Office for Environ-ment: rank 15/cantonal health: 19); followed by RO. For thecantonal health authorities representative, all alternatives treating“total wastewater” ranked far lower than for everybody else. Thestatus quo ranked first for cantonal health authorities (administrativedirector: 2; engineering consultant: 3), but 16�33 for all others.Urine alternatives ranked low, except “urine from urinals” forcantonal health authorities representative (rank 2�17).

Stakeholder Preferences: Weights. The weights given to theobjectives help to explain the results (Table 2B; details: Figure SI-3; Table SI-8; pp SI-37�SI-38). In the general hospital, “goodwastewater quality” received the highest average weight (0.45),followed by “good feasibility” (0.21), “low costs” (0.19), and “goodpublic perception” (0.16; Table 3). In the psychiatric hospital thepattern was similar but “low costs” received higher weights (0.25),while” public perception” only had 0.11. However, differencesbetween the two hospitals were only significant for “low load of

Table 2. Combined Results of the MCDA for Nine Selected Alternativesa to Decrease Pharmaceuticals and Five SelectedStakeholdersb in the General Hospital [(A) Expert Predictions for Each Objective/Attribute (See Figure 1) of the SelectedAlternatives; (B) Weights Given to Objectives by Selected Stakeholders; (C) Final Value Scores of the MCDA of SelectedAlternatives for Selected Stakeholders on a Scale from 0 (Objectives Not at All Achieved) to 1 (All Objectives Fully Achieved)]c

aThe selected alternatives are representative of the main types of alternatives: status quo (“do nothing”, no. 1), total mixed wastewater with pretreatment(here: MBR) and post-treatment (20, 22, 24, 25), vacuum toilets and incineration (34), urine collected with NoMix toilets (44) or where collected anyway(60), and urine from patients with X-rays collected with roadbags (67). As part C and Figure 2 show, the differences between the value scores for similartypes of alternatives are usually fully negligible (e.g., MBRþ ozonation≈MBRþ powdered activated carbon). bThe stakeholders were selected becausethey differ rather strongly in their preferences (weights, part B) and the resulting value scores (part C; also see Figure 2). cThe table summarizes themeaningof the data in theMCDA and references detailed full data tables available in the Supporting Information. Uncertainty estimates are standard deviation (SD),lowest estimate (Min), highest estimate (Max), and distributions (Distr) for error calculations: truncated normal (nor), or triangle (tria). ww =wastewater;ur = urine; X-r ur = urine containing X-ray contrast agents; RQ (ecotoxicological risk quotient) includes dilution in sewer (see Methods).

3854 dx.doi.org/10.1021/es1031294 |Environ. Sci. Technol. 2011, 45, 3848–3857

Environmental Science & Technology POLICY ANALYSIS

pharmaceuticals” (higher in general hospital) and “low ecotox-icological risk potential” (higher in psychiatry; Table 3).Values of Subobjectives. The results were further examined

with the values (wi 3 vi(ai)) for each subobjective (Table SI-6;Figure SI-4: average of 13 stakeholders and clustered alternativestermed “management options”). In the general hospital, manage-ment options with treatment (or incineration) of total mixedwastewater clearly had the highest total value even though theobjective “low costs” achieved lower values than the status quoand urine management alternatives (Figure SI-4A). This isbecause these “total” management alternatives performed betterfor ecotoxicological risk and especially pathogen-removal. Thissubobjective is neither fulfilled by the status quo, where poten-tially infectious wastewater is released to WWTPs, nor by urine-collection (pathogens are mainly in feces). NoMix-toilets requireeffort by staff and patients and thus rate worse than “urinecollected anyway”. Roadbags do not (fully) meet several sub-objectives: low ecotoxicological risk, low load, pathogen-removal,and low effort for staff.Results are similar, but less distinct for the psychiatric hospital

(Figure SI-4B). Averaged over different alternatives, manage-ment options with treatment of total mixed wastewater had totalvalues similar to the status quo, while vacuum toilets performedslightly better. The main discrimination between these “total”management and urine-alternatives is better fulfillment of thetwo subobjectives for good wastewater quality, regardless oflower fulfillment of “low costs”. NoMix-alternatives performedespecially poorly concerning low effort for staff and patients.We caution against overinterpretation of these results; they only

illustrate a trend. There is large uncertainty in these data due tothe uncertainty of predictions (Tables 2A, SI-4, SI-6), whichwe accounted for with Monte Carlo simulations and sensitivityanalyses. Differing judgments of each stakeholder (Tables 2B andSI-8) and the outcomes of different alternatives that were clusteredwithin management options (Figure SI-4) further increaseuncertainty.

’DISCUSSION

The main conclusion from our MCDA is that at-sourceremoval of pharmaceuticals from the general hospital shouldbe preferred to the status quo, where untreated wastewater isreleased to municipal wastewater treatment plants (WWTPs;Table 2C; Figure 2A). Alternatives that remove all pharmaceu-ticals (full treatment of all wastewater or incineration) performedsystematically better, despite higher costs (Table 2A). As ex-planation, compared with the hospitals’ operating costs (CHF190 Mio; Table SI-2A; www.kantonsspitalbaden.ch), costs forremoving pharmaceuticals seem reasonable (max. 1.2 Mio.annual costs; i.e., <1%; Table SI-4A). Additionally, the highest-ranked alternatives remove pathogens and multiantibiotic resis-tant bacteria (Figure SI-4A). However, we did not consideralternatives to upgrade WWTPs (see below).

For the psychiatric hospital, results are less clear (Figure 2B).On average, treatment of the total mixed wastewater achievedvalues similar to the status quo, while vacuum toilets andincineration performed slightly better (Figure SI-4B). In thepsychiatry, the focus lies on the trade-off between low costs andkeeping pharmaceuticals out of wastewater.

Generally, these results can best be explained because theobjective “good wastewater quality” received highest weightsfrom nearly all stakeholders (Table 2B; Figure SI-3; Table SI-8,pp SI-37�SI-38). Based on these stakeholder’s preferences,those alternatives that remove all pharmaceuticals performedbest in the MCDA, despite higher costs. However, weights for“wastewater quality” from hospital-internal and health-sectorstakeholders were far lower than those from other stakeholders.

We wish to emphasize the MCDA methodology again(Table 1). The performance of alternatives is calculated basedon “scientific” prediction-data (e.g., costs; Tables 2A and SI-4)and “subjective” preferences of stakeholders for objectives elicitedin interviews (value functions and weights given to each objec-tive, Tables 2B and SI-8). The values and rankings are not a resultof direct choices by stakeholders for any alternative.21

Table 3. Average Weights Given to Objectives in General (Gen) and Psychiatric Hospital (Psy)a

main objectives wwqual feasib public

costs ww-qual feasib public ecotox load staff patients posmed negmed legal

aver gen 0.19 0.45 0.21 0.16 0.47 0.53 0.44 0.56 0.27 0.36 0.48

psy 0.25 0.44 0.20 0.11 0.67 0.33 0.46 0.54 0.21 0.45 0.50

t test 0.15 0.82 0.77 0.30 0.02 0.02 0.78 0.78 0.44 0.38 0.82

medi gen 0.20 0.42 0.21 0.16 0.50 0.50 0.47 0.53 0.18 0.38 0.48

psy 0.31 0.39 0.20 0.12 0.67 0.33 0.41 0.59 0.20 0.37 0.53

SD gen 0.09 0.12 0.09 0.13 0.15 0.15 0.19 0.19 0.26 0.21 0.24

psy 0.13 0.15 0.09 0.07 0.23 0.23 0.22 0.22 0.12 0.28 0.15

min gen 0.04 0.32 0.08 0.00 0.26 0.31 0.05 0.23 0.00 0.05 0.11

psy 0.04 0.27 0.06 0.04 0.17 0.00 0.23 0.17 0.05 0.11 0.25

max gen 0.33 0.77 0.38 0.45 0.69 0.74 0.77 0.95 0.95 0.83 0.83

psy 0.43 0.74 0.36 0.22 1.00 0.83 0.83 0.77 0.37 0.95 0.67aOverview statistics for 13 stakeholders in each hospital: average (aver), median (medi), standard deviation (SD), lowest weight (min), and highestweight (max). T-test shows significant differences in averages from each hospital (unpaired two-tailed Student’s T-test; bold: p < 0.05). Objectives seeFigure 1; details in Supporting Information (Figure SI-3, Table SI-8, pp SI-37�38). Legend main objectives: low costs (costs), good wastewater quality(wwqual), good feasibility (feasib), good public perception (public). Subobjectives: low ecotoxicological risk potential (ecotox), low load ofpharmaceuticals (load), low effort for nursing staff and patients, high positive (posmed) and low negative media coverage (negmed), high acceptanceby authorities (legal). In “legal”, authorities were excluded. The subobjectives for wwqual were rescaled in the general hospital, because no weights for“low load of pathogens” were elicited in the psychiatric hospital.

3855 dx.doi.org/10.1021/es1031294 |Environ. Sci. Technol. 2011, 45, 3848–3857

Environmental Science & Technology POLICY ANALYSIS

Consensus Solutions in Psychiatric Hospital. Good waste-water quality is not unimportant for the psychiatric hospital,although stakeholders were fully aware that the hospital con-tributes only 5�8% to the pharmaceutical load at the WWTP.The best way to increase acceptance of alternatives that improvewastewater quality would be cost reduction (or sponsoring/subvention). Acceptance could also be stimulated if pharmaceu-ticals were included in water quality standards. However, thecurrent proposal by Swiss authorities rather targets centralizedsolutions (upgrade WWTPs) than decentralized solutions athospitals.4 In the psychiatric hospital, some conflict potential isforeseeable between many stakeholders and cantonal healthauthorities, who placed much more weight on “low costs”(0.38) than on “good wastewater quality” (0.27; pp SI-36�SI-38). As extreme contrast, “wastewater quality” received a highweight by cantonal wastewater authorities (0.74). An explanationis the professional focus of wastewater authorities, whose man-date is to protect water bodies. Moreover, as elsewhere, the Swisshealth sector is under substantial financial pressure. If at-sourcewastewater treatment were implemented in the psychiatry, roundtable discussions between wastewater and health authorities tofind compromise solutions are required. For instance, ozonationmight suffice5,6 because it removes around half of the pharma-ceutical load at lower costs compared to reverse osmosis (ca.CHF 243,000 instead of 382,000/yr; Table SI-4B).Choice of Alternative in General Hospital (Reverse Osmo-

sis, Vacuum Toilets). The main discussion in the generalhospital should now focus on choosing an optimal alternativeto treat wastewater that ideally removes the total pharmaceuticalload (Figure 2A). This clear result is understandable: our inter-view partners were well aware that the general hospital con-tributes ca. 1/3 to the pharmaceutical load at the WWTP.Consequently, lower weights were given to costs in the generalthan in the psychiatric hospital (Table 3; pp SI-36�SI-38).Comparing value rankings shows that densemembranes (reverse

osmosis/nanofiltration) rank first for 11 of 13 stakeholders (FigureSI-5; p SI-40). Vacuum toilets and incineration ranked second, butthis alternative has the highest conflict potential and should bediscussed.Moreover, installing vacuum toilets is only cost-efficient ifthe hospital is renovated. It cannot be emphasized enough that thisdistinction between alternatives would not be statistically significantand that there are large overlapping uncertainty ranges (also seeFigure 2A). We tentatively conclude that the choice of the alter-native is not critical, as long as it removes most pharmaceuticals anddoes not introduce other drawbacks.We caution against implementing reverse osmosis (RO) or

vacuum toilets without further scrutiny. Rather, our study identi-fies reasonable paths-of-action, also from stakeholder perspectives,which helps setting research priorities. Treatment options withdense membranes are highly efficient, removing nearly allcontaminants.14 However, these are concentrated in brine streamsthat require adequate disposal.14,27 RO-concentrate could betreated with ozonation,28 which is arguable because of high DOCand salinity. Likewise, PAC must be disposed of in landfills orincinerated. Additionally, GAC could be regenerated,27 requiringsignificant energy. We were surprised that vacuum toilets per-formed so well, which might be caused by nonconsideration ofadditional objectives. Stakeholders initially suggested including“low carbon footprint”. We propose LCAs now to comparevacuum toilets with RO and other high-ranked alternatives.Finally, we did not account for additional advantages of on-site

treatment of hospital wastewater such as saving wastewater taxes

by direct discharge, lower water bill, heat recovery, and recyclingof iodine from X-ray contrast agents (p SI-41). Further researchis also required concerning multiantibiotic resistant bacteria,16

where hospitals are critical point sources.Urine Collection.NoMix-toilets systematically received much

lower values than the status quo; alternatives with “urinecollected anyway” performed about as well (Figure 2;Tables 2C and SI-6). Main explanation is the high emphasis ofstakeholders on good wastewater quality, which is only partiallyimproved with urine separation compared to the status quo.Other objectives were not achieved, especially pathogen removal.Moreover, acceptance by staff, patients, and the media isexpectedly lower compared with more-conventional alternatives(Tables 2Aand SI-4; ref 29). Interestingly, costs seemed lesscritical (Figure SI-4). There are excellent other reasons for urineseparation such as nutrient recycling and protecting water bodiesfrom eutrophication.8 However, our MCDA analysis clearlyrevealed that separate collection of urine is not a good optionto decrease pharmaceuticals in hospital wastewater, with excep-tion of X-ray contrast agents.X-ray contrast media are used in hospitals in highest amounts:

448 kg of the 7 most-important X-ray contrast agents wereexcreted in 2007 from the general hospital (of totally excreted777 kg Top-100 pharmaceuticals/year = 58%; Table SI-3). Largequantities are found in water bodies.2 According to currentknowledge there is no environmental risk; the Risk Quotientof contrast agents was far below the threshold value of 1 (TableSI-3B; ref 20). Nevertheless, because of their persistence wastewater experts are considering a precautionary approach to specifi-cally prevent X-ray contrast agents from enteringwater bodies. Thisis also because of unsatisfactory removal with the proposed newend-of-pipe measures at WWTPs (ozonation, activated carbon).Because X-ray contrast agents are excreted mainly via urine,10

separate collection with roadbags would be fairly easy. In Berlin,mobile urinals were considered feasible.13 However, in ourMCDA, roadbags ranked lower than the status quo(Figure 2A). This is because we had not classified removal ofX-ray contrast agents as a separate objective. Nevertheless, wespeculate that many stakeholders were aware that the highpharmaceutical load in the general hospital is largely due toX-ray contrast agents (Table SI-3B): in the psychiatric hospital,“low load” received only 1/3 of the weight (“low ecotoxicologicalrisk” 2/3), but in the general hospital, both subobjectives hadequal weights if “pathogen-removal” was excluded (Table 3;pp SI-37�SI-38). Also for roadbags a source-to-sink evaluation isneeded concerning superadsorbers and the legitimacy of dispos-ing roadbags via solid household waste (p SI-41). We furthersuggest clarifying user acceptance of roadbags with pilot tests anddesigning social�psychological intervention strategies that in-crease their adoption (e.g., ref 30).Stakeholder Feedback. Receiving feedback concerning the

plausibility of the MCDA results is an integrated part of MCDAmethodology (step 7, Table 1). Additionally, it was an aim of thisstudy to receive feedback concerning the usefulness of theMCDA approach to support decision-making in this complexreal-world setting. The stakeholders participating in the work-shop generally agreed with their own MCDA results (pp SI-41�SI-42). All would welcome implementing that alternativewhich the MCDA produced as first rank for each stakeholder,based on their own preference-input (elicited value functions/weights of objectives). They mostly found MCDA methodologyunderstandable, but laborious. MCDA was judged attractive

3856 dx.doi.org/10.1021/es1031294 |Environ. Sci. Technol. 2011, 45, 3848–3857

Environmental Science & Technology POLICY ANALYSIS

because it includes subjective preferences of different stake-holders and allows discussing objectives rather than alternatives.However, especially wastewater experts were highly critical of thesensitivity ofMCDAmethodology to boundary conditions. Theywould have preferred including communal WWTPs, which wasbeyond the scope of our study. This might result in measures atWWTPs performing better than those at hospitals. Despite thiscriticism, MCDA did clarify collecting X-ray contrast agents withroadbags: several stakeholders (including wastewater authori-ties) welcome roadbag-pilot projects as next step; or generally moredemonstration projects to remove pharmaceuticals in Swiss hospitals.Above all, participants were surprised and very pleased that the

main results were robust across a wide range of stakeholders.Obviously, protecting water from potentially harmful substancesis important to many, even if it costs something. The participantsfound that MCDAmethodology gave a more holistic view to thisimportant issue.Our MCDA results provide deeper insight into a problem

ranking high on the political agenda of national and internationalwater protection bodies. Because the two case-study hospitals aretypical and were confronted with the problem of pharmaceuticalresidues in wastewater for the first time, we believe it is safe togeneralize conclusions also to other European countries: forstakeholders from the water and health sector, knowing thatpharmaceuticals enter water bodies is disturbing, despite largeuncertainty concerning environmental effects. Implementingmea-sures to decrease pharmaceutical loads will likely meet acceptanceby many. Hospital point-sources merit consideration, especially ifthey contribute substantially to the load at WWTPs and if thetrade-off between costs and pharmaceutical removal is acceptable.

’ASSOCIATED CONTENT

bS Supporting Information. Details concerning data, meth-ods, results, discussion, and references are presented in tables,figures, and explanations. This information is available free ofcharge via the Internet at http://pubs.acs.org.

’AUTHOR INFORMATION

Corresponding Author*E-mail: judit.lienert@eawag.ch; Tel:þ41 (0)58 765 55 74; Fax:þ41 (0)58 765 53 75.

’ACKNOWLEDGMENT

We thank Beate I. Escher for ecotoxicological risk predictions,Ruedi Moser and Raphael Br€ugger for cost calculations, PeterReichert for support with MCDA, Adriano Joss, Max Maurer,Wouter Pronk, Hansruedi Siegrist, and Urs von Gunten fordiscussing engineering options, the many interview partners, andthe Kantonsspital Baden as well as the Integrierte PsychiatrieWinterthur in Embrach for their helpful collaboration. We thankthree anonymous reviewers for their very useful comments, andEawag (Discretionary Funds) for financial support.

’REFERENCES

(1) Ankley, G. T.; Brooks, B. W.; Huggett, D. B.; Sumpter, J. P.Repeating history: Pharmaceuticals in the environment. Environ. Sci.Technol. 2007, 41 (24), 8211–8217.(2) Perez, S.; Barcelo, D. Fate and occurrence of X-ray contrast media

in the environment. Anal. Bioanal. Chem. 2007, 387 (4), 1235–1246.

(3) Altenburger, R.; Walter, H.; Grote, M. What contributes to thecombined effect of a complex mixture? Environ Sci. Technol. 2004, 38(23), 6353–6362.

(4) FOEN. Micropollutants in the Aquatic Environment. Assessmentand Reduction of the Pollutant Load Due to Municipal Waste Water;Federal Office for the Environment: Berne, Switzerland, 2009; http://www.bafu.admin.ch/publikationen/publikation/01051/index.html?lan-g=en (accessed 10.06.2010).

(5) Hollender, J.; Zimmermann, S. G.; Koepke, S.; Krauss, M.;McArdell, C. S.; Ort, C.; Singer, H.; von Gunten, U.; Siegrist, H.Elimination of organic micropollutants in a municipal wastewatertreatment plant upgraded with a full-scale post-ozonation followed bysand filtration. Environ. Sci. Technol. 2009, 43 (20), 7862–7869.

(6) Reungoat, J.; Macova, M.; Escher, B. I.; Carswell, S.; Mueller,J. F.; Keller, J. Removal of micropollutants and reduction of biologicalactivity in a full scale reclamation plant using ozonation and activatedcarbon filtration. Water Res. 2010, 44 (2), 625–637.

(7) Guest, J. S.; Skerlos, S. J.; Barnard, J. L.; Beck, M. B.; Daigger,G. T.; Hilger, H.; Jackson, S. J.; Karvazy, K.; Kelly, L.; Macpherson, L.;Mihelcic, J. R.; Pramanik, A.; Raskin, L.; van Loosdrecht, M. C. M.; Yeh,D.; Love, N. G. A new planning and design paradigm to achievesustainable resource recovery from wastewater. Environ. Sci. Technol.2009, 43 (16), 6126–6130.

(8) Larsen, T. A.; Alder, A. C.; Eggen, R. I. L.; Maurer, M.; Lienert, J.Source separation: Will we see a paradigm shift in wastewater handling?Environ Sci. Technol. 2009, 43 (16), 6121–6125.

(9) Agerstrand,M.;Wester,M.; Ruden, C. The Swedish EnvironmentalClassification and Information System for Pharmaceuticals� An empiricalinvestigation of the motivations, intentions and expectations underlying itsdevelopment and implementation. Environ. Int. 2009, 35 (5), 778–786.

(10) Lienert, J.; B€urki, T.; Escher, B. I. Reducing micropollutantswith source control: Substance flow analysis of 212 pharmaceuticals infaeces and urine. Water Sci. Technol. 2007, 56 (5), 87–96.

(11) Lienert, J.; G€udel, K.; Escher, B. I. Screening method forecotoxicological hazard assessment of 42 pharmaceuticals consideringhuman metabolism and excretory routes. Environ. Sci. Technol. 2007, 41(12), 4471–4478.

(12) Lenz, K.; Mahnik, S. N.; Weissenbacher, N.; Mader, R. M.;Krenn, P.; Hann, S.; Koellensperger, G.; Uhl, M.; Knasmuller, S.; Ferk,F.; Bursch, W.; Fuerhacker, M. Monitoring, removal and risk assessmentof cytostatic drugs in hospital wastewater. Water Sci. Technol. 2007, 56(12), 141–149.

(13) Heinzmann, B.; Schwarz, R. J.; Schuster, P.; Pineau, C. Decen-tralized collection of iodinated x-ray contrast media in hospitals-resultsof the feasibility study and the practice test phase. Water Sci. Technol.2008, 57 (2), 209–215.

(14) Beier, S.; Koster, S.; Veltmann, K.; Schroder, H. F.; Pinnekamp,J. Treatment of hospital wastewater effluent by nanofiltration andreverse osmosis. Water Sci. Technol. 2010, 61 (7), 1691–1698.

(15) Verlicchi, P.; Galletti, A.; Masotti, L. Management of hospitalwastewaters: The case of the effluent of a large hospital situated in a smalltown. Water Sci. Technol. 2010, 61 (10), 2507–2519.

(16) Kummerer, K. Resistance in the environment. J. Antimicrob.Chemother. 2004, 54 (2), 311–320.

(17) Langford, K. H.; Thomas, K. V. Determination of pharmaceu-tical compounds in hospital effluents and their contribution to waste-water treatment works. Environ. Int. 2009, 35 (5), 766–770.

(18) Weissbrodt, D.; Kovalova, L.; Pazhepurackel, V.; Ort, C.;Moser, R.; Hollender, J.; Siegrist, H.; McArdell, C. S. Mass flows ofX-ray contrast media and cytostatics in hospital wastewater. Environ. Sci.Technol. 2009, 43, 4810–4817.

(19) Ort, C.; Lawrence, M. G.; Reungoat, J.; Eaglesham, G.; Carter,S.; Keller, J. Determining the fraction of pharmaceutical residues inwastewater originating from a hospital. Water Res. 2010, 44 (2),605–615.

(20) Escher, B. I.; Baumgartner, R.; Koller,M.; Treyer, K.; Lienert, J.;McArdell, C. S. Environmental toxicology and risk assessment ofpharmaceuticals from hospital wastewater. Water Res. 2011, 45, 75–92.

3857 dx.doi.org/10.1021/es1031294 |Environ. Sci. Technol. 2011, 45, 3848–3857

Environmental Science & Technology POLICY ANALYSIS

(21) Keeney, R. L. Decision analysis: An overview. Oper. Res. 1982,30 (5), 803–838.(22) Eisenf€uhr, F.; Weber, M.; Langer, T. Rational Decision Making;

Springer Verlag: Heidelberg, 2010.(23) McDaniels, T.; Longstaff, H.; Dowlatabadi, H. A value-based

framework for risk management decisions involving multiple scales: Asalmon aquaculture example. Environ. Sci. Policy 2006, 9 (5), 423–438.(24) Schuwirth, N.; Reichert, P.; Lienert, J. Multi-criteria decision

analysis for policy support: A case study on pharmaceutical removal fromhospital wastewater. Submitted.(25) McArdell C. S.; Kovalova L.; Eugster J.; Hagenbuch M.;

Wittmer A.; Siegrist H. Elimination of pharmaceuticals from hospitalwastewater in a pilot membrane bioreactor with PAC or ozone post-treatment. Conference Proceeding: SETAC Europe, 20th Annual Meet-ing, Seville, Spain, May 23�27, 2010; http://www.eawag.ch/for-schung/uchem/schwerpunkte/projektuebersicht/projekt2/index_EN.(26) Maurer, M.; Pronk, W.; Larsen, T. A. Treatment processes for

source separated urine. Water Res. 2006, 40 (17), 3151–3166.(27) Snyder, S. A.; Adham, S.; Redding, A. M.; Cannon, F. S.;

DeCarolis, J.; Oppenheimer, J.;Wert, E. C.; Yoon, Y. Role of membranesand activated carbon in the removal of endocrine disruptors andpharmaceuticals. Desalination 2007, 202, 156–181.(28) Benner, J.; Salhi, E.; Ternes, T.; von Gunten, U. Ozonation of

reverse osmosis concentrate: Kinetics and efficiency of beta blockeroxidation. Water Res. 2008, 42 (12), 3003–3012.(29) Lienert, J.; Larsen, T. A. High acceptance of urine source

separation in seven European countries: A review. Environ. Sci. Technol.2010, 44 (2), 556–566.(30) Mosler, H.-J.; Martens, T. Designing environmental campaigns

by using agent-based simulations: Strategies for changing environmentalattitudes. J. Environ. Manage. 2008, 88, 805–816.