Research Collection
Doctoral Thesis
Distributed decision-making under incomplete information
Author(s): Malekovic, Ninoslav
Publication Date: 2016
Permanent Link: https://doi.org/10.3929/ethz-a-010710706
Rights / License: In Copyright - Non-Commercial Use Permitted
This page was generated automatically upon download from the ETH Zurich Research Collection. For moreinformation please consult the Terms of use.
ETH Library
https://doi.org/10.3929/ethz-a-010710706http://rightsstatements.org/page/InC-NC/1.0/https://www.research-collection.ethz.chhttps://www.research-collection.ethz.ch/terms-of-use
DISS. ETH NO. 23346
DISTRIBUTED DECISION-MAKING UNDER INCOMPLETE INFORMATION
A thesis submitted to attain the degree of
DOCTOR OF SCIENCES of ETH ZURICH
(Dr. sc. ETH Zurich)
presented by
NINOSLAV MALEKOVIC
MPA, Harvard University
born on 22. 7. 1976.
citizen of Croatia
accepted on the recommendation of
Prof. Dr. Juliana Sutanto, ETH Zurich, supervisor
Prof. Dr. Stefano Brusoni, ETH Zurich, co-examiner
2016
1
This thesis is dedicated to my loving parents,
Cela and Josip
2
3
Acknowledgments
First, I would like to thank Professor Juliana Sutanto for giving me the opportunity to
work toward my doctoral degree in her group. Supportive, knowledgeable, and kind, she is a
role model academician. Without her I would not have had the opportunity to be here.
My equal gratitude goes to the co-examiner of my doctoral thesis, Professor Stefano
Brusoni. His capacity for terse discernment has already left a timeless impression. His
commitment to knowledge, as revealed by his scholarship will remain inspiring.
My gratitude also goes to other academicians who showed me the way forward on my
path, each in his own authentic way, but all of them by personal magic and example. Most
memorable among them are late Ante Fulgosi the intelligence researcher, late Alija Kulenovic
the quant in cognitive science, both of the Psychology Department of University of Zagreb,
and inimitable Clifford Taubes of Taubes’ Gromov Invariant and Harvard Mathematics
Department.
This thesis would not have been possible without the support of colleagues in the MIS
group and elsewhere. I am grateful to have been able to share my work with Mihai Calin,
Patrizia de Lorenzo, Balint Dioszegi, Basil Hess, and Onur Saglam.
Last but not least, I am grateful to my family, Josip, Cela, Lucija and Hrvoje, as well
as my friends Sari Graff, Suzana Helshani, Rosanna Monteleone, and Vjeran Skurjeni. I thank
them for being there when I needed their support.
4
5
Abstract:
This cumulative thesis investigated collaborative decision-making as an instance of
distributed decision-making. Eighty-two studies were reviewed, six research themes were
identified, and five research directions were proposed to guide further advances. These
directions recommend to examine collaborative decision-making for agents’ shared
knowledge, reasoning limits, spatiotemporal configurations, and collaborative failures.
Moreover, the thesis examined two non-collaborative tendencies that characterize a
collaborative failure: Agents’ regret avoidance behaviors and basic manipulative tendency
were analyzed under incomplete information. Relating to agents’ competing interests, features
of incomplete information influence these non-collaborative tendencies: Uncertain competition
can profile agents for their capacity to decide. Moderated by uncertain product availability, the
uncertain competition can direct agents’ regret avoidance behaviors. Ignorant of competing
interests, agents refrain from manipulating collective decisions. Otherwise, information
aggregation complexity can suppress effectiveness of agents’ manipulation, and authenticate
their disclosures. Reasonable goals can still be attained, even if agents’ collaboration fails.
These findings contribute to the understanding of distributed decision-making.
Keywords: distributed and collaborative decision-making; collaborative failures;
regret avoidance; manipulative tendency; incomplete information; uncertainty; information
asymmetry; information aggregation complexity.
6
7
Zusammenfassung:
Diese kumulative Dissertation untersucht kooperatives Entscheiden als eine Instanz des
verteilten Entscheidens. Um Fortschritte zu fördern, wurden zweiundachtzig Studien
ausgewertet, sechs Forschungsthemen bestimmt und fünf Forschungsrichtungen
vorgeschlagen. Diese Richtungen haben als Ziel, gemeinsames Entscheidungsträgerwissen,
Grenzen ihres logischen Denkens, ihre Raum-Zeit-Konfiguration und scheiternde Kooperation
zu untersuchen. Darüber hinaus, betrachtet die Dissertation zwei nicht-kooperative Tendenzen,
die eine scheiternde Kooperation charakterisieren: Die Bedauernsvermeidung der
Entscheidungsträger und ihre grundlegende Manipulationstendenz wurden unter
unvollständiger Information analysiert. Unvollständige Informationserscheinungen, die sich
auf konkurrierende Interessen beziehen, beeinflussen diese nicht-kooperative Tendenzen:
Ungewisser Wettbewerb kann die Entscheidungsfähigkeit der Entscheidungsträger auswerten.
Der von ungewisser Produktverfügbarkeit moderierte Wettbewerb kann ihre
Bedauernsvermeidung leiten. Über konkurrierende Interessen nicht informierte
Entscheidungsträger sehen davon ab, kollektive Entscheidungen zu manipulieren. Sonst kann
die Komplexität der Informationsaggregation die Effektivität der
Entscheidungsträgermanipulation abschwächen und ihre Information als echt beweisen.
Entsprechende Ziele können immer noch erreicht werden, auch wenn die
Entscheidungsträgerkooperation scheitert. Diese Ergebnisse tragen zum Wissen des verteilten
Entscheidens bei.
Schlüsselwörter: verteiltes und kooperatives Entscheiden; scheiternde Kooperation;
das Bedauern; die Manipulation; unvollständige Information; Ungewissheit;
Informationsasymmetrie; die Komplexität der Informationsaggregation.
8
Table of Contents
Acknowledgments 3 Abstract
5
Zusammenfassung
7
Table of contents
8
Introduction
10
Three Complementary Studies
12
Collaborative Decision-Making
14
Avoiding Regret in Online Auctions
16
Manipulating Distributed Decision-Making
19
Conclusion
21
References
22
Appendices
29
9
10
1. Introduction
Cutting across decision, computing, and economic sciences, distributed decision-
making is a rapidly developing branch of general decision theory. “Large organizations are
split up into various divisions, and complex decision problems are separated into more
tractable components. Capable of handling and communicating ever increasing amounts of
data, these decomposed units no longer need be treated as being almost isolated but may also
be coordinated more closely” ([68], p. 1).
Solving a decision problem by efficiently separating and coordinating agents’ decisions
[67], distributed decision-making can be regarded as a means of reducing organizing and
computing complexity [68]. Diverse approaches to distributed decision-making have been
developed, including agency-based approaches, voting methods, auction designs, distributed
artificial intelligence [68] etc. “Multi-agent systems, networking and principal-agent theory
point towards these modern developments” ([68], p. 1).
Dependence of distributed decision-making on agents’ understanding of a decision
problem can make distributed decision-making collaborative [59]. Collaborative decision-
making “makes explicit the aggregation of individuals’ understandings of the frame of a
decision to be made, the alternatives to be considered, the sources of value and risk, and,
finally, the reasons for a resulting collaborative choice” ([59], p. 29). Under collaboration,
agents’ shared knowledge is the organizing principle behind efficiency of agents’ decisions
[18, 32]. If a decision problem requires collaboration, then this principle makes collaborative
decision-making “significantly more valuable… than alternatives envisioned by decision-
makers” ([59], p. 1). However, according to our review of collaborative decision-making
research, collaborative failures remain less well understood.
Characterized as agents’ lack of shared knowledge about a social or strategic interaction
[35, 58], incomplete information reveals a possibility for agents’ collaboration to fail. In
conditions of incomplete information, distributed decisions may have to be made under
uncertainty or information asymmetry. Any information made available to agents may
aggregate in complex ways with information that is already available to them [11, 15, 27].
Distributed decisions may then have to be made under variable information aggregation
complexity. Single-variable or multivariable uncertainty, information asymmetry, and
information aggregation complexity are but complementary features of incomplete
information.
11
This cumulative thesis examined an agents’ collaborative failure for effects of
incomplete information. Our research questions focus on non-collaborative behaviors
pertaining to an agents’ collaborative failure: What are some such behaviors? How do such
behaviors depend on incomplete information? If collaboration fails, can agents’ decisions
benefit from different features of incomplete information? Before answering these questions,
we discuss the logic of our research efforts.
12
2. Three Complementary Studies
Responding to the above questions, the thesis reviewed the research of collaborative
decision-making since the late 1980-s. Collaborative decision-making “considers alternative
understandings of a problem, competing interests, priorities and constraints” ([43], p. 1). If
distributed decision-making emerges from agents’ communications, it accounts for agents’
competing interests and other concerns. In such conditions, distributed decision-making is
more likely to be collaborative. Otherwise, enforcing a model of distributed decision-making
onto agents is more likely to lead to their collaborative failure [67].
Our review has been submitted to Theory and Decision (Please see Appendix 1). The
review also informed our choice of non-collaborative behaviors that characterize collaborative
failures: Regret avoidance and manipulative behaviors can be elicited, by respectively
enforcing auctioning and voting onto a collection of agents. These two models are
complementary: While the former exemplifies non-collective decision-making, the latter
exemplifies collective decision-making. In addition, their enforcement complements
emergence of collaborative options, characterizing collaborative decision-making. In effect,
their enforcement frames non-collaborative behaviors characterizing a collaborative failure.
Thus, auctioning and voting were chosen for further analyses.
Two empirical studies follow the reviewed research. The first study examined agents’
regret avoidance behaviors for competition uncertainty and product availability uncertainty.
Auctioning was enforced through a website accessible by mobile phones. The article has been
submitted to Information Systems Research (Please see Appendix 2). The second study
examined agents’ manipulative imputation for information asymmetry and aggregation
complexity. Voting was enforced and implemented in a computer lab. The article was
published in Decision Support Systems (Please see Appendix 3). Table 1 summarizes the three
studies, and includes the author’s contribution.
13
Table 1 Summary of the three studies
Title Author and co-authors Author’s contributions Status
Collaborative decision-making: Literature review and research directions
Author:
Ninoslav Malekovic
Co-author:
Prof. Juliana Sutanto
Conceptual positioning
Search strategy
Literature Review
Identification of research directions
Manuscript
Submitted to Theory and Decision
Avoiding regret in online auctions: The effects of uncertain competition and uncertain product availability
Author:
Ninoslav Malekovic
Co-authors:
Dr. Lazaros Goutas
Prof. Juliana Sutanto
Conceptual positioning
Data analysis
Manuscript
Submitted to Information Systems Research
Manipulative imputation in distributed decision support settings: The implications of information asymmetry and aggregation complexity
Author:
Ninoslav Malekovic
Co-authors:
Dr. Lazaros Goutas
Prof. Juliana Sutanto
Prof. Dennis Galletta
Research design
Conceptual positioning
Literature review
Data collection and analysis
Manuscript
Published in Decision Support Systems:
Malekovic, N. Sutanto, J. Goutas, L. (2016), Decision Support Systems 85, 1–11
14
3. Collaborative Decision-Making
Our review selected and examined fifty journal and thirty-two conference articles.
Technological efforts have driven the research of collaborative decision-making since the late
1980-s (e.g., [1, 16, 22, 39]). Agent-based modeling prevailed among design research and other
technological efforts (e.g., [2, 5, 39, 41, 55]).
Our review classified these articles into six research themes. The themes range from
collaborative enterprises and conflict resolution, to collaborative reasoning and knowledge
organization, to complexity and spatiotemporal issues in collaborative decision-making. Table
2 presents the identified themes.
One research theme features standardized and emergent protocols for collaborative
enterprises (e.g., [24, 25, 26, 76, 80, 81]). Another theme pertains to agents’ conflicting
tendencies and methods to resolve them (e.g., [62, 72]). The third theme deals with reasoning
that integrates knowledge into collaborative decision-making (e.g., [43, 44]). The most
valuable effects in collaborative decision-making occur between decision-making and
knowledge-based processes [32]. “Collaborative decision-making is a core organizational
activity that comprises a series of knowledge representation and processing tasks” ([32], p. 1).
Thus, the fourth theme delves into knowledge-dependence of collaborative options [18, 19,
32]. The fifth theme focuses on collaborative complexity that can be traced to the nature of
decision problems, the number of decision criteria and the multiplicity of agents (e.g., [2, 75]).
Finally, the sixth theme is about spatiotemporal effects, and often inconclusive, reverse-causal
issues to which they can give rise (e.g., [12, 42, 57, 63]).
Our proposed research directions motivate theoretical advances in collaborative
decision-making. First, emergence of a collectively owned hybrid option from agents’ shared
knowledge has to be made a central theoretical consideration in collaborative decision-making
[59]. Conditions of such emergence have to be identified. Second, as specified by
Table 2 The identified themes
1. Collaborative enterprises 2. Conflict resolution 3. Collaborative reasoning
Extant protocols support agents’ collaborative enterprises.
If agents do not resolve conflicts, opinion factions may polarize.
Argumentation integrates agents’ knowledge in collaborative
decisions.
4. Organization of knowledge 5. Collaborative complexity 6. Spatiotemporal challenges
Agents represent shared knowledge, using ontologies.
Complexity can be traced to decision problems, criteria, and
agent multiplicity.
Agents’ spatiotemporal configurations play into collaborative decisions.
15
argumentation logics, agents’ reasoning limits bound this emergence. Therefore, implications
of reasoning limits (e.g., [14, 79]) for emergent, collaborative options have to be carefully
examined. Third, agents’ spatiotemporal configurations also bound such options. Hence,
dependence of such options on spatiotemporally configured agents has to be cogently
explained. Fourth, collaborative failures have to be profiled for mixtures of agents’ non-
collaborative behaviors. Fifth, all of these considerations have to be designed into architecture
of collaborative enterprises.
Agents’ collaborative failures are less well understood than efficiency of agents’
decisions. Such failures can be gleaned from existing decision-analytic accounts. “In
collaborative decision-making, we do not strive for an optimum, a compromise, or a satisficing
solution” ([59], p. 1). In fact, these approaches fall short of collaborative decision-making, and
exemplify failed collaboration.
Existing accounts inform our analyses. Manipulation is a symptom of a collaborative
failure [36, 69]. It includes deceptive, strategic, and rigging behaviors [6, 34, 36, 52, 53, 54].
An instance of optimization, regret avoidance behaviors also characterize collaborative failures
[3, 7, 10, 38, 51, 73, 74]. Thus, manipulative and regret avoidance behaviors were chosen for
a closer scrutiny.
16
4. Avoiding Regret in Online Auctions
Our quasi-experimental study of online auctions refers to bidders as agents. Each agent
considers whether to bid more at a risk of overpaying for a product, or to bid less, at a risk of
failing to obtain it. While the former tendency leads agents to regret winning, the latter leads
agents to regret losing. Our study examined agents’ simplest regret avoidance behaviors:
Avoidance of winner regret by underbidding and avoidance of loser regret by overbidding. The
study analyzed these behaviors for uncertainty effects, as derived from signaling theory.
No regret avoidance behavior has been empirically observed in sealed-bid auctions, i.e.,
sporadic observations were attributed to flawed methods (e.g., [21, 33, 45]). Our study
complemented the earlier findings, by building on specific strategic equivalences between
sealed-bid and open-bid auctions. Second-price sealed-bid and open ascending-bid auctions
are strategically equivalent [48]. Similarly, first-price sealed-bid and open descending-bid
auctions are strategically equivalent [48]. Drawing inferences from these equivalences, our
study examined agents’ regret avoidance behaviors in open-bid auctions. Adapted from [48],
Figure 1 depicts the logic of our contribution.
FIRST-PRICE SEALED-BID
AUCTION
OPEN DESCENDING-BID AUCTION
For independent private values, due to the strategic equivalence, these auctions should lead to
the same price. However, they were empirically found to result in different prices.
Bidders’ regret avoidance behaviors can explain the empirically different prices.
SECOND-PRICE SEALED-BID
AUCTION
OPEN ASCENDING-BID AUCTION
Our study compares agents’ regret avoidance behaviors between descending and open ascending-bid auctions. The study answers whether a non-disclosure of the second highest bid indeed guards bidders against regret.
There can be no winner regret in second-price sealed bid auctions because they mechanically guard bidders against winner regret. First-price sealed-bid auctions do not have this mechanical feature. Because no regret avoidance behaviors have been observed in first-price sealed bid auctions, there must be some regret in first-price sealed bid auctions.
Figure 1 The logic of our contribution
Due to the strategic equivalence, these auctions
should lead to the same price. However, they were empirically found to result in different prices.
Bidders’ regret avoidance behaviors do not explain the empirically different prices.
17
Agents’ regret avoidance behaviors in open-bid auctions were analyzed for competition
uncertainty and product availability uncertainty. Characterizing a difference in bid disclosures
between open ascending-bid and open descending-bid auctions, competition uncertainty refers
to an agent’s difficulty in assessing competition intensity. Product availability uncertainty
refers to an agent’s difficulty in assessing an adequate product availability. Figure 2 depicts
our research model.
Figure 2 The Research Model
Several findings were confirmed. The more uncertain the competition, the less likely is
the focal agent to avoid any regret (H1). While uncertain product availability negatively
moderates the effect of uncertain competition on the focal agent’s avoidance of winner regret
(H2a), the opposite holds true for the focal agent’s avoidance of loser regret (H2b).
Agents’ regret avoidance behaviors extract information from competition and product
availability. Regret is suggested to depend on a disclosure of two highest bids: The earlier
studies suggested that a non-disclosure of these bids would weaken agents’ regret [21, 28, 29,
30, 31, 33]. By comparing open ascending-bid and open descending-bid auctions, our study
cogently explained this consideration: Not only do agents measure regret relative to disclosed
competing bids, but they also learn from disclosed competing bids how to avoid it. By
confirming this, our study concluded the analyses of regret in auctions.
Apart from auction theory, our study contributed to signaling theory. “When
information asymmetries exist, signaling theory suggests that the interacting parties send
signals to one another in order to adjust their purchasing behaviors…” ([23], p. 37). “Signals
are particularly important in online auctions where uncertainty exists in the product, seller,
and other bidders” ([23], p. 37). By pointing out that signaling can also be influenced by
availability of products [20, 40], our study contributed to the understanding of
multidimensionality of a signaling medium.
Finally, our study of regret avoidance under uncertain competition may be extended.
Agents can stifle competition, by collusively refraining from placing bids [65]. An auctioneer
Avoidance of Winner and Loser
Regret
Availability Uncertainty
Competition Uncertainty
H1(-)
H2a(-)H2b(+)
18
can encourage competition, by placing false bids [65]. Rigging behaviors can also lead to non-
clearing market prices, e.g., bubbles and crises [4]. A study of regret in conditions of rigging
can motivate further research of regret avoidance under uncertain competition.
19
5. Manipulating Distributed Decision-Making
Our experimental study examined an agents’ basic manipulative tendency in distributed
decision-making. Supported by distributed communications, agents can attempt to manipulate
collective decisions [8, 9, 52, 53, 54]. Nonetheless, effectiveness of such behaviors is unclear
[8, 9, 52]. Thus, our study analyzed both incidence and effectiveness of agents’ manipulative
imputation. Figure 3 depicts our research model.
Figure 3 The research model with main, moderating, control, and dependent variables
Agents’ manipulative imputation was analyzed for information asymmetry and
information aggregation complexity, as derived from collective choice [6, 36] and information
aggregation theories [11, 15]. Several findings were confirmed: Providing information on
competing interests increases the incidence of the focal agent’s manipulative imputation (H1a).
Given the simplest information aggregation rule, doing so further increases the focal agent’s
effectiveness (H1b). Complexity of an aggregation rule decreases the incidence and
effectiveness of the focal agent’s manipulative imputation (H2).
Our findings contributed to our theoretical framework. This framework explains
distributed decision-making “through principles of minimization of uncertainty or
maximization of predictability under information pooling over many individuals” ([11], p.
598). It has already been known that agents inaccurately disclose information [17]. In non-
Manipulative Tendency &
Effectiveness of Manipulative Imputation
Complexity of a Decision Rule
Information Asymmetry
H1(+)
H2(-)
Control Variables
Social Value Orientation Age
Familiarity Score
Moderating Effect
Main Effect
Outcomes
Gender
Setting Types Decision Scenarios
20
collective settings, agents under information asymmetry distort information by trading [78],
and information aggregation efficiency decreases this tendency [78]. In collective settings,
agents under information asymmetry distort information by manipulating collective outcomes.
However, it is information aggregation complexity that decreases this tendency, and
authenticates agents’ information disclosures.
By allowing for more complex manipulations, our analysis can be extended. By
proposing dependent options that cannot be chosen, agents may manipulate collective decisions
[34, 49, 60]. By forming coalitions and rigging agendas, agents may do so, too [13, 34, 46, 50,
64, 77]. Agents’ bounded rationality and heuristics [37, 70, 71] may also play a role in these
issues. Such effects await further scrutiny.
21
6. Conclusion
The cumulative thesis contributes to the understanding of distributed decision-making.
Incomplete information can alleviate symptoms of an agents’ collaborative failure. Uncertain
competition can profile agents for their capacity to decide. Moderated by uncertain product
availability, uncertain competition can direct an agents’ non-collaborative tendency. Ignorant
of competing interests, agents refrain from manipulating collective decisions. Otherwise,
information aggregation complexity can decrease this tendency, and authenticate agents’
information disclosures. Reasonable goals can still be attained, even if agents fail to
collaborate.
22
References
[1] A. Adla, Hybrid reasoning-based system for collaborative decision- making, International
Journal of Reasoning-based Intelligent Systems 3, 205 – 211.
[2] B.M. Adler, W. Baets, R. König, Complexity perspective on collaborative decision making
in organizations: The ecology of group performance, Information and Management 48, 2011,
157-165.
[3] H. Aissi, Approximation and resolution of min-max and min-max regret versions of
combinatorial optimization problems, A Quarterly Journal of Operations Research 4, 2006,
347-350.
[4] F. Allen, and D. Gale, Bubbles and crises, The Economic Journal 110, 2000, 236-255.
[5] M. Al-Shawa, Modeling and analyzing agents’ collective options in collaborative decision
making, Brain Informatics, Lecture Notes in Computer Science 6889, 2011, 111-123.
[6] K.J Arrow, Social choice and individual values, 1951, Yale University Press: New Haven,
CT.
[7] I. Averbakh, Minimax regret solutions for minimax optimization problems with
uncertainty, Operations Research Letters 27, 2000, 57-65.
[8] R. Barkhi, V.S. Jacob, V.S., H. Pirkul, The influence of communication mode and incentive
structure on GDSS process and outcomes, Decision Support Systems 37, 2004, 287-305.
[9] R. Barkhi, V.S. Jacob, L. Pipino, H. Pirkul, A study of the effect of communication channel
and authority on group decision processes and outcomes, Decision Support Systems 23, 1998,
205-226.
[10] D.E Bell, D. E. Regret in decision making under uncertainty, Operations Research 30,
1982, 961-981.
[11] L.M.A. Bettencourt, The rules of information aggregation and emergence of collective
intelligent behavior, Topics in Cognitive Science 1, 2009, 598–620.
[12] W.E. Berzins and M.D Dhavala, Time versus trust: impact upon collaborative decision-
making, Journal of Management in Engineering 4, 1998, 320-324.
[13] C. Burnett, T.J. Norman, K. Sycara, N., Oren, Supporting trust assessment and decision
making in coalitions, IEEE Intelligent Systems 29, 2014, 18-24.
23
[14] G. Chaitin, Limits of reason, Scientific American 294(3), 2006, 74-81.
[15] C. Chambers and A. D. Miller, Rules for aggregating information, Social Choice and
Welfare 36, 2011, 75-82.
[16] A.J. Chapman, and J.G. Pohl, Collaborative decision support systems for facility
management, Proceedings of InterSymp-1998: The 10th International Conference on Systems
Research, Informatics and Cybernetics, 71-79.
[17] L. Chen, J. R. Marsden, Z. Zhang, Reliability (or lack thereof) of on-line preference
revelation, a controlled experimental analysis, Decision Support Systems 56, 2013, 270–274.
[18] A.R. Dennis, J.A. Rennecker, S. Hansen, Invisible whispering: restructuring collaborative
decision making with instant messaging, Decision Sciences 41, 2010 845–886.
[19] A.V. Deokar, O.F. El-Gayar, N. Taskin, R. Aljafari, An ontology-based approach for
model representation, sharing and reuse, 14th Americas Conference on Information Systems 5,
2008, 3194-3202.
[20] A. Dimoka, Y. Hong, P.A. Pavlou, On product uncertainty in online markets: Theory and
evidence,” MIS Quarterly 36(2), 2012, 395-426.
[21] A. Dodonova, Y. Khoroshilov, Behavioral biases in auctions: An experimental study,
Economics Bulletin 29, 2009, 2218-2226.
[22] M.V. D’Ortenzio, F.Y. Enomoto, S.L. Johan, Collaborative decision environment for
UAV operations, Collection of Technical Papers - InfoTech at Aerospace: Advancing
Contemporary Aerospace Technologies and Their Integration, 2005, American Institute for
Aeronautics and Astronautics, NASA Ames Research Center.
[23] J.R. Drake, D.J. Hall, C. Cegielski, T.A. Byrd, An exploratory look at early online auction
decisions: Extending signal theory, Journal of Theoretical and Applied Electronic Commerce
Research 10, 2015, 35-48.
[24] M.V. Drissen-Silva, R.J. Rabelo, A model for dynamic generation of collaborative
decision protocols for managing the evolution of virtual enterprises, Proceedings of the 8th
IFIP International Conference on Information Technology for Balance Automation Systems,
2008, 105-114.
[25] M.V. Drissen-Silva, R.J. Rabelo, Managing decisions on changes in the virtual enterprise
evolution, In P. L. M. Camarinha-Matos et al. (Eds.), Proceedings PRO-VE - Leveraging
knowledge for innovation in collaborative networks, 2009, 463-475.
24
[26] M.V. Drissen-Silva, R.J. Rabelo, A collaborative decision support framework for
managing the evolution of virtual enterprises, International Journal of Production Research
47(17), 2012, 4833-4854.
[27] U. Endriss, U. Grandi and D. Porello, Complexity of judgment aggregation, Journal of
Artificial Intelligence Research 45, 2012, 481–514.
[28] R. Engelbrecht-Wiggans, The Effect of Regret on Optimal Bidding in Auctions,
Management Science 35(6), 1989, 685-692.
[29] R. Engelbrecht-Wiggans, E. Katok, Regret in auctions: theory and evidence, Economic
Theory: Symposium on Behavioral Game Theory 33(1), 2006, 81-101.
[30] R. Engelbrecht-Wiggans, E. Katok, Regret and Feedback Information in First-Price
Sealed-Bid Auctions, Management Science 54(4), 2008, 808-819.
[31] R. Engelbrecht-Wiggans, E. Katok, A Direct Test of Risk Aversion and Regret in First
Price Sealed-Bid Auctions, Decision Analysis 6(2), 2008, 75– 86
[32] C.E. Evangelou, N. Karacapilidis, A multidisciplinary approach for supporting
knowledge-based decision-making in collaborative settings, International Journal of Artificial
Intelligence Tools 16, 2007, 1069.
[33] E. Filiz-Ozbay, E.Y. Ozbay, Auctions with anticipated regret: Theory and experiment,
The American Economic Review 97(4), 2007, 1407-1418
[34] S. French, Web-enabled strategic GDSS, e-democracy and Arrow's theorem: A Bayesian
perspective, Decision Support Systems 43, 2007, 1476- 1484.
[35] A. Friedenberg, M. Meier, The context of the game, Economic Theory, 2015, 1-40
[36] A. Gibbard, Manipulation of voting schemes: A general result, Econometrica 41 1973,
587–601.
[37] G. Gigerenzer and D. Goldstein, Reasoning the fast and frugal way: Models of bounded
rationality, Cognitive Science 103, 1996, 650-666.
[38] P. Guo, P. One-shot decision theory: A fundamental alternative for decision under
uncertainty, Studies in Computational Intelligence 502, 2014, 33-55.
[39] A. Hamel, S. Pinson, M. Picard, A new approach to agency in a collaborative decision-
making process, IEEE/WIC/ACM International Conference on Intelligent Agent Technology,
2005, 273 – 276.
25
[40] Y. Hong, P.A. Pavlou, Product Fit Uncertainty: Nature, Effects, and Antecedents,
Information Systems Research 25(2), 2014, 328-344.
[41] M. Indiramma, K.R Anandakumar, Collaborative decision-making framework for multi-
agent system, Computer and Communication Engineering 11, 2008, 40 – 46.
[42] P. Jankowski and T.L. Nyerges, GIS-supported collaborative decision- making: Results
of an experiment, Annals of the Association of American Geographers 91, 2001, 48–70.
[43] N. Karacapilidis, D. Papadias, C. Pappis, Computer-mediated collaborative decision
making: Theoretical and implementation issues, Proceedings of the 32nd Hawaii International
Conference on System Sciences, 1999.
[44] N.M. Karacapilidis, D. Papadias, Computer supported argumentation and collaborative
decision-making: the HERMES system, Information Systems 26(4), 2001, 259–277.
[45] P. Katuscak, F. Michelucci, M. Zajicek, Does feedback really matter in one-shot first-
price auctions? Journal of Economic Behavior and Organization 119, 2015, 139-152.
[46] D.M. Kilgour, K.W. Hipel, X. Peng, L. Fang, Coalition analysis in group decision support,
Group Decision and Negotiation 10, 2001, 159-175.
[47] G.L. Klein, J.L. Drury, M.S. Pfaff, CO-action: Collaborative option awareness impact on
collaborative decision making,” 2011 IEEE International Multi-Disciplinary Conference on
Cognitive Methods in Situation Awareness and Decision Support, 2011, 171 – 174.
[48] V. Krishna, Auction Theory, 2009, 2nd ed., San Diego, CA: Academic Press.
[49] H.E. Landemore, Democratic reason: The mechanisms of collective intelligence in
politics, In Collective Wisdom: Principles and Mechanisms, Hélène Landemore and Jon Elster,
eds., 2012, Cambridge University Press, Cambridge.
[50] K.W. Li, T. Inohara, H. Xu, Coalition analysis with preference uncertainty in group
decision support, Applied Mathematics and Computation 231, 2014, 307-319.
[51] G. Loomes and R. Sugden, Regret theory: An alternative theory of rational choice under
uncertainty, Economic Journal 92, 1982, 805–24.
[52] K. Marett and J.F. George, Barriers to deceiving other group members in virtual settings,
Group Decision and Negotiation 22, 2013, 89–115.
[53] O. Meddeb, F.B. Abdelaziz, J.R. Figueira, On the manipulability of the fuzzy social choice
functions, Fuzzy Sets and Systems 159, 2008, 177–184.
26
[54] O. Meddeb, F.B. Abdelaziz, J.R. Figueira, Generalized manipulability of fuzzy social
choice functions, Journal of Intelligent & Fuzzy Systems: Applications in Engineering and
Technology 26, 2014, 253-257.
[55] B. Nachet, and A. Adla, An agent-based distributed collaborative decision support system,
Intelligent Decision Technologies 8, 2014, 15-34.
[56] D.E. O’Leary, User participation in a corporate prediction market, Decision Support
Systems 78, 2015, 28-38.
[57] M.B. O’Leary, J.N. Cummings, The spatial, temporal, and configurational characteristics
of geographic dispersion in teams, MIS Quarterly 31, 2007, 433-452.
[58] M. Oliu-Barton, Differential Games with Asymmetric and Correlated Information,
Dynamic Games and Applications 5(3), 2015, 378-396
[59] D. Owen, Collaborative decision-making, Decision Analysis 12, 2015, 29–45.
[60] R. Paramesh, Independence of irrelevant alternatives, Econometrica 41, 1973, 987-991.
[61] G. Phillips-Wren, E. Hahn, G. Forgionne, Consensus-building in collaborative decision-
making, Collaborative Decision Making: Perspectives and Challenges - Frontiers in Artificial
Intelligence and Applications 176, 2008, 221-230.
[62] K. Ramsey, GIS, modeling, and politics: On the tensions of collaborative decision support,
Journal of Environmental Management 90, 2009, 1972–1980.
[63] M. Raubal, S. Winter, A spatio-temporal model towards ad-hoc collaborative decision-
making, Geospatial Thinking, Lecture Notes in Geo- information and Cartography 0, 2010,
279-297.
[64] D. Ray and R. Vohra, Coalition formation, In Peyton Y. & Shmuel Z., Handbook of Game
Theory (4th ed.), 2014, North-Holland.
[65] A.E. Roth and M.A. Oliveira Sotomayor, Two-sided matching: A study in game-theoretic
modeling and analysis, first ed., 1990, Cambridge University Press; Cambridge.
[66] L.I. Rusu, W. Rahayu, T. Torabi, F. Puersch, W. Coronado, A.T. Harris, K. Reed, Moving
towards a collaborative decision support system for aeronautical data, Journal of Intelligent
Manufacturing 23, 2012, 2085-2100.
[67] C. Schneeweiss, Hierarchies in distributed decision-making, 1999, Springer-Verlag Berlin
Heidelberg.
27
[68] C. Schneeweiss, Distributed decision making--a unified approach, European Journal of
Operational Research 150, 2003, 237-252.
[69] N. Selvaraj and B. Fields, Rethinking collaborative decision-making across distributed
work communities in complex work settings, Proceedings of the 30th European Conference on
Cognitive Ergonomics, 2012, 8-14.
[70] H.A. Simon, Theories of bounded rationality, In McGuire C. B. & Radner, R. eds.,
Decision and Organization, 1972, 161-176, Amsterdam: North- Holland Publishing Company.
[71] H.A. Simon, Models of bounded rationality, 1997, Cambridge, MA: MIT Press.
[72] W.W. Smari, K. Weigand, G. Petonito, Y. Kantamani, R. Madala, S. Donepudi, An
integrated approach to collaborative decision making using computer-supported conflict
management methodology, International Conference on Information Reuse and Integration,
2005, 182-191.
[73] J. Stoye, Minimax regret treatment choice with finite samples, Journal of Econometrics
151, 2009, 70-81.
[74] J. Stoye, Statistical decisions under ambiguity, Theory and Decision 70, 2011 129-148.
[75] L. Susskind, Complexity science and collaborative decision-making, Negotiation Journal
26(3), 2010, 367–370.
[76] P. Wang, W. Zhong, F. Lu, Collaborative ordering of enterprises with differentiated
products under the protection of sensitive information, EBISS International Conference on E-
Business and Information System Security, 2009, 1-3.
[77] H.A.M. Wilke, Coalition formation from a socio-psychological perspective, Advances in
Psychology 24, 1985, 115-171.
[78] S. Yang, T. Li, E. van Heck, Information transparency in prediction markets, Decision
Support Systems 78, 2015, 67–79.
[79] N.S. Yanofsky, The outer limits of reason: What science, mathematics, and logic cannot
tell us, 2013, Cambridge: MIT Press.
[80] S.W. Yoon, S.Y. Nof, Demand and capacity sharing decisions and protocols in a
collaborative network of enterprises, Decision Support Systems 49(4), 2010, 442-450.
[81] S.W. Yoon, S.Y. Nof, Affiliation/dissociation decision models in demand and capacity
sharing collaborative network, International Journal of Production Economics 130(2), 2011,
28
135-143.
29
APPENDICES
30
APPENDIX 1
COLLABORATIVE DECISION-MAKING: LITERATURE REVIEW AND
RESEARCH DIRECTIONS
Abstract:
Collaborative decision-making is a less developed branch of decision theory. Having
outlined key properties of collaborative decision-making, we selected eighty-two articles on
this topic. We classified the articles into six themes. The research themes range from
collaborative enterprises to conflict resolution, to argumentation and organization of
knowledge, to complexity and spatiotemporal issues in collaborative decision-making. Our
proposed directions motivate further advances in this line of research. First, we propose to
identify problem-solving conditions in which a collectively owned hybrid option emerges from
agents’ shared knowledge. Second, we propose to examine how agents’ reasoning limits bound
this emergence. Third, we propose to examine how spatiotemporally configured agents bound
this emergence. Fourth, we propose to profile collaborative failures for mixtures of agents’
non-collaborative behaviors. Fifth, we propose to design all of these considerations into
architecture of collaborative enterprises. Our five directions can guide collaborative decision-
making towards a cogent theory. They can also stimulate related empirical advances.
Keywords: collaborative decision-making; literature review; research directions.
1. Introduction
Ill-structured problems are common in healthcare [8, 24, 43, 54, 70, 103],
engineering [30, 38, 52, 64, 67, 79, 80, 102, 104, 109], and public policy [6, 7, 18, 51, 56, 59,
62, 69, 77, 86, 96, 100, 125]. Solutions to such problems depend on decision-makers’
understanding [1, 39, 40, 41, 49, 95]. Aggregation of decision-makers’ understanding makes
collaborative choice significantly more valuable than alternatives envisioned by any decision-
maker [89]. Nonetheless, collaborative decision-making remains a less developed branch of
decision theory [48, 84, 89].
Strengthening the understanding of collaborative decision-making, we reviewed the
available research in scholarly databases. The selected articles largely come from two
approaches: The socio-cognitive approach is intended to overcome bounds on agents’
information-processing [3, 5, 55, 61, 82, 85, 90, 130]. The design research approach proposed
ways to formalize, manage, and integrate knowledge [39, 40, 41, 44, 49, 66, 105, 132], analyze,
31
assess and support arguments [11, 12, 63], estimate polarized opinion factions [13, 14, 15], and
build consensus [21, 91]. For collaborative decision-making, a collectively owned hybrid
option has to emerge from decision-makers’ shared knowledge [89]. However, decision-
analytic accounts of this emergence have yet to be proposed.
In response, we give five research directions. First, we propose to examine conditions
in which problem solving requires a collectively owned hybrid option to emerge from agents’
shared knowledge. Second, we propose to examine collaborative decision-making for
implications of agents’ reasoning limits. Third, we propose to examine collaborative decision-
making for effects of agents’ spatiotemporal configurations. Fourth, we propose to study the
nature of agents’ collaborative failures. Fifth, we propose to embed all of these considerations
into collaborative enterprises. Our research directions are intended to motivate theoretical
advances in collaborative decision-making.
The rest of the paper is organized as follows: The next section discusses key properties
of collaborative decision-making. The subsequent two sections explain our article search
strategy and summarize the identified research themes. The final section proposes research
directions, and concludes the paper.
2. Definitions
An early definition of collaborative decision-making was socio-cognitive [85, 90, 116].
“To decide effectively, agents need the ability to represent and maintain a model of their own
mental attitudes, reason about other agents' mental attitudes, and influence other agents'
mental attitudes” ([90], p. 107). This process ranges from identification of a problem and group
generation, to social practical reasoning and negotiation [90].
The design research definition treats argumentation as a key property of collaborative
decision-making [66]. Under this definition, collaborative choice is intended to “efficiently
capture users’ rationale, stimulate knowledge elicitation and argumentation on the issues
under consideration, while constantly checking for inconsistencies among users’ preferences
and considering the whole set of the argumentation items asserted to update the discourse
status” ([66], p. 1). This definition “considers alternative understandings of a problem,
competing interests, priorities and constraints” ([63], p. 1).
The decision-analytic definition of collaborative decision-making makes “… explicit
the aggregation of individuals’ understandings of the frame of the decision to be made, the
alternatives to be considered, the sources of value and risk, and, finally, the reasons for the
32
resulting collaborative choice” ([89], p. 29). This definition makes “collaborative connection”
central to collaborative decision-making ([89], p. 40). Combining valuable elements of extant
alternatives, agents connect to an emerging, collectively owned “hybrid option” ([89], p. 34).
A rapid understanding of collaborative options depends on agents’ “combined decision
space” ([71], p. 172). While efficiently separated and coordinated decisions motivate
distributed decision-making [81, 107, 108], dependence of options on agents’ shared
knowledge makes distributed decision-making collaborative [41].
3. Literature on Collaborative Decision-Making
On December 3rd, 2015, we searched for articles with “collaborative choice” or
“collaborative decision” appearing in titles, abstracts, or keywords. We searched for such
articles in Scopus, ProQuest, Emerald, Wiley Interscience, Google Scholar, and the Social
Science Research Network. Our selection excluded articles marginally related to the above
definitions. Included were fifty journal and thirty-two conference articles since 1988. The
research objective, theory, method, and key findings for each article are listed in Appendix A.
Table 1 presents the identified themes.
Table 2 outlines the articles relative to the research themes and theories. Only thirty-
nine out of eighty-two studies employed existing theoretical frameworks (e.g., [21, 39, 40, 55,
92]). Only four studies proposed new theoretical advances [48, 84, 89, 90].
Table 1 The identified themes
1. Collaborative enterprises 2. Conflict resolution 3. Collaborative reasoning
Extant protocols support agents’ collaborative enterprises.
If agents do not resolve conflicts, opinion factions may polarize.
Argumentation integrates agents’ knowledge in collaborative
decisions.
4. Organization of knowledge 5. Collaborative complexity 6. Spatiotemporal challenges
Agents represent shared knowledge, using ontologies.
Complexity can be traced to the nature of decision problems and criteria and agent multiplicity.
Agents’ spatiotemporal configurations play into collaborative decisions.
33
Table 2 The theme-theory classification1
THEORIES
Social Science Theories
Formal Theories
THEMES
Collaborative Enterprises 3 2
Conflict Resolution 2 2
Collaborative Reasoning 3 6
Organization of Knowledge 3 3
Collaborative Complexity 4 5
Spatiotemporal Challenges 6 0
Table 3 outlines the articles relative to the themes and methods. Appended case studies
provided some empirical validation of prevailing design and modeling efforts (e.g., [21, 101,
105]). However, proper experimental findings were scarce (e.g., [57, 86, 114]).
Table 3 The theme-method classification23
METHODS
Modeling & Simulation
Design Research Experiments & Quasi-
experiments
Surveys Cases & Ethnography
THEMES
Collaborative Enterprises 7 5 0 1 2
Conflict Resolution 5 4 1 0 7
Collaborative Reasoning 8 1 4 1 1
Organization of Knowledge 10 3 4 0 6
Collaborative Complexity 19 8 0 1 8
Spatiotemporal Challenges 6 4 1 0 3
4. Research Themes
4.1 Collaborative enterprises: Advances have already been made in the architecture of
collaborative enterprises [22, 23, 53, 83, 84, 111]. Standardized decision protocols can ensure
efficient demand and capacity sharing in collaborative enterprises [127]. They can ensure
conditions for agents to profit from joining or splitting forces [128]. Emergent decision
protocols can improve quality of collaboration [35, 36, 37, 92, 94].
4.2 Conflict resolution: Agent heterogeneity can aggravate agents’ conflicts [75, 114,
1 The total number does not exceed thirty-nine because a number of studies employed identical theories. 2 The cell entries correspond to the frequencies of studies. 3 The total number exceeds eighty-two because a number of studies employed multiple methods.
34
124]. A structured step-wise approach has been proposed to manage agents’ conflicts [109].
Supportive designs profile and statistically merge conflicting decision criteria [91, 113].
Iterative multi-attribute classification can aggregate agents’ preferences into a collective
solution [91, 101]. A distance between agents’ preferences and a collective solution, as well as
a solution acceptance rate, can further strengthen consensus measures [21, 74]. Agents’
capacity to decompose problems and exchange resources defuses conflicts [75, 114, 124].
4.3 Collaborative reasoning: Argumentation is the primary means of integrating
knowledge in collaborative decision-making [40, 41, 63, 106]. Different reasoning frameworks
support argumentation [63, 66, 95]. Agents can reason deductively, sequentially exchanging
premises, demonstrably reaching valid conclusions [63, 66]. They can also reason defeasibly,
non-demonstrably reaching contingent conclusions [95, 122]. Any proposed reasoning
framework only provides assessments of the various positions at any stage of a discussion,
ultimately resolving conflicting interpretations [95]. However, agents also have to agree on a
measures that resolves them [95]. Existing measures can assess collective solutions for
credibility of arguments [11, 12]. Efficiency of measures that profile opinion factions [13, 14,
15] and capture a decision rationale [11, 12] has also been demonstrated.
4.4 Organization of knowledge: Semantics convey function, data, execution, quality,
and trust in collaborative decision-making [44]. Apart from eliciting knowledge [39], agents
have to share and manage knowledge [49, 60, 68, 105, 121, 132]. Knowledge sharing
consolidates aggregate understanding, ensuring commensurability of reasoning frameworks
and efficiency of search for consensus [10]. Knowledge sharing can also result in false
outcomes [46]. In response, clearly specified, shared knowledge representations improve
efficient reasoning [17, 31, 78]. A deeper form of knowledge acquisition [47], an ontology
supports such representations [34, 73, 102]. Formation of ontology in collaborative decision-
making has been merely acknowledged [119].
4.5 Collaborative complexity: Collaborative complexity can be traced to the nature of
decision problems, the number of decision criteria, and the multiplicity of agents [2, 3, 5, 21,
28, 50, 55, 58, 75, 82, 114, 118, 124]. While uncertain problems make groups smaller and
closely knit, complex problems make them greater and loosely organized [53]. Complexity of
decision problems more easily leads to conflict resolutions [75, 114, 124]. By retrieving
information, simultaneously applied multi-attribute methods can contribute to efficiency of
collaborative decision-making [76]. However, rather than complete information [3, 61], agent
network properties and problem decompositions are more likely to achieve this efficiency [22,
35
92, 123, 130].
4.6 Spatiotemporal challenges: Metadata reveals spatiotemporal effects on
collaborative outcomes [65, 98, 99, 115]. Agents’ geographical dispersion, average spatial
distance, and mutual isolation detract from collaborative decision-making [88]. Agents use
maps to evaluate positions rather than to structure a problem they have to solve [57, 58, 87].
Time variably impacts agents’ decision-making [19, 29]. Distributed computing can
dynamically reframe agents’ spatiotemporal configurations [106, 117, 120, 131]. Being subject
to decision-making [19, 57, 87, 97], spatiotemporal effects can also give rise to reverse
causality issues. Thus, such effects on collaborative decision-making remain mixed [33, 72,
112, 126]. Table 4 outlines our themes.
5. Research Directions
Theoretical accounts of collaborative decision-making are scarce. Only fourteen
reviewed studies were confirmatory analyses. Only thirty-nine reviewed studies employed
some theoretical frameworks (e.g., [21, 39, 40, 55, 92]). Only four reviewed studies were
theoretical advances [48, 84, 89, 90]. The need for a cogent theory of collaborative decision-
making could not be more obvious. Our research directions are intended to motivate such
research efforts.
Our first research direction is to identify problem-solving conditions that require
collaborative solutions. In collaborative decision-making, an agents’ collectively owned hybrid
option has to be more valuable than any pre-conceived alternative [89]. It has recently been
acknowledged that such an option has to emerge from agents’ shared knowledge [89].
Table 4 Summary of the key threads
1. A range of standardized and emergent protocols were designed to support collaborative enterprises.
2. Specific techniques aid agents’ conflict resolution that also depends on behavioral variables.
3. Reasoning frameworks that integrate shared knowledge in collaborative decision-making interact with opinion factions’ dynamics.
4. Shared knowledge representation ensures reasoning efficiency, as required by the integration of knowledge in collaborative decision-making.
5. Rather than complete information, specific network agencies and problem decompositions are more helpful in achieving collaborative efficiency.
6. Spatiotemporal effects on collaborative decision-making are mixed.
36
Therefore, we propose to analyze agents’ decisions, by identifying conditions in which a
solution to a decision problem requires this emergence.
Our second research direction is to examine reasoning limits in collaborative decision-
making. Specifically, an ill-structured decision problem is characterized by undecidability,
uncertainty, and complexity, i.e., reasoning limits [25, 129]. Agents’ knowledge exists and is
integrated into collaborative decision-making under reasoning limits. These limits can be
variably specified by argumentation logics. They may provide yet another perspective on the
emergence of an agents’ collectively owned hybrid option. Thus, we propose to examine how
agents’ reasoning limits bound this emergence.
Our third research direction is to examine a dependence of collaborative solutions on
spatiotemporal configurations. The dependence of collaborative decision-making on
spatiotemporally configured agents has to be cogently explained [19, 33, 57, 72, 87, 88, 97].
The emergence of a collectively owned hybrid option from agents’ shared knowledge will
remain central to this dependence. Therefore, we propose to examine this dependence by
explaining how agents’ spatiotemporal configurations bound this emergence.
Our fourth research direction is to profile collaborative failures for mixtures of agents’
behaviors. “In collaborative decision-making, we do not strive for an optimum, a compromise,
or a satisficing solution” ([89], p. 1). Manipulative [110] and error reducing [4, 16] behaviors
are also non-collaborative. Thus, we propose to profile collaborative failures for mixtures of
such non-collaborative behaviors.
Our fifth research direction is to embed all of the above considerations into the
architecture of collaborative enterprises. We propose to study how to design these
considerations into collaborative enterprises [22, 23, 130]. We also propose to study their
effects within decision protocols intended for collaborative enterprises [35, 36, 37, 127, 128].
Table 5 outlines our research directions.
Table 5 Summary of the proposed research directions
1. Identify problem-solving conditions that require collaborative solutions.
2. Examine collaborative decision-making for implications of agents’ reasoning limits.
3. Examine collaborative outcomes for spatiotemporally configured agents.
4. Profile collaborative failures for mixtures of agents’ non-collaborative behaviors.
5. Embed the above effects into the architecture of collaborative enterprises.
37
In conclusion, our review is a milestone for the research of collaborative decision-
making. By making emergent, collectively owned hybrid options a central consideration in
collaborative decision-making, our directions can guide this research towards a cogent theory.
They can also encourage related empirical advances.
References
[1] A. Adla, Hybrid reasoning-based system for collaborative decision-making, International
Journal of Reasoning-based Intelligent Systems 3(3-4), 2011, 205 – 211.
[2] A. Adla, B. Nachet, A. Ould-Mahraz, Multi-agents model for web-based collaborative
decision support systems, CEUR Workshop Proceedings, 2012.
[3] B.M. Adler, W. Baets, R. König, Complexity perspective on collaborative decision making
in organizations: The ecology of group performance, Information and Management 48(4-5),
2011, 157-165.
[4] H. Aissi, Approximation and resolution of min-max and min-max regret versions of
combinatorial optimization problems, A Quarterly Journal of Operations Research 4(4), 2006,
347-350.
[5] M. Al-Shawa, Modeling and analyzing agents’ collective options in collaborative decision
making, Brain Informatics - Lecture Notes in Computer Science 6889, 2011, 111-123.
[6] S. Aldawood, F. Aleissa, R. Alnasser, A. Alfaris, A. Al-Wabil, Interaction design in a
tangible collaborative decision support system: The city schema DSS, Communications in
Computer and Information Science 425, 2014, 508-512.
[7] R.G. Aldunate, F. Pena-Mora, G.E. Robinson, Distributed decision making for large-scale
disaster relief operations: Drawing analogies from robust natural systems, Complexity 11(2),
2005, 28–38.
[8] O. Anya, H. Tawfik, A. Nagar, S. Amin, E-Workbench: A case for collaborative decision
support in e-health, Proceedings of the 11th International Conference on Computer Modelling
and Simulation, 2009, 634-639.
[9] D. Apostolou, G. Mentzas, Lj. Stojanovic, B. Thoenssen, T. Pariente-Lobo, A collaborative
decision framework for managing changes in e-Government services, Government Information
Quarterly 28(1), 2011, 101-116.
38
[10] P.-E. Arduin, M. Grundstein, C. Rosenthal-Sabroux, From knowledge sharing to
collaborative decision making, International Journal of Information and Decision Sciences
5(3), 2013, 295-311.
[11] R.S. Arvapally, X.F. Liu, Analyzing credibility of arguments in a web-based intelligent
argumentation system for collective decision support based on K-means clustering algorithm,
Knowledge Management Research and Practice 10, 2012, 326–341.
[12] R.S. Arvapally, X.F. Liu, Collective assessment of arguments in an online intelligent
argumentation system for collaborative decision support, Proceedings of International
Conference on Collaboration Technologies and Systems, 2013a 411-418.
[13] R.S. Arvapally, X.F. Liu, Polarization assessment in an intelligent argumentation system
using fuzzy clustering algorithm for collaborative decision support, Argument and
Computation 4(3), 2013b, 181-208.
[14] R.S. Arvapally, X. Liu, W. Jiang, Identification of faction groups and leaders in web-
based intelligent argumentation system for collaborative decision support, Proceedings of the
International Conference on Collaboration Technologies and Systems, 2012, 509-526.
[15] R.S. Arvapally, X.F. Liu, D.C. Wunsch, Fuzzy c-means clustering based polarization
assessment in intelligent argumentation system for collaborative decision support, Proceedings
of the 37th IEEE Conference on International Computer Software and Applications, 2013, 59-
64.
[16] I. Averbakh, Minimax regret solutions for minimax optimization problems with
uncertainty, Operations Research Letters 27(2), 2000, 57-65.
[17] J. Baumeister, A. Striffler, M. Brandt, M. Neumann, Towards continuous knowledge
representations in episodic and collaborative decision making, Proceedings of The 9th
Workshop on Knowledge Engineering and Software Engineering 1070, 2013.
[18] P.M. Bednar, V. Katos, C. Hennell, Cyber-crime investigations: Complex collaborative
decision making, Proceedings of the 3rd International Annual Workshop on Digital Forensics
and Incidents Analysis, 2008, 3-11.
[19] W.E. Berzins, M.D. Dhavala, Time versus trust: impact upon collaborative decision
making, Journal of Management in Engineering 4(4), 1988, 320-324.
[20] R.S. Bolia, W.T. Nelson, S.H. Summer, D.R. Arnold, J.L. Atkinson, R.M. Taylor, R.
Cottrell, C. Crooks, Collaborative decision making in network-centric military operations,
39
Proceedings of the Human Factors and Ergonomics Society Annual Meeting 3(50), 2006, 284-
288.
[21] S. Boroushaki, J. Malczewski, Measuring consensus for collaborative decision-making:
A GIS-based approach, Computers, Environment and Urban Systems 34(4), 2010, 322–332.
[22] X. Boucher, Collaborative decision-making support system to enhance competencies
within enterprise networks, Journal of Decision Systems 18(3), 2009, 319-346.
[23] F. Carton, F. Adam, Studying the impact of ERP on collaborative decision making: A case
study, Frontiers in Artificial Intelligence and Applications 176, 2008, 295-307.
[24] A. Castiglione, R. Pizzolante, A. De Santis, C. D'Ambrosio, A collaborative decision-
support system for secure analysis of cranial disorders, Proceedings of the International
Conference on Intelligent Networking and Collaborative Systems 189-196.
[25] G. Chaitin, Limits of reason, Scientific American 294(3), 2006, 74-81.
[26] A. J. Chapman, J.G. Pohl, Collaborative decision support systems for facility
management, Proceedings of The 10th International Conference on Systems Research,
Informatics and Cybernetics, 1998, 71-79.
[27] M. Y. Chim, C. J. Anumba, P. M. Carrillo, Internet-based collaborative decision making
system for construction, Advances in Engineering Software 35(6), 2004, 357-371.
[28] S. Christodoulou, N. Karacapilidis, M. Tzagarakis, Advancing collaborative decision
making through alternative visualizations and reasoning mechanisms, Intelligent Decision
Technologies - Frontiers in Artificial Intelligence and Applications 255, 2013, 38-47.
[29] B. Coury, M. Terranova, Collaborative decision making in dynamic systems, Proceedings
of the Human Factors and Ergonomics Society 35(13), 1991, 944-948
[30] M.V. D’Ortenzio, F.Y. Enomoto, S.L. Johan, A collaborative decision environment for
UAV operations, Collection of Technical Papers - InfoTech at Aerospace: Advancing
Contemporary Aerospace Technologies and Their Integration, American Institute for
Aeronautics and Astronautics NASA Ames Research Center, 2005.
[31] R. Davis, H. Shrobe, P. Szolovits, What is a knowledge representation? AI Magazine
14(1), 1993, 17-33.
40
[32] M. De Castro, J.M. De Souza, J. Strauch, Decisio: A collaborative decision support system
for environmental planning, Proceedings of the 5th International Conference on Enterprise
Information Systems 2, 2003, 217-222.
[33] A.R. Dennis, J.A. Rennecker, S. Hansen, Invisible whispering: restructuring collaborative
decision making with instant messaging, Decision Sciences 41(4), 2010, 845–886.
[34] A.V. Deokar, O.F. El-Gayar, N. Taskin, R. Aljafari, An ontology-based approach for
model representation, sharing and reuse, 14th Americas Conference on Information Systems 5,
2008, 3194-3202.
[35] M.V. Drissen-Silva, R.J. Rabelo, A model for dynamic generation of collaborative
decision protocols for managing the evolution of virtual enterprises, Proceedings of the 8th
IFIP International Conference on Information Technology for Balance Automation Systems,
2008, 105-114.
[36] M.V. Drissen-Silva, R.J. Rabelo, Managing decisions on changes in the virtual enterprise
evolution, In P. L. M. Camarinha-Matos et al. (Eds.), Proceedings PRO-VE - Leveraging
knowledge for innovation in collaborative networks, 2009, 463-475.
[37] M.V. Drissen-Silva, R.J. Rabelo, A collaborative decision support framework for
managing the evolution of virtual enterprises, International Journal of Production Research
47(17), 2012, 4833-4854.
[38] T.R. Ender, C. Haynes, J. Murphy, T. McDermott Enabling collaborative decision
making: A process for integrating novel systems engineering tools and methods for renewable
energy portfolio analysis, Incose International Symposium 19(1), 2009, 720–734.
[39] C.E. Evangelou, N. Karacapilidis, O.A. Khaled, H.C. Drissi, On the elicitation of
knowledge in collaborative decision making settings, Proceedings of the 6th European
Conference on Knowledge Management, 2005, 184-189.
[40] C.E. Evangelou, N. Karacapilidis, M. Tzagarakis, On the development of knowledge
management services for collaborative decision making, Journal of Computers 6(1), 2006, 19-
28.
[41] C.E. Evangelou, N. Karacapilidis, A multidisciplinary approach for supporting
knowledge-based decision-making in collaborative settings, International Journal of Artificial
Intelligence Tools 16, 2007, 1069.
41
[42] R.P. Ferreira, A.L. Soares, A collaborative decision support method to design performance
evaluation systems in CNOs. In L.M. Camarinha-Matos et al. (Eds.), Collaborative networks
for a sustainable world (336, pp. 561-568). IFIP Advances in Information and Communication
Technology, 2010.
[43] A.R. Gagliardi, F. Webster, M.C. Brouwers, N.N. Baxter, A. Finelli, S. Gallinger, How
does context influence collaborative decision-making for health services planning, delivery and
evaluation? BMC Health Services Research 14, 2014, 545.
[44] M.G. Gillespie, H. Hlomani, D. Kotowski, D.A., Stacey, A knowledge identification
framework for the engineering of ontologies in system composition processes, IRI Proceedings
of the IEEE International Conference on Information Reuse and Integration, 2011, 77-82.
[45] M. Grappe, M. Bui, Study of cockpit's perspective on human-human interactions to guide
collaborative decision making design in air traffic management, Proceedings of the First
International Conference on Advances in Computer-Human Interaction, 2008, 107-113.
[46] N. Gronau, E. Weber, P. Heinze, Interpretation of collaborative decisions by meta-metrics,
Proceedings of the International Conference on Knowledge Management and Information
Sharing, 2011, 158-166.
[47] T.R. Gruber, A translation approach to portable ontologies, Journal of Knowledge
Acquisition 5(2), 1993, 199-220.
[48] S.P. Gudergan, G.P. Gudergan, A dynamic theory of collaboration and decision making,
Proceedings of the 35th Annual Hawaii International Conference on System Sciences, 2002,
95.
[49] A. Gupta, E. Mattarelli, S. Seshasai, J. Broschak, Use of collaborative technologies and
knowledge sharing in co-located and distributed teams: Towards the 24-h knowledge factory,
Journal of Strategic Information Systems 18(3), 2009, 147–161.
[50] A. Hamel, S. Pinson, M. Picard, A new approach to agency in a collaborative decision-
making process, Proceedings of International Conference on Intelligent Agent Technology,
2005, 273 – 276.
[51] S. Hamsah, S. Burhanuddin, T. Harianto, Collaborative decision making for solid waste
management: A Delphi analytical hierarchy process approach, International Journal of Applied
Engineering Research 7(17), 2014, 835-850.
[52] J.E. Hernandez, D. Savin, A.C. Lyons, K. Stamatopoulos, Enhancing collaborative
42
decision-making processes using a web-based application: A case study of a UK precision
engineering SME, Group Decision and Negotiation - Lecture Notes in Business Information
Processing 180, 2014, 11-19.
[53] S. Holloway, A. Parmigiani, When collaboration trumps rivalry: examining organizational
forms in the construction industry, Academy of Management 1, 2011, 1-6.
[54] M.S. Househ, F.Y. Lau, Collaborative technology use by healthcare teams, Journal of
Medical Systems 29(5), 2005, 449-461.
[55] M. Indiramma, K.R. Anandakumar, Collaborative decision making framework for multi-
agent system, Computer and Communication Engineering 11, 2008, 40 – 46.
[56] S.V. Ivanov, S.S. Kosukhin, A.V. Kaluzhnaya, A.V. Boukhanovsky, Simulation-based
collaborative decision support for surge floods prevention in St. Petersburg, Journal of
Computational Science 3(6), 2012, 450–455.
[57] P. Jankowski, T.L. Nyerges, GIS-supported collaborative decision-making: Results of an
experiment, Annals of the Association of American Geographers 91(1), 2001, 48–70.
[58] P. Jankowski, T.L. Nyerges, A. Smith, T.J. Moore, E. Horvath, Spatial group choice: a
SDSS tool for collaborative spatial decision-making, International Journal of Geographical
Information Science 11(6), 1997, 577-602.
[59] T.I. Jefferson, J.R. Harrald, Collaborative technology: Providing agility in response to
extreme events, International Journal of Electronic Governance 1(1), 2007, 79–93.
[60] T. Kajdanowicz, Efficient usage of collective classification algorithms for collaborative
decision making, Cooperative Design, Visualization, and Engineering - Lecture Notes in
Computer Science 80(91), 2013, 73-80.
[61] E. Kamar, Y. Gal, B.J. Grosz, Modeling information exchange opportunities for effective
human-computer teamwork, Artificial Intelligence 195, 2013, 528-550.
[62] N. Kapucu, V. Garayev, Collaborative Decision-Making in emergency and disaster
Management, International Journal of Public Administration 34(6), 2011, 366-375.
[63] N.M. Karacapilidis, D. Papadias, Computer supported argumentation and collaborative
decision-making: the HERMES system, Information Systems 26(4), 2001, 259–277.
[64] N.M. Karacapilidis, M. Tzagarakis, Towards a seamless integration of human and
machine reasoning in data-intensive collaborative decision making settings: The Dicode
43
approach, In A. Respicio, & F. Burstein (Eds.), Fusing decision support systems into the fabric
of the context - Proceedings of the 16th International Conference on Decision Support Systems,
2012, 223-228.
[65] N. Karacapilidis, D. Papadias, M. Egenhofer, Collaborative spatial decision-making with
qualitative constraints, Proceedings of the 3rd ACM International Workshop on Advances in
Geographic Information Systems, 1995, 53-59.
[66] N. Karacapilidis, D. Papadias, C., Pappis, Computer-mediated collaborative decision
making: Theoretical and implementation issues, Proceedings of the Hawaii International
Conference on System Sciences 1, 1999, 1019.
[67] T. Kaupp, A. Makarenko, H. Durrant-Whyte, Human-robot communication for
collaborative decision making - A probabilistic approach, Robotics and Autonomous Systems
58(5), 2010, 444-456.
[68] M. Keith, H. Demirkan, M. Goul, The influence of collaborative technology knowledge
on advice network structures, Decision Support Systems 50(1), 2010, 140–151.
[69] B. Kempinen, From the benches and trenches criminal justice innovations in Wisconsin:
Collaborative Decision Making, Justice System Journal 30(3), 2009, 327-346.
[70] A.L., Kent, A. Casey, K. Lui, Collaborative decision-making for extreme premature
delivery, Journal of Paediatrics and Child Health 43(6), 2007, 489-91.
[71] G.L. Klein, J.L. Drury, M.S. Pfaff, CO-Action: Collaborative option awareness impact on
collaborative decision making, IEEE International Multi-Disciplinary Conference on
Cognitive Methods in Situation Awareness and Decision Support, 2011, 171 – 174.
[72] N. Kock, Asynchronous and distributed process improvement: The role of collaborative
technologies, Information Systems Journal 11(2), 2001, 87–110.
[73] H.Y. Lee, M.M. Sohn, Collaborative decision-making framework for supporting multi-
mobile decision-makers using ontology, Proceedings of the 6th International Conference on
Innovative Mobile and Internet Services in Ubiquitous Computing, 2012, 45-51.
[74] X. Li, H. Zhang, R. Mao, X. Wang, A consensus reaching model for collaborative decision
making in web 2.0 communities, Proceedings of the 6th International Conference on Business
Intelligence and Financial Engineering, 2014, 53 – 56.
44
[75] X.F. Liu, E.C. Barnes, J.E. Savolainen, Conflict detection and resolution for product line
design in a collaborative decision making environment, Proceedings of the ACM Conference
on Computer Supported Cooperative Work 13, 2012, 27-36.
[76] J. Ma, H. Adeinat, S.J. Kweon, J. Mines, G.E. Okudan, Synergistic use of AHP and trust
matrix in collaborative decision making, In A. Krishnamurthy and W.K.V. Chan (Eds.),
Proceedings of the Industrial and Systems Engineering Research Conference 12, 2013, 8-19.
[77] W.K. McQuay, B. Stilman, V. Yakhnis, Distributed collaborative decision support
environments for predictive awareness, Proceedings of the Conference on Enabling
Technologies for Simulation Science 9, 2005, 5805.
[78] E. Mercier, S. Higgins, Creating joint representations of collaborative problem solving
with multi-touch technology, Journal of Computer Assisted Learning 30(6), 2014, 497–510.
[79] R. Michaelides, S.C. Morton, W. Liu, A framework for evaluating the benefits of
collaborative technologies in engineering innovation networks, Production Planning and
Control: The Management of Operations 2-3, 2013, 246-264.
[80] S. Misono, S. Koide, N. Shimada, M. Kawamura, S. Nagano, Distributed collaborative
decision support system for rocket launch operation, IEEE/ASME International Conference on
Advanced Intelligent Mechatronics 13, 2005, 18 – 23.
[81] B.E. Munkvold, K. Eim, Ø. Husby, Collaborative IS decision-making: Analyzing decision
process characteristics and technology support, Groupware: Design, Implementation, and Use
- Lecture Notes in Computer Science 3706, 2005, 292-307.
[82] B. Nachet, A. Adla, An agent-based distributed collaborative decision support system,
Intelligent Decision Technologies 8(1), 2014, 15-34.
[83] A. Nakakawa, P. Van Bommel, H.A.E. Proper, Definition and validation of requirements
for collaborative decision-making in enterprise architecture creation, International Journal of
Cooperative Information Systems 20(1), 2011, 83–136.
[84] A. Nakakawa, P. Van Bommel, H.A.E. Proper, Towards a theory on collaborative decision
making in enterprise architecture, Global Perspectives on Design Science Research - Lecture
Notes in Computer Science 6105, 2010, 538-541.
[85] A. Nijholt, Competing and collaborating brains: Multi-brain computer interfacing,
Intelligent Systems Reference Library 74, 2015, 313-335.
45
[86] T. Nyerges, P. Jankowski, D. Tuthill, K. Ramsey, Collaborative water resource decision
support: Results of a field experiment, Annals of the Association of American Geographers
96(4), 2006, 699–725.
[87] T.L. Nyerges, P. Jankowski, Enhanced adaptive structuration theory: A theory of gis-
supported collaborative decision making, Geographical Systems 4(3), 1997, 225-259.
[88] M. B. O’Leary, J.N. Cummings, The spatial, temporal, and configurational characteristics
of geographic dispersion in teams, MIS Quarterly 31(3), 2007, 433-452.
[89] D. Owen, Collaborative decision-making, Decision Analysis 12(1), 2015, 29–45.
[90] P. Panzarasa, N.R. Jennings, T.J. Norman, Formalizing collaborative decision-making and
practical reasoning in multi-agent systems, Journal of Logic and Computation 12(1), 2002, 55-
117.
[91] G. Phillips-Wren, E. Hahn, G. Forgionne, Consensus-building in collaborative decision-
making, Collaborative Decision Making: Perspectives and Challenges - Frontiers in Artificial
Intelligence and Applications 176, 2008, 221-230.
[92] A.V. Pince, P. Humphreys, How efficient networking can support collaborative decision
making in enterprises, In P. Zarate, J. P., Belaud, G. Camilleri, F., Ravat, (Eds.) Proceedings
of the 2008 Conference on Collaborative Decision Making: Perspectives and Challenges -
Frontiers in artificial intelligence and applications 176, 2008, 187-198.
[93] M.C. Politi, R.L. Street, The importance of communication in collaborative decision
making: Facilitating shared mind and the management of uncertainty, Journal of Evaluation in
Clinical Practice 17(4), 2011, 579-84.
[94] R.J. Rabelo, A.A. Pereira-Klen, E.R. Klen, Effective management of dynamic supply
chains, International Journal of Networking and Virtual Organisations 2(3), 2004, 193-208.
[95] T.S. Raghu, R. Ramesh, A.-M. Chang, A.B. Whinston, Collaborative decision making: A
connectionist paradigm for dialectical support, Information Systems Research 12(4), 2001,
363-383.
[96] T.S. Raghu, R. Ramesh, A.B. Whinston, Addressing the homeland security problem: A
collaborative decision-making framework, Journal of the American Society for Information
Science and Technology 56(3), 2005, 310–324.
[97] K. Ramsey, GIS, modeling, and politics: On the tensions of collaborative decision support,
46
Journal of Environmental Management 90(6), 2009, 1972–1980.
[98] M. Raubal, S. Winter, A spatiotemporal model towards ad-hoc collaborative decision-
making, Geospatial Thinking - Lecture Notes in Geo-information and Cartography, 2010,
279-297.
[99] F.F. Reis, G. Pestana, J. Damásio, Enhancing the collaborative decision making process
by using rich context, Proceedings of the IADIS International Conference Information Systems,
2010.
[100] M.L. Rhodes, J. Murray, Collaborative decision making in urban regeneration: A
complex adaptive systems perspective, International Public Management Journal 10(1), 2007,
79-101.
[101] G. Rigopoulos, J. Psarras, D.Th. Askounis, Th. Web support system for group
collaborative decisions, Journal of Applied Sciences 8, 2008, 407-419.
[102] J. Rockwell, I.R. Grosse, S. Krishnamurty, J.C. Wileden, A decision support ontology
for collaborative decision making in engineering design, International Symposium on
Collaborative Technologies and Systems, 2009, 1-9.
[103] S.O. Rogers, J.Z. Ayanian, C.Y. Ko, K.L. Kahn, A.M. Zaslavsky, R.S. Sandler, N.L.
Keating, Surgeons’ volume of colorectal cancer procedures and collaborative decision-making
about adjuvant therapies, Annals of Surgery 250(6), 2009, 895-900.
[104] L.I. Rusu, W. Rahayu, T. Torabi, F. Puersch, W. Coronado, A.T. Harris, K. Reed,
Moving towards a collaborative decision support system for aeronautical data, Journal of
Intelligent Manufacturing 23(6), 2012, 2085-2100.
[105] I. Saad, M. Grundtsein, C. Rosenthal-Sabroux, How to improve collaborative decision
making in the context of knowledge management, In P., Zaraté, J. P. Belaud, G., Camilleri, F.,
Ravat (Eds.), Collaborative Decision Making: Perspectives and Challenges - Artificial
Intelligence and Applications, 2008, 493-500.
[106] A.P. Sani, C. Rinner, A scalable Geo-Web tool for argumentation mapping, Geomatica
65(2), 2011, 145-156.
[107] C. Schneeweiss, Distributed decision-making - a unified approach, European Journal of
Operational Research 150, 2003, 237-252.
[108] C. Schneeweiss, Hierarchies in distributed decision-making, 1999, Heidelberg:
47
Springer-Verlag Berlin.
[109] M. Schwartz, C. Eichhorn, Collaborative decision making: Use of multi-attribute utility
analysis to involve stakeholders in resolving controversial transportation issues, Journal of
Advanced Transportation 31(2), 1997, 171–183.
[110] N. Selvaraj, B. Fields, Rethinking collaborative decision-making across distributed work
communities in complex work settings, Proceedings of the 30th European Conference on
Cognitive Ergonomics, 2012, 8-14.
[111] F. Shafiei, D. Sundaram, S. Piramuthu, Multi-enterprise collaborative decision support
system, Expert Systems with Applications 39(9), 2012, 7637-7651.
[112] J. Sheffield, Design theory for collaborative technologies: Electronic discourse in group
decision, Proceedings of the 53rd Meeting of the International Society for the Systems
Sciences, 2009.
[113] W.W. Smari, K. Weigand, G. Petonito, Y. Kantamani, R. Madala, S. Donepudi, An
integrated approach to collaborative decision making using computer-supported conflict
management methodology, International Conference on Information Reuse and Integration,
2005, 182-191.
[114] P. Souren, R. Sumati, Manifested intra-group conflict in collaborative technology
supported multi-cultural virtual teams: Findings from a laboratory experiment, Proceedings of
the Annual Hawaii International Conference on System Sciences, 2010, 1-11.
[115] C. Stock, I.D. Bishop, A.N. O'Connor, T. Chen, C.J. Pettit, J.P. Aurambout, SIEVE:
Collaborative decision-making in an immersive online environment, Cartography and
Geographic Information Science 35(2), 2008, 133-144.
[116] A. Stoica, D.F. Barrero, K. McDonald-Maier, Improved targeting through collaborative
decision-making and brain computer interfaces, Proceedings of the International Conference
on Collaboration Technologies and Systems, 2013, 435-442.
[117] A. Sun, The enabling of collaborative decision-making in watershed management using
cloud-computing services, Environmental Modelling and Software 41, 2013, 93–97.
[118] L. Susskind, Complexity science and collaborative decision-making, Negotiation
Journal 26(3), 2010, 367–370.
48
[119] Y. Tang, S. Christiaens, K. Kerremans, R. Meersman, PROFILE COMPILER: Ontology-
based, community-grounded, multilingual online services to support collaborative decision
making, Proceedings of the 2nd International Conference on Research Challenges in
Information Science, 2008, 279-288.
[120] H. Thimm, Cloud-based collaborative decision making: Design considerations and
architecture of the GRUPO-MOD system, International Journal of Decision Support System
Technology 4(4), 2012, 39-59.
[121] S.H.T. Thompson R. Nishant, M. Goh, S. Aggarwal, Leveraging collaborative
technologies to build a knowledge sharing culture at HP analytics, MISQ Executive 10(1),
2011, 198-214.
[122] G. Vreeswijk, Reasoning with defeasible arguments: Example and applications, In G.
Wagner, D. Pearce (Eds.), JELIA Proceedings of the European Workshop on Logic in AI -
Lecture Notes in Computer Science 633, 1992, 189–211.
[123] P. Wang, W. Zhong, F. Lu, Collaborative ordering of enterprises with differentiated
products under the protection of sensitive information, International Conference on E-Business
and Information System Security, 2009, 1-3.
[124] E.R. Watson, P.G. Foster-Fishman, The exchange boundary framework: Understanding
the evolution of power within collaborative decision-making settings, American Journal of
Community Psychology 51(1-2), 2013, 151-163.