Negotiation

Martin Beer, School of Computing & Management

Sciences, Sheffield Hallam University, Sheffield,

United Kingdom

m.beer@shu.ac.uk

Introduction

• Principles of Negotiation– Game Theoretic Approaches

• Evaluation Criteria• Voting• Auctions• General Equilibrium Markets• Contract Nets

– Heuristic-based Negotiation– Argumentation-based Negotiation

Principles of Negotiation

• Negotiation = Interaction amongst Agents based on communication for the purpose of coming to an agreement

• Distributed conflict resolution• Decision Making• Proposal accepted, refined,

criticized or refuted

Principles of Negotiation

Collectively Motivated AgentsCommon Goals

Coordination to achieve Common goal

Self-interested AgentsOwn Goals

Coordination for Coherent Behaviour

Coordination

Distributed Search through a space of possible solutions

Negotiation Includes

• A communication language

• A negotiation Protocol

• A decision Process by which an agent decides upon its– Position– Concessions– Criteria for agreement etc.

Negotiation Strategies

• Negotiation can take a number of different forms– Single or multi-party negotiation– May include a single shot message by

each party or a conversation with several messages going back and forth

Negotiation Techniques

• Game Theoretic Negotiation– Evaluation Criteria– Voting

Auctions– General Equilibrium Market Mechanisms– Contract Nets

• Heuristic-based Negotiation• Argument-based negotiation

Game Theoretic Negotiation• Evaluation Criteria

– Criteria to evaluate negotiation protocols among self-interested agents

– Agents are supposed to behave rationally– Rational behavior = an agent prefers a greater utility

(payoff) than a smaller one– Payoff maximization

• Individual payoffs• Group payoffs• Social welfare

– Social Welfare• The sum of agents’ utilities (payoffs) in a given solution• Measures the global good of the agents• Problem: How do we compare utilities?

Utility• Utility is the means by which we

optimise the results of the negotiation• Utility often equates to price, this may

not always be the case. It may be– Speed of response– “closeness”– Some combination of functions etc.

• Any function that can be readily computed can be used.

Pareto Efficiency• A solution x

– i.e. a payoff vector p(x1, … xn) is Pareto efficient (i.e. Pareto optimal) if there is no other solution x’ such that at least one agent is better off in x’ than in x and no agent is worse off in x’ than in x.

• Measures global good– Does not require utility comparison

• Social Welfare Pereto Efficiency

Individual Rationality (IR)

• IR of an agent participation = The agent’s payoff in the negotiated solution is no less than the payoff that the agent would get by not participating in the negotiation

• A mechanism is IR if the participation is IR for all agents

Stability• A protocol is stable if once the agents arrived at a

solution they do not deviate from itDominant Strategy

– The agent is best off using a specific strategy no matter what strategies the other agents use

Nash Equilibrium– The strategy profile S*

A = < S*1 …. S*

n >– For each I, S*

I is the agent’s best strategy given that the other agents use strategies

< S*1 …. S*

i-1 ,S*i+1 …. S*

n >P

– Problems• No Nash Equilibrium• Multiple Nash Equilibria• Guarantees stability only in the beginning of the game

Prisoners Dilemma

• Two prisoners are collectively charged with a crime and held in separate cells, with no way of meeting or communicating.

• They are told that:– If one confesses and the other does not, the

confessor will be freed, and the other will be jailed for three years

– If both confess, then each will be jailed for two years

• Both prisoners know that if neither confesses, then they will each be jailed for one year.

Prisoner’s Dilemma – Possible Outcomes

Note – 4 is immediate release

Player j

Cooperate Defect

Player i Cooperate 4,4 1,4

Defect 4,1 3,3

Prisoner’s Dilemma – Strategic Considerations

• The individual rational action is to defect• This guarantees a payoff of at least 2 whereas

cooperating guarantees a payoff of at most 1• Logic Says that this is not the best alternative – if

both cooperate the payoff is 3• This is the fundamental problem with multi-agent

systems• It appears to imply that cooperation will not occur with

self-interested agents• Can we get over this?

Yes – by repeating the problem many times

Computational Efficiency

• To achieve perfect rationality– The number of options to consider is too big– Sometimes no algorithm finds the optimal solution

• Bounded Rationality– Limits the time/computation for options

consideration– Prunes the search space– Imposes restrictions on the types of spaces

Voting• Truthful Voters

– Rank feasible social outcomes based on agents’ individual ranking of those outcomes

– A – set of n agents– O – set of m feasible outcomes– Each agent has a preference relation

<I : O x O, asymmetric and transitive

• Social Choice Rule– Input: the agent preference relations (<1, …. <n)– Output: elements of O sorted according to the

input – gives the social preferences relation <*

Properties of the Social Choice Rule

• A social preference ordering <* should exist for all possible inputs (individual preferences)

• <* should be defined for every pair (o,o’) O• <* should be asymmetric & transitive over O• The outcomes should be Pareto efficient

If I A, o<I o’ then o <* o’

• The scheme is independent of irrelevant alternativesIf I A, < and <‘ satisfy o <I o’ and o <’I o’ then the social

ranking of o and o’ is the same in these two situations

• No agent should be a dictator in the sense thato <I o’ implies o<* o’ for all preferences of the other agents

Arrow’s Impossibility Theorem• No Social choice rule satisfies all of the six conditions

listed on the last slide• Alternatives

– Binary Protocol• Alternatives are voted pair-wise• The chosen alternative depends on the agenda• That is the order of the pairings• This is the method used in parliamentary debates

– Borda Protocol• Assigns an alternative |O| points for the highest preference, |O|

-1 points for the second and so on …..• The counts are summed across the voters and the alternative

with the highest count becomes the social choice• Winner turns loser and loser turns winner if the lowest ranked

alternative is removed

Auctions - Theory• The auctioneer wants to sell an item at the highest possible

payment and the bidders want to acquire the item at the lowest possible price

• A centralised protocol includes– One auctioneer– Several bidders

• The auctioneer introduces the item for sale, which can be a combination of other items, or have multiple attributes

• Bidders make offers. This may be repeated for several times, depending on the auction type

• The auctioneer determines the winner

• Auction Characteristics– Simple protocols– Centralised– Allows collusion “behind the scenes”– May favour auctioneer

Auction Settings• Private Value Auctions

– The value to a bidder agent depends on its private preferences

– Assumed to be known exactly

• Common Value Auctions– The good’s value depends entirely on other

agents’ valuation

• Correlated Value Auctions– The good’s value depends on internal and external

valuations

English (First-price Open Cry) Auction

– Each bidder announces openly its bid– When no bidder is willing to raise anyone, the auction ends– The highest bidder wins the item at the price of the bid

• Strategy– In private value auctions the dominant strategy is to always

bid a small amount more than the current highest bid and stop when the private value is reached

– In correlated value auctions the bidder increases the price at a constant rate or at a rate it thinks appropriate

First-Price Sealed-Bid Auction

• Each bidder submits one bid without knowing the other’s bids

• The highest Bidder wins the item and pays the amount of their bid

• Strategy– No dominant strategy– Bid less than its true valuation but it is dependant

on the other agents bids which are not known

Dutch (descending) Auction

• The auctioneer continuously lowers the price until one of the bidders takes the item at the current price

• Strategy– Strategically equivalent to the first-price

sealed-bid auction– Efficient for real time

Vickery (Second-Price Sealed Bid) Auctions

• Each bidder submits one bid without knowing the others’ bids

• The highest bidder wins but pays the price of the second bid

• Strategy– The bidder dominant strategy is to bid its

true valuation

All-Pay Auctions

• Each participating bidder has to pay the amount of their bid (or some other amount) to the auctioneer

Problems with Auction Protocols

• They are not collusion proof• Lying Auctioneer

– Problem with Vickery Auction– Problem with English Auction – use skills that bid in the

auction to increase bidders’ valuation of the item– The auctioneer bids the highest second price to obtain its

reservation price – may lead to the auctioneer keeping the item

– Common value auctions suffer from the winner’s curse: agents should bid less than their valuation prices (as winning the auction means its valuation is too high)

– Interrelated auctions – the bidder may lie about the value of an item to get a combination of items at its valuation price

General Equilibrium Markets• General Equilibrium Theory =A macroeconomic

theory• A set of goods available at different prices• Two types of agents – Consumers & Producers• The producer’s profits are profits are divided among

the consumers according to predetermined proportions that need not be equal

• The producers’ profits are divided among consumers according the shares they ‘own’

• Prices may change and the agents may change their consumption and production plans but– Actual production and consumption only occur when the

market has reached a general equilibrium

Contract Nets• General Equilibrium Market Mechanisms use

– Global prices– A centralised mediator

• Drawbacks– Not all prices are global– Bottleneck of the mediator– Mediator – point of failure– Agents have no direct control over the agents to which they

send information• Need a more distributed solution• Task allocation via negotiation – Contract Net

– A kind of bridge between game theoretic negotiation and heuristic-based one

– Formal model for making bids and awarding decisions

Contract Net• Protocol

– The agents suggest contracts to each other and make their accepting/rejecting decisions based on marginal cost calculations

– The agent can take the roles of both Contractor and Contractee

– It can also contract out tasks that it received earlier via another contract

– The agents do not know the tasks and cost functions of other agents

– Task allocation improves with each step - hill climbing in the space of task allocations where the high-metric of the hill is social welfare

– It is an anytime algorithm• Contracting can be terminated anytime• The worth of each agent’s solution increases monotonically

Social Function increases monotonically

Contract Net• Problem

– Task allocation stuck in a local equilibrium = no contract net is individually rational and the task allocation is not globally optimal

• Possible solution– Different contract types

• O – one task• C – cluster contracts• S – swap contracts• M – multi-agent contracts

– For each of the four contract types there exists task allocations \for which there is an IR contract under one type but no IR contracts under the other three types

– Under all four contract types there are initial task allocations for which no IR sequence of contracts will lead to the optimal solution (social welfare)

Contract Net• Main differences as compared to game theoretic

negotiation– An agent may reject an IR contract– An agent may accept a non-IR contract– The order of accepting IR contracts may lead to different pay

offs– Each contract is made by evaluating just a single contract

instead of doing look ahead in the future• Untruthful Agents

– An agent may lie about its marginal costs– An agent may lie about what tasks it ahs

• Hide tasks• Phantom tasks• Decoy tasks

– Sometimes lying may be beneficial

Heuristic-based Negotiation• Produce good rather than optimal solution• Heuristic-based negotiation

– Computational approximations of game theoretic techniques

– Informal negotiation models• No central mediator• Utterances are private between negotiating

agents• The protocol does not prescribe an optimal

course of action• Central concern: the agent’s decision making

heuristically during the course of negotiation

Argumentation-based Negotiation

• Arguments used to persuade the party to accept a negotiation proposal

• Different types of agents• Each Argument type defines preconditions for

its usage. If the preconditions are met, then the agent may use the argument

• The agent needs a strategy to decide which argument to use

• Most of the times assumes the BDI model

Argumentation-based Negotiation

• Strategies– Appeal to past promise– Promise a future reward– Appeal to self interest– Threat