Using Dialogue Games to Form Coalitions with Self-Interested Agents

Luke RileyDepartment of Computer Science

University of LiverpoolL.J.Riley@Liverpool.ac.uk

Supervisors: Katie Atkinson & Terry Payne

Click to edit the outline text format

Second Outline Level

Third Outline LevelFourth Outline Level

Fifth Outline Level Sixth Outline Level Seventh Outline

Level Eighth Outline Level

Ninth Outline LevelClick to edit Master text styles

Second level

Third level

Fourth level

Fifth level

2

1. Coalition Formation in Cooperative Game Theory.

2. Coalition Formation in Argumentation.

3. The Issues and Problems Between these Two Approaches.

4. My Research.

Talk Overview

3

1. Coalition Formation in Cooperative Game Theory (CGT)

4

Background

N-person cooperative games (coalition games) were first proposed in detail by von Neumann & Morgenstern in 1944[1]:

Where...

[1] J. von Neumann and O. Morgenstern. The Theory of Games and Economic Behavior. Princeton University Press, 1944.

Characteristic Function:

Agent set:

5

In its most traditional style the CGT outcome of a coalition game is:

Solving a Coalition Game

e.g. When Ag = {a,b,c} a possible CS could be {{a},{b,c}} and a possible payoff vector could be x(1,2,3) where:

Agent Payoffa 1b 2c 3

x = a vector of each individual agent's payoff in the game.

Where...

CS = a set of coalitions (the coalition structure)

6

Finding a Stable Outcome – The Core

A Coalition structure is core-stable if no subset of agents can benefit from defecting to another coalition.

The core [2] is the set:

e.g. [3] Example 1: Given a coalition game where

Ag = {a,b}, v({a}) = v({b}) = 5 and v({a,b}) = 20

the proposed core outcome is <{a,b}, x(10,10) >

[2] D. Gillies. Some theorems on n-person games. PhD thesis, Princeton University, 1953.[3] Wooldridge, M. An Introduction to MultiAgent Systems Second Edition. John Wiley & Sons, 2009

e.g. [3] Example 2: Given a coalition game where

Ag = {a,b}, v({a}) = v({b}) = 5 and v({a,b}) = 20

the proposed core outcome is <{a,b}, x(15,5) >

7

Finding a Stable Outcome – The Core

A Coalition structure is core-stable if no subset of agents can benefit from defecting to another coalition.

The core [2] is the set:

Yet core payoffs can sometimes be unfair

[2] D. Gillies. Some theorems on n-person games. PhD thesis, Princeton University, 1953.[3] Wooldridge, M. An Introduction to MultiAgent Systems Second Edition. John Wiley & Sons, 2009

8

Epsilon-Core

Solution [5] →

The epsilon value can be seen as the cost of deviating.

Also the core can sometimes be empty

e.g. [4] Example 4: Given the coalition game of

example 3, the payoff vector x(1/3,1/3,1/3) is

1/3-core stable.

[4] G. Chalkiadakis, E. Elkind, and M. Wooldridge. Computational Aspects of Cooperative Game Theory. Morgan & Claypool Publishers, 2011.[5] Shapley, Lloyd S. and Shubik, M. Quasi-cores in a monetary economy with non-convex preferences , Econometrica (The Econometric Society) 34(4): 805–827, 1966.

e.g. [4] Example 3: Given a coalition game where Ag = {a,b,c}, forall subsets C if |C| = 2 agents then v(C) = 1 else v(C) = 0

9

2. Coalition Formation in Argumentation

10

Argumentation Background Argumentation Frameworks are a means to

represent and reason with different possibly conflicting data.

AFs use graphs of nodes and arcs:

.

= Preferred extension

A set of arguments S are acceptable if for every argument a1 that attacks an element a2 in S then there exists another a2 in S that defeats a1.

The preferred extension is the maximal acceptable set

11

Dung's Initial Work Dung showed that Argumentation

Frameworks were natural ways to represent n-person games, for example theorem 6 of [6]:

[6] P. M. Dung. On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games. Artificial Intelligence, 77:321–357, 1995.

x(3,4,8)

x(3,3,5) x(3,3,3)

The AF represents 3 possible payoff vectors of the coalition game:

v({a}) = v({b}) = v({c}) = 3

v({a,c}) = 8

v({b,c}) = 12

or v(C) = 0

12

Amgoud's Further Research Amgoud in [7] extended this research,

where she highlighted:

Attacks can come from multiple attributes such as coalitions sharing an agent.

How to always find a solution to a coalition game.

Outlines how agents can collaboratively build AFs for coalition games.

How a dialogue game can be used to check if a certain coalition was in the best coalition structure.

[7] L. Amgoud. An argumentation-based model for reasoning about coalition structures. In ArgMAS, pages 217-228, 2005.

13

3. The Issues of Joining the Two Approaches

CGT: Lacks flexible communication protocols to form stable coalition structures.

CGT: Generally does not take into account the computation and communication costs of finding stable coalition structures from a MAS perspective.

Arg: There is little research showing how payoff vectors are found and justified by MAS.

Arg: No research on how to stabilise coalitions games, using the epsilon-core

Arg: Only some limited direct mapping between the argumentation models and the CGT coalition game types (e.g. static, dynamic, skill games,...)

14

Various Issues

15

My Current Research Question

How can self-interested agents make use of argumentation within their communication to enable them to form a stable optimal coalition

structure with an approximately fair payoff distribution?

16

4. The Proposed Method

17

Dialogue Games & Argumentation SchemesDialogue Games can be used to build argumentation

frameworks in real time, where agents can assert and retract arguments.

Argumentation schemes are patterns of reasoning that when instantiated provide presumptive justification for the particular conclusion of the scheme

e.g:...

Solution – restrict the payoffs allowed:

Agents have to propose an equal split of the payoff or each agent should be given at least the same value it can get from a coalition of agents willing to defect (must assert the best coalition for itself)

Agents can object to a proposed payoff by finding a better one.

Once a core payoff is found, the dialogue stops

18

Approximately fair payoffs

AFs can easily represent the core

...But the core can be unfair

19

Dialogue Games & Argumentation Schemes I have devised a dialogue game [8] to find

an optimal coalition structure with a restricted core payoff

Moves:

e.g:

[8] L. Riley, K. Atkinson, and T. Payne. Coalition structure generation for self interested agents in a dialogue game. Technical Report ULCS-12-004, University of Liverpool, 2012.

20

Core example

  v({a}) = 4

v({b}) =

3

v({c}) = 2

v({a.b}) = 14

v({a,c}) = 18

v({b,c}) = 5

v({a,b,c}) = 7

A1 [4]            A2 [4] [3]          A3 [4] [3] [2]        

Coalitions and associated payoff

Mov

e nu

mbe

r

Join phase

= Preferred extension

The coalition structure of move A3 is {{a},{b},{c}}, the payoff vector is x(4,3,2)

21

Core example

  v({a}) = 4

v({b}) =

3

v({c}) = 2

v({a.b}) = 14

v({a,c}) = 18

v({b,c}) = 5

v({a,b,c}) = 7

A1 [4]            A2 [4] [3]          A3 [4] [3] [2]        A4   [3]     [9,9]    

Coalitions and associated payoff

Mov

e nu

mbe

r

Join phase

Negotiation phase

= Preferred extension

The coalition structure of move A4 is {{a,c}, {b}}, the payoff vector is x(9,3,9)

No cycles are created as later arguments have ‘’fairer’’ payoffs than earlier arguments

22

Core example

  v({a}) = 4

v({b}) =

3

v({c}) = 2

v({a.b}) = 14

v({a,c}) = 18

v({b,c}) = 5

v({a,b,c}) = 7

A1 [4]            

A2 [4] [3]          

A3 [4] [3] [2]        

A4   [3]     [9,9]    

A5     [2] [10,4]      

Coalitions and associated payoff

Mov

e nu

mbe

r Join phase

Negotiation phase

= Preferred extension

The coalition structure of move A5 is {{a,b}, {c}}, the payoff vector is x(10,4,2)

23

Core example

The coalition structure of move A6 is {{a,c}, {b}}, the payoff vector is x(11,3,7) and is core stable

  v({a}) = 4

v({b}) =

3

v({c}) = 2

v({a.b}) = 14

v({a,c}) = 18

v({b,c}) = 5

v({a,b,c}) = 7

A1 [4]            

A2 [4] [3]          

A3 [4] [3] [2]        

A4   [3]     [9,9]    

A5     [2] [10,4]      

A6   [3]     [11,7]    

Coalitions and associated payoff

Mov

e nu

mbe

r Join phase

Negotiation phase

= Preferred extension

24

Changes in agents payoff in a core stable game

1 2 30

5

10

15

20

25

30

35

Agent Payoff Graph

Agent jAgent k

Number of arguments for the coalition

Age

nt p

ayof

f

Core payoffs for agent j and k

Core payoffs for agent j

25

When no payoff changes are needed

Core payoffs for agent j and k

Core payoffs for agent j

1 2 3 4 50

5

10

15

20

25

30

35

40

45

Agent Payoff Graph

Agent jAgent k

Number of Arguments for the coalition

Age

nt p

ayof

f

26

Recognising when the core is empty

Core payoffs for agent j and k

Core payoffs for agent j

When there does not exist an agent in the coalition whose payoff is strictly increasing or decreasing → then the core is empty (given rules I have outlined)

  v({a}) = 5

v({b}) = 5

v({c}) = 5

v({a.b}) = 18

v({a,c}) = 20

v({b,c}) = 22

v({a,b,c}) = 10

4 [5]      [10/10]    

5 [5]       [11/11]   

6 [5]   [8/12]     

7    [5]  [9/9]    

8  [5]     [11/11]   

27

Empty core example

No agent of coalition {b,c} has a strictly increasing or decreasing payoff in the arguments for that coalition

Coalitions and associated payoff

Mov

e nu

mbe

r

  Epsilon value

v({a}) = 5

v({b}) = 5

v({c}) = 5

v({a.b}) = 18

v({a,c}) = 20

v({b,c}) = 22

v({a,b,c}) = 10

8 0  [5]       [11/11]   

1 v({C}) = 17 

v({C}) = 19 

v({C}) = 21

v({C}) = 9 

9 1 [6]   [7/12]     

2 v({C}) = 16 

v({C}) = 18 

v({C}) = 20

v({C}) = 8 

10 2     [7] [8/8]      

3 v({C}) = 15 

v({C}) = 17 

v({C}) = 19

v({C}) = 7 

11 3 [8]         [9.5/9.5]  

  4 v({C}) = 14

v({C}) = 16 

v({C}) = 18

v({C}) = 6

28

Epsilon-Core Example

Coalition Structure of move 11 is {{a},{b,c}}, the payoff vector is x(8,9.5,9.5) and is 3-core stable

29

Next Steps…? Extend the dialogue game to find epsilon-core

stability and identify under what conditions the least core can be found.

Experiment with ideas further and find proofs.

Modify dialogue game so that other coalition games can be modeled.

Optimise process: Combine mechanism design approach of [9] with efficient distribution methods of [10].

[9] T. Sandholm, K. Larson, M. Andersson, O. Shehory and F. Tohmé, Coalition structure generation with worst case guarantees, Artificial Intelligence, Volume 111, Issues 1–2, July 1999, Pages 209-238.

[10] T. Rahwan. Algorithms for Coalition Formation in Multi-Agent Systems. PhD thesis, University of Southampton, 2007.

30

Thanks For Listening

Questions?