Post on 19-Dec-2015
transcript
Distributed Multiagent Resource Allocation In Diminishing Marginal Return Domains
Yoram Bachrach(Hebew University)
Jeffrey S. Rosenschein (Hebrew University)
Outline
Multiagent Resource Allocation (MARA) General problem Applications
Centralized and decentralized mechanisms Selfish behavior challenge
Specific restricted domain VCG solution in restricted domain
Allocation by interaction Market motivation behind method
Allocation protocol and suggested strategies Convergence to optimal allocation Strategic and selfish behaviour Expected time to convergence
Conclusions and future research
Multiagent Resource Allocation
Allocating resources to users Scarce resources Selfish agents with private information
Both users and resource owners
An allocation maps resources to users
MARA Applications
Industrial procurement Satellite resources Tasks in manufacturing systems Grid computing RF spectrum and coverage …
MARA Domain Properties
Divisible / Indivisible Can parts of a single resource be allocated to several agents?
Sharable / Non-Sharable Can a resource be allocated to several agents simultaneously?
Single-Unit / Multi-Unit Are there bundles of identical resources?
Transferable / Non Transferable Utility Can agents compensate by transferring utility among them?
MARA Approaches
Attempt to maximize social welfare Other possible goals – Maximin, fairness, … There may be more than one optimal allocation
Centralized mechanisms A central mechanism gets the agents’ preferences and chooses
an outcome
Decentralized approaches Agents actively participate in choosing the outcome
Problem – agents are selfish and attempt to maximize their own utility
Centralized Mechanisms
The mechanism must elicit the agents’ private information about allocations But agents may manipulate to increase their own utility
We are interested in incentive compatible mechanisms Agents reply truthfully, under a certain rational behavior Rational behavior captured in a game theoretic solution concept
Vickery-Clarke-Groves (VCG) approach Tax agents to make truth telling is a dominant strategy Strategyproof, allocatively efficient but only weakly budget balanced
Distributed Mechanisms
Central mechanisms may not be appropriate in distributed environmentsHard to establish a trusted central authorityScalability concerns – the central mechanism may be a
performance bottleneck Have agents interact among themselves to
choose the allocationNeed to define the protocol for interactionSelfish agents may still manipulate
Specific Domain
Set of identical agents Each agent only requires a single resource, and does not benefit
from being allocated more than one resource
Set of resources Cannot be divided among agents Can be shared among agents
Diminishing marginal production The total utility of the agents who are allocated a certain
resource drops as more agents use that resource
Diminishing Marginal Return
10
10
7
14
7
5
5
15
5
Diminishing Marginal Return
10
10
7
14
7
5
5
15
5
Total production is 10
Diminishing Marginal Return
10
10
7
14
7
5
5
15
5
Total production increases to 14
Diminishing Marginal Return
10
10
7
14
7
5
5
15
5
Total production increased by 4 when adding a single agent Marginal production of 4
Diminishing Marginal Return
10
10
7
14
7
5
5
15
5
Total production increased by 1 when adding a single agentMarginal production of 1
What needs to be decided?
A mechanism must decide: An allocation – which agent gets which resource
We want to maximize the social welfare – total production
Utility transfers Agents gain utility due to the allocation
Resource owners receive nothing
Resource owners hold the private information Eliciting this information requires incentivizing the resource owners to report
their production function Requires giving resource owners some of the utility
We assume the total production across all the resources can be redistributed in any way
VCG in Restricted Domain
Easy to compute an optimal allocation Resources report total production functions Find maximal social welfare by a greedy algorithm
Assign to the resource with maximal marginal production
Induce truthfullness by VCG tax Requires establishing a trusted central authority
Trust and security issues, central bottleneck, … Weakly budget balanced – some of the total production is
kept in the mechanism and not distributed
Allocation by Interaction
Define a protocol for interaction between agents and resource owners Simulate a market for services
Interaction proceeds in discrete time rounds Each round determines both an allocation and transfers
Design protocol and suggest interaction strategies so that the optimal allocation is always reached
Challenges Achieve the optimal allocation despite selfishness Make sure the optimal allocation is reached quickly
Interaction Protocol
R1
R2
R3
Round Payment (5)
Currently on R1, getting utility 5
Interaction Protocol
R1
R2
R3
Resource Request
Currently on R1, getting utility 5
Interaction Protocol
R1
R2
R3
Payment Bid (10)
Interaction Protocol
R1
R2
R3
Accept
Switch to R2 with utility 10
Interaction Protocol
R1
R2
R3
Decline
Stay on R1, with utility 5
Interaction Protocol
R1
R2
R3
Round Payment 10
Currently on R2 with utility 10
Interaction Protocol
R1
R2Payment Change (5)
Currently on R2 with utility 5
The Resource Owner’s Perspective
4
4
13
4
5
5
12
Production – 12Payments – 10Utility – 2
Production – 13Payments – 12Utility – 1
Chosen Allocation
The interaction decides both the allocation and redistribution of the utility Agents are allocated the last resource whose bid they accepted Agents get the utility as in the last payment bid they accepted Resource owners keep the reminder of the production on the
resource not redistributed to the agents
The allocation may change at the end of every round An allocation is stable if once reached it never changes
Depends on the strategies of the participants Agents and resource owners
Suggested Strategy - Agents
Each round, randomly choose a resource and request using the resource If the bid in that resource is better than the
current bid, switch to that resource (accept) If the bid is lower than the current resource
offers, stay with current resource
Suggested Strategy – Resource Owners Keep the agents’ share of the utility in the level of
the marginal production on the resource On round start, offer all the agents allocated to the
resource the current last marginal production Answer resource requests with bid of the next
marginal production on the resource If accepted, set the bid for all the agents to the new
marginal production by a Payment Change message If declined – do nothing
Resource Owners - Example
10
10
4
14
4
1
1
15
1
MP = 4 MP = 1
Protocol Stable Allocation
Given a set of strategies for the agents and resource owners, a protocol stable allocation is one that, once reached, never changes Under these strategies, no interaction results in an agent switching to a
different resource Protocol stable under the suggested strategies
No agent is ever given a bid higher than what he is currently getting on his current resource
Resource owners bid the next marginal production There is no resource where the next marginal production is greater than
the current marginal production on other resources Similar to greedily allocating agents to resources according to marginal production
Convergence to Optimum
Under the suggested strategies, the chosen allocation always converges to the optimal allocation Monotonic improvement
If an agent switches resources, the social welfare increases Stability in optimum
The optimal allocation is protocol stable No “local” optima – protocol stable is optimal
If a non optimal allocation is chosen, there is a possible round where an agent switches resources
What about strategic behavior?
Strategic Behavior
Agents and resource owners have to follow the protocol, but not the suggested strategies Might obtain higher utility by choosing a different strategy
Agents may accept a bid lower than what they currently have Resource owners may suggest a bid different than the current
marginal production Higher, to attract more agents Lower, to give a lower share of utility to the agents
Is such strategic behavior rational for self interested agents?
Strategic Agents (Our domain)
If an agent gained from strategic behavior, we still reach an optimal allocation If a single agent has deviated from the
suggested strategy and gained utility Gained utility: a protocol stable allocation has been
reached, in which the agent gets a higher utility
Then the reached protocol stable allocation is also optimal
Strategic Resource Owners
Resource owners who set too high a bid Attract more agents but pay more and lose utility
Resource owners who set too low a bid Pay less, but lose agents to competing resources
who offer higher bids
When the domain is competitive for resource owners, such a manipulation is irrational
Highly competitive settings Condition that occurs mostly in environments where there are
many resources with similar marginal production values Similar resources or slight changes in marginal production
Strategic Resource Owners
In our specific domain Diminishing marginal return Highly competitive for resource owners
If a resource owner gained from strategic behavior, we still reach an optimal allocation If a single resource owner has deviated from the
suggested strategy and gained utility Gained utility: a protocol stable allocation has been reached, in
which the resource owner gets a higher utility
Then the reached protocol stable allocation is optimal
Convergence Time
When agents and resource owners behave rationally, we converge to an optimal allocation But how quickly is the optimal allocation reached?
Under the suggested strategiesExpected time to convergence: Bound on convergence time:
Quick polynomial convergence
Related Work
TFG-MARA survey Y. Chevaleyre, P. E. Dunne, U. Endriss, J. Lang, M. Lemaître, N. Maudet, J. Padget, S. Phelps, J. A.
Rodríguez-Aguilar, and P. Sousa. Issues in Multiagent Resource Allocation.
Distributed mechanism design approaches J. Feigenbaum and S. Shenker. Distributed algorithmic mechanism design: Recent results and
future directions.
Scheduling domains B. Heydenreich, R. Muller, and M. Uetz. Decentralization and mechanism design for online
machine scheduling. Negotiations over resources
U. Endriss, N. Maudet, F. Sadri, and F. Toni. Negotiating socially optimal allocations of resources. T. W. Sandholm. Contract types for satisficing task allocation.
Conclusions
A distributed approach to resource allocation in a specific domain Achieves optimal allocation (maximal social welfare) No central authority required All utility divided among agents and resource owners
“Strongly budget balanced”
Quick convergence
Can a similar approach be applied to other domains (or more general domains)?