Date post: | 05-Jan-2016 |
Category: |
Documents |
Upload: | bertha-hutchinson |
View: | 216 times |
Download: | 2 times |
Agent-Organized Networks for Dynamic Team Formation
Multi-Agent Planning and Learning Laboratory (MAPLE)
Department of Computer Science and EE
University of Maryland Baltimore County
Matt Gaston
Marie desJardins
Agent-Organized Networks 2
Overview
Introduction and Motivation
Team Formation
Agent-Organized Networks
Experimental Results
Related Projects: Connections Model AONs AONs for Production and Exchange Stable Team Formation
Future Work and Conclusions
Agent-Organized Networks 3
Introduction and Motivation
Agent-Organized Networks 4
Introduction: Multi-Agent Systems
Agent: Autonomous, intelligent software system. Physical (robot, autonomous vehicle, mobile sensor) or virtual
(search, travel planning, trading / e-commerce, information retrieval)
MAS: “Community” of agents – competitive or cooperative Connections form a “social network” of agents
Agent-Organized Networks 5
Why Adapt?
Multi-agent systems are growing in popularity and size Technologies like the Semantic Web support the
deployment and evolution of large-scale, dynamic multi-agent systems
Agent cognitive capacities are limited, preventing all agents from knowing/interacting with all other agents
Previous findings suggest that network structure plays an essential role in understanding team formation dynamics in multi-agent systems
Identifying the “best” network structure is difficult or impossible to do a priori
Solution: Agent-Organized Networks
Agent-Organized Networks 6
Team Formation[AAMAS 2005]
Agent-Organized Networks 7
Multi-Agent Team Formation Model
Agents must form teams to complete tasks
Agent states: Uncommitted Committed Active
Tasks are advertised to the network of agents
A valid team: Connected path in network Task skill requirements met Formed within time
constraints
1 1
22
2
22
3
1, 1, 2, 3
2, 2
1, 2, 2, 2, 4
4
Agent-Organized Networks 8
Multi-Agent Team Formation Model
Some model details Parameters
Number of agents: N Skill diversity: Task introduction interval: Team/task size: T Advertisement duration: Task duration: Network structure
Organizational Efficiency# of tasks successfully completed
total # of tasks advertisedefficiency =
1 1
22
2
22
3
4
Agent-Organized Networks 9
Team Joining Strategy
With some initiation probability,start a new team if needed:
Always join a team if it’s already beenstarted, and it needs your skill.
Considering each task in random order...
Agent-Organized Networks 10
Agent-Organized Networks
Agent-Organized Networks 11
Agent-Organized Networks
Definition: An agent-organized network (AON) is an organizational network structure, or agent-to-agent interaction topology, that is the result of local rewiring decisions made by the individual agents in a networked multi-agent system.
Design considerations: Local perception of global performance Adaptation triggers Rewiring strategies
Evaluation metrics: Learning rate Stability Structural properties of resulting networks
Agent-Organized Networks 12
Structure-Based Adaptation
Adapt based on preferential attachment Natural network formation process that leads to scale-free networks
Adaptation trigger (random): Probability of adaptation for each uncommitted agent: 1/N
Rewiring strategy: Disconnect from a random neighbor Connect to some neighbor’s neighbor with probability
Agent-Organized Networks 13
Performance-Based Adaptation
Adaptation trigger: Adapt if performance drops below neighbors’ average performance:
Rewiring strategy: Drop the lowest-performing neighbor:
Add a connection to the highest-performing neighbor ak of the highest-performing neighbor al:
Agent-Organized Networks 14
Results
Agent-Organized Networks 15
Experimental Setup
Initial network structure: Random geometric graph Randomly place agents in a unit square Connect agents that are closer than d units apart Use the minimal d that guarantees all neighbors have at least one
edge
Run team formation with no adaptation to establish baseline Run with each adaptation strategy separately Results are an average of 50 runs
Agent-Organized Networks 16
Results: Summary
Significant performance improvement (over baseline) for both AON methods
Agent-Organized Networks 17
Stability of Networks
Structure-based AONs outperform performance-based AONs, but result in substantially more rewirings
Performance-based AONs are more efficient (“better value” if adaptation cost is in similar units to performance measure)
Agent-Organized Networks 18
Evolution of the Network: Structure-Based
Converges to a network with hub structure and short average path length
Agent-Organized Networks 19
Evolution of the Network: Performance-Based
Convergence to short-average-path-length structure happens more slowly
Qualitatively similar structure to strategy-based (but in this case not by design!)
Agent-Organized Networks 20
Connections Model AONs[AAAI 2005 Workshop on Multi-Agent Learning]
Agent-Organized Networks 21
The (Symmetric) Connections Model
Symmetric when ij = and cij = c for all i and j
0 < < 1 is the value of a relationship, discounted by distance
c is the cost of a direct connection
(Jackson & Wolinsky 1996; Jackson 2002)
Agent-Organized Networks 22
Dynamic Network Formation in SCM
Based on pairwise stability (Watts 2001): At each iteration:
Two agents meet (are selected) at random (synchronous) If they have a connection, they remove the connection if at least one
of them benefits -- unilateral deletion If the do not have a connection, they add a connection if it is
mutually beneficial -- bilateral creation
But . . .
Agent-Organized Networks 23
Experiment: Watts Dynamic Network Formation
= 0.9, c = 0.8, optimal = 7878.42
Agent-Organized Networks 24
A (Simple) Multi-Agent Learning Approach
Goals: Eliminate need for “global” knowledge Eliminate need for “global” computation Maintain bilateral network formation (agents agree to create link) Follow dynamic network formation process of Watts On-line learning
Approach Stateless Q-Learning (Claus & Boutilier 1998) A = { add, delete, nothing }
Agents add connection if both have largest Q value for add (bilateral) Agents remove connection if one has largest Q value for delete (unilateral) Reinforcement signal comes from omniscient oracle (!)
Agent-Organized Networks (AONs) “Distributed Annealing”
Agent-Organized Networks 25
Experiment: Learning to Form Networks
Adaptive Learning Rate: Win or Lose Fast (WoLF) (Bowling & Veloso 2002)
= 0.9, c = 0.8, optimal = 7878.42
Agent-Organized Networks 26
Experiment: Adding an Unselfish Agent
= 0.9, c = 0.8, optimal = 7878.42
Agent-Organized Networks 27
AONs for Production and Exchange[AAAI 2005]
Agent-Organized Networks 28
A Model of Production and Exchange
n agents in an artificial economy with two goods
Each agent i possesses g1i units of good 1 and g2
i units of good 2
Each agent is a producer of either good 1 or good 2
At each iteration of the model, the agents are selected in random order and choose between initiating trade with another agent or producing their respective good in order to maximize utility
Agent utility:
(Wilhite 2001: 2003)
fully rational behavior
Agent-Organized Networks 29
Push Referral AON Strategies
Random referral: agent selected randomly from Nj(i)
Degree referral:
Production referral:
Definition: Assuming that agent i is adapting its connection to agent j, a push referral is a local rewiring by i from j to an agent in Nj(i)
Agent-Organized Networks 30
Results
production referral
degree referral
random selection
n = 400, q = 30, = 0.05= = 0.1= = 0.1initialized values to 1
Agent-Organized Networks 31
Stable Team Formation[AAAI 2004 Workshop on Team/Coalition Formation]
Agent-Organized Networks 32
Economic Model of Team Formation
Share-based scheme for pay-off distribution Team’s revenue is stored in “team account” Team members get shares for joining and working Share value = team account / # outstanding shares
Agents bound to the team by a contract Joining Shares, Sjoin : sign-on bonus
Commission, Scomm : shares given to the agent for every task completed by the team in which the agent actively participates
Dividend, Sdiv : shares given to the agent for every task completed by the team in which the agent does not participate
(Dividend < Commission) Penalty, p : the amount to be paid to the team when leaving the team
Agent-Organized Networks 33
Results: Effect of Deadlines
Agent-Organized Networks 34
Results: Stable vs. Dynamic Agents
Agent-Organized Networks 35
Conclusions and Future Work
Summary: AONs based only on local knowledge can improve team
formation in networked MAS AON ideas can also be applied to other MAS domains and
models Stability can be achieved through a contractual model of team
formation
Future Work: Quantitative analysis of post-adaptation network structures Learning individual agent team selection strategies
[JAAMAS 2006] Skill placement and replacement for dynamic team formation
Agent-Organized Networks 36