V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
ESSLLI 2018 courseLogics for Epistemic and Strategic Reasoning
in Multi-Agent SystemsLecture 3: Logics for strategic reasoning
with complete information
Valentin GorankoStockholm University
ESSLLI 2018August 6-10, 2018
Sofia University, Bulgaria1 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Tentative outline of the lecture
• Agents and multi-agent systems (MAS),
• Multi-agent transition systems and concurrent game models
• The multi-modal logic of coalitional strategic abilities CL.
• Axiomatization of the validities of CL.
• Logical decision problems for CL. Model checking.
• Some extensions of CL
2 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Introduction: Agents and multi-agent systems
I Agents:
� relatively autonomous.
� have knowledge/information: about the system, themselves, and theother agents (incl. the environment).
� have abilities to perform certain actions.
� have goals, and can act in their pursuit.
� can plan their actions ahead and can execute plans (strategies).
� Communicate (exchange information) and cooperate with other agents.
I Multi-agent system (MAS): a set of agents acting in a commonframework (’system’), in pursuit of their goals, by following individual orcollective strategies.
Examples: open systems, distributed systems, concurrent processes,computer networks, social networks, stock markets, etc.
3 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Multi-agent transition systems intuitively
� Agents (players) act in a common environment (the “system”) bytaking actions in a discrete succession of rounds.
� At any moment the system is in a current state.
� At the current state all players take simultaneously actions, eachchoosing from a set of available actions.
� The resulting collective action effects a transition to a successor state,where the same happens, resulting in a new transition, etc.
This dynamics is captured by a multi-agent transition system.
4 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Example: single-agent transition systemA robot pushing a trolley along a track:
s0
s2 s1
s4 s3
push+
push−
wait
push−
push+
wait
wait
park
push−
push+
wait
wait
park
5 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Example: a two-agent transition systemTwo robots, Yin and Yang, are pushing a trolley along a track. Yin can onlypush clockwise and Yang can only push anticlockwise, with the same force.
s0
s2 s1
s4 s3
(push,wait)
(wait,push)
(wait,wait)(push,push)
(wait,push)
(push,wait)
(wait,wait)(push,push)
(wait,wait)
(park,push)(park,wait)
(wait,push)
(push,wait)
(wait,wait)(push,push)
(wait,wait)
(push,park)(wait,park)
6 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Single-agent vs multi-agent transition systems
Ignoring the agents, the two transition systems in the examples are thesame.
However:the abilities of the robot to cause transitions in the 1st example are notthe same as the abilities of any of the robots in the 2nd one.
The robot in the first case has full control on the transitions in thesystem, whereas none of the robots in the two-robot case does.
7 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
A game-theoretic perspectiveon multi-agent transition systems
Agents do not just take actions at every state of the system.
They play games.
The outcomes of these games are, inter alia, transitions to other games,etc., producing (possibly) infinite plays.
Thus, multi-agent transition systems can also be regarded as concurrentgame models.
8 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
A 2-player strategic game
RowColumn
L M R
U (3, 3) (0, 4) (2, 1)
D (4, 0) (1, 1) (0, 2)
9 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
A 2-player strategic game form
RowColumn
L M R
U s11 s12 s13
D s21 s22 s23
10 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Addendum: Multi-player Strategic Game Forms
A normal (strategic) game form is a tuple
〈A,W , {Acti}i∈A, out〉
where:
• A is a (finite) set of agents (players);
• W is a set of possible outcomes;
• Acti is a set of actions (moves, strategies) for player i ∈ A;
• out :∏
i∈A Acti →W is the outcome function.
Normal (strategic) game: strategic game form with payoff outcomes.
11 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Concurrent Game Models intuitively
Concurrent game structure (CGS): structure consisting of a set of gamestates St such that every state is associated with a strategic game formwith outcomes being again states in St.
CGS model successive plays of strategic games, as follows:
- at every given state all players take actions simultaneously,each choosing from a set of available actions;
- the collective action effects a transition into a successor state,determined by an outcome function;
- then the same happens from that successor state, etc.
Concurrent game model (CGM): a CGS plus a labeling of all states withsets of atomic propositions (true at the respective states).
12 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
A fragment of a concurrent game model
A = {Row,Col}Prop = {p, q}
{p}
s1
s1
{p}s2
{q}
s3
{p, q}s4
{}
13 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Concurrent Game Models formally
〈A,St,Act, act, out,Prop, L〉where:• A = {a,b, . . .} is a finite set of agents (players);
• St is a set of game states;
• Act is a set of possible actions;
• act : St→ (A→ P(Act)) – mapping assigning at every state s toevery agent i a set act(i, s) of actions available to i at s.
For any state s denote by Σs ⊆ ActA the set of all action profiles –tuples of actions, one for each agent – that are available at s.
• out : St→ (ActA− → St) is a global outcome function,assigning for every s ∈ St and action profile σ ∈ Σs ,the successor (outcome) state out(s, σ).
• Prop is the set of atomic propositions.
• L : St→ P(Prop) is the labeling (state description) function.14 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
The two-robot example as a concurrent game model
s0
s2 s1
s4 s3
(push,wait)
(wait,push)
(wait,wait)(push,push)
(wait,push)
(push,wait)
(wait,wait)(push,push)
(wait,wait)
(park,push)(park,wait)
(wait,push)
(push,wait)
(wait,wait)(push,push)
(wait,wait)
(push,park)(wait,park)
• A = {Yin,Yang}; St = {s0, s1, s2, s3, s4}; Prop = {pos0, pos1, pos2, pos3, pos4}.
• L : St → P(Prop) defined by L(si ) = {posi}, for i = 0..4.
• Act = {push,wait, park}.
• action function: as on the figure.
• outcome function: as on the figure.15 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
The logic for coalitional strategic reasoning CL
16 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
The logic for coalitional strategic reasoning CL
Coalitional Logic (CL) introduced by Marc Pauly ca. 2000.
CL extends the classical PL with coalitional strategic modal operators[C ], for any coalition of agents C .
Formulae of CL:ϕ := p | ¬ϕ | ϕ1 ∨ ϕ2 | [C ]ϕ
The intuitive reading of [C ]ϕ:
“The coalition C has a joint actionthat ensures an outcome (state) satisfying ϕ,regardless of how the other agents act.”
17 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Expressing properties in CL: some examples
We will write [i] instead of [{i}].
[i]Win→ ¬ [A\{i}]¬Win
If the agent has a strategy to win the game then the coalition of all otheragents cannot prevent him from winning.
¬ [Yin]Goal ∧ ¬ [Yang]Goal ∧ [{Yin,Yang}]Goal
None of the agents Yin and Yang has an action ensuring an outcomesatisfying Goal, but they both have a joint action ensuring an outcomesatisfying Goal.
[A] ([B] GoalB → ¬ [C ] GoalC)
The coalition A has a joint action to ensure at the outcome state that ifthe coalition B has a joint action to achieve its goal thenthe coalition C does not have a joint action to achieve its goal.
18 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Formal semantics of CL
Consider a CGM M = 〈A, St,Act, act, out,Prop, L〉.Given a coalition C ⊆ Agt, a joint action for C in M is a tuple ofindividual actions σC ∈ ActC . For any such joint action and state s ∈ Ssuch that σC is available at s, we define its set of possible outcomes:
Out[s, σC ] = {u ∈ S | ∃σ ∈ Σs : σ|C = σC and out(s, σ) = u}
where σ|C is the restriction of σ to C .
Truth of a CL-formula ψ at a state s of a CGM M, denoted M, s � ψ,is defined by structural induction on formulae, via the clauses:
• M, s |= p iff s ∈ V (p), for p ∈ AP;
• M, s |= ¬φ iff M, s 6|= φ.
• M, s |= φ1 ∨ φ2 iff M, s |= φ1 or M, s |= φ2.
• M, s |= [C ]φ iff there exists a joint action σC available at s,such that M, u |= φ for each u ∈ Out[s, σC ].
19 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Truth of CL formulae in a CGM: example 1
A = {row, col}AP = {p, q}
{p}
s1
s1
{p}s2
{q}
s3
{p, q}s4
{}
s1 |= p ∧ [col] p s1 |= ¬ [row] q s1 |= ¬ [col] q
s1 |= [row, col] q s1 |= [row, col] (p ∧ q) s1 |= [row, col] (¬p ∧ ¬q)
20 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Truth of CL formulae in a CGM: example 2
s0
s2 s1
s4 s3
(push,wait)
(wait,push)
(wait,wait)(push,push)
(wait,push)
(push,wait)
(wait,wait)(push,push)
(wait,wait)
(park,push)(park,wait)
(wait,push)
(push,wait)
(wait,wait)(push,push)
(wait,wait)
(push,park)(wait,park)
M, s0
?
|= [Yin] pos1 No. M, s0
?
|= [Yang] pos1 No.
M, s0
?
|= [Yin,Yang] pos1 Yes. M, s0
?
|= [Yang]¬ [Yin] pos3 Yes.
M, s0
?
|= [Yang] [Yang] ¬pos1 Yes. M, s0
?
|= [Yang] [Yin] pos3 No.
21 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Validity and satisfiability in CL
A CL formula φ is:
• (logically) valid if M, s � φ for every CGM M and a state s ∈M.
• satisfiable if M, s � φ for some CGM M and a state s ∈M.
22 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Axiomatizing the validities of CL
Pauly (2000) obtained a complete axiomatization of CL, extending theclassical propositional logic PL with the following axioms and rule:
• A-Maximality: ¬ [∅] ¬ϕ→ [A] ϕ
• Safety: ¬ [C ] ⊥
• Liveness: [C ] >
• Superadditivity: for any C1,C2 ⊆ A such that C1 ∩ C2 = ∅:
([C1] ϕ1 ∧ [C2] ϕ2)→ [C1 ∪ C2] (ϕ1 ∧ ϕ2)
• [C ] -Monotonicity Rule:
ϕ1 → ϕ2
[C ] ϕ1 → [C ] ϕ2
23 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Logical decision problems in CL
I Local model checking: given an CL formula ψ, a finite CGM M and astate s ∈M, determine whether M, s � ψ.
I Global model checking: given an CL formula ψ and a finite CGM M,determine extension of ψ in M, i.e. the set
‖ψ‖M = {s ∈ St | M, s � ψ}
I Satisfiability testing: given an CL formula ψ, determine whether ψ issatisfiable, i.e., whether M, s � ψ for some CGM M and a state s ∈M.
I Constructive satisfiability testing: given an CL formula ψ, determinewhether ψ is satisfiable, and if so, construct a CGM M and a states ∈M such that M, s � ψ.
24 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
The operator Pre
Consider a CGM M = 〈A, St,Act, act, out,Prop, L〉
Given a coalition C ⊆ A, a state s, and a set X ⊆ St, we say that C iseffective for X at s, if C has a joint action at s that guarantees theoutcome to be in X , no matter how the remaining agents act at s.
We define Pre(M,C ,X ) as the set of states at which the coalition C iseffective for X .
Formally:
Pre(M,C ,X ) := {s ∈ St | ∃αC ∀αA\C out(s, αC , αA\C ) ∈ X}
where αC denotes a tuple of actions, one for each agent in C .
In particular, for a formula ϕ, Pre(M,C , ‖ϕ‖M) is precisely ‖ [C ]ϕ‖M.
25 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
The operator Pre: examplesConsider the CGM M below with agents 1 and 2, and actions a, b.
s1{p}
s2{p,q}
s3{q}
s4{}
(a,a)
(a,b)(b,a)
(b,b)
(b,a)
(b,b)
(a,a)(a,b)
(b,a)(b,b)
(a,a)(a,b)
(b,a)(a,a)(a,b)(b,b)
Pre(M, {1}, {s2, s3}) = {s2}; Pre(M, {2}, {s2, s3}) = {s2};Pre(M, {1}, {s1, s2}) = {s1, s2, s3, s4}; Pre(M, {2}, {s1, s2}) = {s1, s4};Pre(M, ∅, {s1, s2}) = ∅; Pre(M, ∅, {s1, s4}) = {s3, s4};Pre(M, {1, 2}, {s1}) = {s1, s2, s3, s4}; Pre(M, {1, 2}, {s2}) = {s1, s2}.
26 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
PTime model-checking algorithm for CL: a sketch
Given an CGM M and a CL formula ϕ, the algorithm computes theextension ‖ϕ‖M inductively on the structure of ϕ:
‖p‖M = {s ∈ St | p ∈ L(s)} for atomic propositions p,
‖¬ϕ‖M = St \ ‖ϕ‖M,
‖ϕ ∨ ψ‖M = ‖ϕ‖M ∪ ‖ψ‖M,
‖ [C ]ϕ‖M = Pre(M,C , ‖ϕ‖M).
Each of these steps is computed in time linear in the size(number of states + number of transitions) of the model.
27 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
A recursive algorithm for global model checking of CL
1: procedure GlobalMC(CL)(M, ϕ)
2: case ϕ = p ∈ Prop : return ‖ϕ‖M = {s ∈ St | p ∈ L(s)}
3: case ϕ = ¬ψ : return ‖ϕ‖M = St \ ‖ψ‖M4: case ϕ = ψ1 ∨ ψ2 : return ‖ϕ‖M = ‖ψ1‖M ∪ ‖ψ2‖M5: case ϕ = [C ]ψ : return ‖ϕ‖M = Pre(M,C , ‖ψ‖M)
6: end procedure
28 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Global model checking of CL formulae: exercises
s1{p}
s2{p,q}
s3{q}
s4{}
(a,a)
(a,b)(b,a)
(b,b)
(b,a)
(b,b)
(a,a)(a,b)
(b,a)(b,b)
(a,a)(a,b)
(b,a)(a,a)(a,b)(b,b)
‖ [1] q‖M = {s2} ‖p → [1] p‖M = {s1, s2, s3, s4}
‖p ∨ [2] p‖M = {s1, s2, s4} ‖ [1] [2] p‖M = {s3, s4}
‖ [∅]¬q‖M = {s3, s4} ‖q ∨ ¬ [∅]¬q‖M = {s1, s2, s4}
‖ [1, 2] (p ∧ q)‖M = {s1, s2} ‖ [1, 2] (p ∧ ¬q)‖M = {s1, s2, s3, s4}NB: one answer is wrong. A surprise reward for the first correct correction!29 of 30
V Goranko
Strategic games and multi-player game models Multi-agent logics for strategic reasoning
Some extensions of Coalition LogicCoalition Logic is nice but not very expressive.
In particular, it can only express strategic abilities of one coalition at atime, treating the remaining agents as adversaries.
Two recent extensions (Sebastian Enqvist and VG, AAMAS 2018):
SF Socially friendly coalitional operator SF[C ] (φ;ψ1, . . . , ψk), meaning: “C has a joint action σC thatguarantees φ and enables the complementary coalition C to realiseany one of the goals ψ1, . . . , ψk by a suitable joint action”.
GIP Group-interests-protecting coalitional operator GIP〈[C1 . φ1, ...,Cn . φn]〉, meaning: “There is an action profile σ for thecoalition C1 ∪ ...∪ Cn such that for each i , the restriction of σ to thecoalition Ci is an action profile that forces φi”.
Also, CL can only express strategic abilities, for achieving immediateobjectives, but not for long-term, temporal objectives.
A temporal logic naturally extending CL is coming up next.30 of 30