A Simple Model for Sequences of Relational State Descriptions
EC
ML
- 08
Ingo Thon, Niels Landwehr, Luc De Raedt
Thursday 18. September 2008
EC
ML
- 08
Outline
Introduce a rarely investigated setting
Sequences where states are relational interpretations (simplified: graphs)
Define a simple model allowing for efficient inference techniques
Experimental evaluation shows1.able to capture real worlds domain dynamics
2.scalability
EC
ML
- 08
Sequences of Relational State Descriptions
Interactions of (human/artificial) agentsBusiness applications
Computer games
EC
ML
- 08
Sequences of Relational State Descriptions
Interactions of (human/artificial) agentsBusiness applications
Computer games
Computer network modeling
EC
ML
- 08
Sequences of Relational State Descriptions
Interactions of (human/artificial) agentsBusiness applications
Computer games
Computer network modeling
Music
EC
ML
- 08
Sequences of Relational State Descriptions
Interactions of (human/artificial) agentsBusiness applications
Computer games
Computer network modeling
Music
Traffic Modeling
EC
ML
- 08
Sequences of Relational State Descriptions
Interactions of (human/artificial) agentsBusiness applications
Computer games
Computer network modeling
Music
Traffic Modeling
....
EC
ML
- 08
Sequences of Relational State Descriptions
Interactions of (human/artificial) agentsBusiness applications
Computer games
Computer network modeling
Music
Traffic Modeling
....
EC
ML
- 08
Sequences of Relational State Descriptions
Computer games
EC
ML
- 08
Welcome to Travian
Travian: A massively multiplayer real-time strategy browser gameCommercial game run by TravianGames GmbH
~3.000.000 players spread over different “worlds”
~25.000 players in one world
Classical strategy gamePlayed on a grid map
Cities, resources, buildings, technology, armies
Attacks/Conquests, trading
Alliances: Boxes/Color-coded
Players: Diamonds
Cities: Circles
Fat arrows Conquests
GreeksArchilleus
Menelaos
Paris
Troys
Troy
Sparta
Mykena
Pelion
EC
ML
- 08
City/Player/Alliance Graph
Alliances: Boxes/Color-coded
Players: Diamonds
Cities: Circles
Fat arrows Conquests
GreeksArchilleus
Menelaos
Paris
Troys
Troy
Sparta
Mykena
Pelion
~10 alliances~200 players~600 cities
border
border
border
border
Alliance 1
Alliance 2
Alliance 3
Alliance 4
Alliance 5
Alliance 6
Alliance 7
Alliance 9
Alliance 10
Alliance 11
P 1
742
885
809
888
971
753
647
650
460
706
P 2
1010
994
978
P 3
1029
846
1009
991645
P 5
900
1002
647
955
968
1036
P 6
1025
701
647
P 7
984
867
831
931
930
937
828
997
789
845
697
862
883
735
630
631
P 8
971
949
970
977
P 12
1002
1003
854
P 13
626
606
P 14
798
709
P 15
885
969
721
953
P 16
994
745886
P 17
922
816
P 18
961
937
932942
959
1004
869
920
896
P 19
1051
1023
972
569
759
852
859
P 21
750
776
918
948
P 22
833
813
885
663
P 24
958 912
659
892840
609
854
P 26
767
P 28
915
P 30
941
872
P 33
872762
P 34
1030
825
P 36
904
1003
1028 1036
972 976
982
920
823
783
717
P 37
1033
919
900
766
P 38
732
856
P 40
707
657
P 41
951
984
811
765574
P 43
815
901
P 45
735
P 46
1005
863
P 47
991
826
638
P 50
902
877
887
P 51
994
1031
760
896
606
P 55
672
629
P 56
1046
1017
802
986
P 57
1008
946
940
941
1038
838
951
1020
922
10041015
900
861
P 59
667
903 559
P 61
1021
736
845
672
860
P 63
965
978
808
P 64
979 948
881
902
P 66
903
864947
891
811
710
643
P 67
526
435
P 68
657
636
592
P 70
946845
943
818
P 72
802
644
697
P 74
811
795
1007
839
931
784709
P 75
1080
976
813
1008
P 76
999
802
P 77
9171005
667
P 78
950
549
P 81
1010
984
961
839
1054
1044
905
1048
934
768
930
842
P 82
799
P 83
761
P 85
552
667
586
P 86
763
725
784
P 88
874
830769 746
P 89
862
601
P 91
1049
863
P 93
P 95
749
972
937
964
P 97
944
P 98
1006
P 99
919
P 100
851 1021
815
P 101
859818
772
P 105
922
872
776
856
826
P 106
762
P 107
750
P 111
992
1014
858
472
772
648 517
P 116
1002
986
592
P 120
922
976
968
916
799
732
777
661
P 121
1059
1029
980
998
962
769
P 122
1089
933
1001
P 123
799
P 124
10151063
957
954
P 125
826
869
920
634
P 126
1190
640
859
912P 127
810
806
924
849
880
625
731
P 128
898
P 129
648
1001
675
882
P 132
782
593
480
P 134
740
892
707
P 136
725
757
732
869
965
759
844
P 137
668
681
754
765
677
P 138
687
982
737
716
789
904P 139
806
982
615
P 140
948
964
770
P 141
876
832
885
909585
P 143
1033
700
P 144
895
P 146
1106
1068
P 150
1008
811
832
P 153
919
969
783
905
991
954
946
910
888804
596
P 154
926 884
P 156
760
947
625 611
P 158
922
P 159
965
P 167
873
858
762
847
833
785697
P 172
872
632
641
742
994
P 174
533582
639
P 176
605
P 177
649
746 731
P 179
742
887861
574
P 180
967
P 181
1036
1009
966
1039 1046
1020
683
668
1026
1030
P 183
916
863
716
P 185
965
968
994
844
854
803
P 187
934
P 189
1005
751
883
989
796
1008
1022
825
810
P 191
924
856
P 193
756
P 195
772
576
P 199
975
959
P 200
850
P 201
736
612
P 202
911
956
870
960
879
867
710
809
798
902
692
836
809
P 203
786
P 205
917
945
582
946
915
732
673
P 212
929
523698
701
566 585
P 213
623
P 217
1000
980
1071
978
1015
1066
940
P 220
948
943
792
P 221
806
794
704
625
587
P 222
899
813
790
P 223
795
P 224
901
791
477
P 228
709619
677
P 230
852
P 231
808
886
833
725
P 233
611
P 234
754
P 235
982
495
908
804
P 236
1230
784
591
617P 238
556
P 239
957
903
1046
1036
989
925
1055
1063
828
1046
878
P 240
1015
940
860
P 242
1069
909
956
1059 898
929
1035
820
892
1059
1039
950
997
853 797
929
965
757
916
P 244
745
724
808
718
P 246
781
787
776
939
851
EC
ML
- 08
City/Player/Alliance Graph
Alliances: Boxes/Color-coded
Players: Diamonds
Cities: Circles
Fat arrows Conquests
GreeksArchilleus
Menelaos
Paris
Troys
Troy
Sparta
Mykena
Pelion
~10 alliances~200 players~600 cities
border
border
border
border
Alliance 1
Alliance 2
Alliance 3
Alliance 4
Alliance 5
Alliance 6
Alliance 7
Alliance 9
Alliance 10
Alliance 11
P 1
742
885
809
888
971
753
647
650
460
706
P 2
1010
994
978
P 3
1029
846
1009
991645
P 5
900
1002
647
955
968
1036
P 6
1025
701
647
P 7
984
867
831
931
930
937
828
997
789
845
697
862
883
735
630
631
P 8
971
949
970
977
P 12
1002
1003
854
P 13
626
606
P 14
798
709
P 15
885
969
721
953
P 16
994
745886
P 17
922
816
P 18
961
937
932942
959
1004
869
920
896
P 19
1051
1023
972
569
759
852
859
P 21
750
776
918
948
P 22
833
813
885
663
P 24
958 912
659
892840
609
854
P 26
767
P 28
915
P 30
941
872
P 33
872762
P 34
1030
825
P 36
904
1003
1028 1036
972 976
982
920
823
783
717
P 37
1033
919
900
766
P 38
732
856
P 40
707
657
P 41
951
984
811
765574
P 43
815
901
P 45
735
P 46
1005
863
P 47
991
826
638
P 50
902
877
887
P 51
994
1031
760
896
606
P 55
672
629
P 56
1046
1017
802
986
P 57
1008
946
940
941
1038
838
951
1020
922
10041015
900
861
P 59
667
903 559
P 61
1021
736
845
672
860
P 63
965
978
808
P 64
979 948
881
902
P 66
903
864947
891
811
710
643
P 67
526
435
P 68
657
636
592
P 70
946845
943
818
P 72
802
644
697
P 74
811
795
1007
839
931
784709
P 75
1080
976
813
1008
P 76
999
802
P 77
9171005
667
P 78
950
549
P 81
1010
984
961
839
1054
1044
905
1048
934
768
930
842
P 82
799
P 83
761
P 85
552
667
586
P 86
763
725
784
P 88
874
830769 746
P 89
862
601
P 91
1049
863
P 93
P 95
749
972
937
964
P 97
944
P 98
1006
P 99
919
P 100
851 1021
815
P 101
859818
772
P 105
922
872
776
856
826
P 106
762
P 107
750
P 111
992
1014
858
472
772
648 517
P 116
1002
986
592
P 120
922
976
968
916
799
732
777
661
P 121
1059
1029
980
998
962
769
P 122
1089
933
1001
P 123
799
P 124
10151063
957
954
P 125
826
869
920
634
P 126
1190
640
859
912P 127
810
806
924
849
880
625
731
P 128
898
P 129
648
1001
675
882
P 132
782
593
480
P 134
740
892
707
P 136
725
757
732
869
965
759
844
P 137
668
681
754
765
677
P 138
687
982
737
716
789
904P 139
806
982
615
P 140
948
964
770
P 141
876
832
885
909585
P 143
1033
700
P 144
895
P 146
1106
1068
P 150
1008
811
832
P 153
919
969
783
905
991
954
946
910
888804
596
P 154
926 884
P 156
760
947
625 611
P 158
922
P 159
965
P 167
873
858
762
847
833
785697
P 172
872
632
641
742
994
P 174
533582
639
P 176
605
P 177
649
746 731
P 179
742
887861
574
P 180
967
P 181
1036
1009
966
1039 1046
1020
683
668
1026
1030
P 183
916
863
716
P 185
965
968
994
844
854
803
P 187
934
P 189
1005
751
883
989
796
1008
1022
825
810
P 191
924
856
P 193
756
P 195
772
576
P 199
975
959
P 200
850
P 201
736
612
P 202
911
956
870
960
879
867
710
809
798
902
692
836
809
P 203
786
P 205
917
945
582
946
915
732
673
P 212
929
523698
701
566 585
P 213
623
P 217
1000
980
1071
978
1015
1066
940
P 220
948
943
792
P 221
806
794
704
625
587
P 222
899
813
790
P 223
795
P 224
901
791
477
P 228
709619
677
P 230
852
P 231
808
886
833
725
P 233
611
P 234
754
P 235
982
495
908
804
P 236
1230
784
591
617P 238
556
P 239
957
903
1046
1036
989
925
1055
1063
828
1046
878
P 240
1015
940
860
P 242
1069
909
956
1059 898
929
1035
820
892
1059
1039
950
997
853 797
929
965
757
916
P 244
745
724
808
718
P 246
781
787
776
939
851
Can we model world dynamics?
How do players act, and why?
Can we predict how the world will evolve?
EC
ML
- 08
City/Player/Alliance Graph
EC
ML
- 08
World Dynamics
Can players’ actions be explained from “social context”?
Typical patterns conquer a city which is close
conquer a city of a player that has been attacked by alliance member
tendency of alliance to conquer whole areas on the map (clusters)
retaliate against attacks at an alliance level
change alliance if you are under attack
border
border
border
border
Alliance 2
Alliance 4
Alliance 6
P 2
9181090
931
779
977
835
781
9581087
808
701
P 3
838
947
1026
1081
833
1002987
827
994
663
P 5
1032
1026
1024
1049
905
926
P 6
986
712
985
920
877
807
P 7
895
959
P 10
824
EC
ML
- 08
World Dynamics
Can players’ actions be explained from “social context”?
Typical patterns conquer a city which is close
conquer a city of a player that has been attacked by alliance member
tendency of alliance to conquer whole areas on the map (clusters)
retaliate against attacks at an alliance level
change alliance if you are under attack
border
border
border
border
Alliance 2
Alliance 4
P 2
948
951
786
990
856
795
980
828
730
P 3898
803
860
964
1037
1085
925
689
10051007
899
1005
760
P 5
1051
1051
1040
860
774
1061
886
944
844
945
713
P 10
839
796838
EC
ML
- 08
World Dynamics
Can players’ actions be explained from “social context”?
Typical patterns conquer a city which is close
conquer a city of a player that has been attacked by alliance member
tendency of alliance to conquer whole areas on the map (clusters)
retaliate against attacks at an alliance level
change alliance if you are under attack
border
border
border
border
Alliance 2
Alliance 4
P 2
948
951
786
990
856
795
980
828
730
P 3898
803
860
964
1037
1085
925
689
10051007
899
1005
760
P 5
1051
1051
1040
860
774
1061
886
944
844
945
713
P 10
839
796838
Relations Important
➙Statistical Relational Learning
EC
ML
- 08
CPT-L
EC
ML
- 08
CPT-L
Standard SRL:Very general, expressive models
Not specifically tailored to sequences (of interpretations)
➡ No guarantee for tractability
EC
ML
- 08
CPT-L
Standard SRL:Very general, expressive models
Not specifically tailored to sequences (of interpretations)
➡ No guarantee for tractability
Generative probabilistic model for sequences of interpretations
Based on CP-Logic (Causal Probabilistic Logic)
Efficiency byFocussing on sequences
Assuming data fully observable
Use of efficient data structures (BDD) for inference
CPT-L:
Idea: Model is Markov Chain
EC
ML
- 08
CPT-L
P (I0, . . . IT |T ) = P (I0)!
t=1...T
P (It|It!1, T )
Idea: Model is Markov Chain
EC
ML
- 08
CPT-L
P (I0, . . . IT |T ) = P (I0)!
t=1...T
P (It|It!1, T )
Close
GreeksArchilleus
Menelaos
Paris
Troys
Troy
Sparta
Mykena
Pelion
C
C2
Owner Attacker
GreeksArchilleus
Menelaos
Paris
Troys
Troy
Sparta
Mykena
Pelion
P (It|It!1, T )
conquer a city which is close
EC
ML
- 08
CPT-Rules
border
border
border
border
Alliance 2
Alliance 4
Alliance 6
P 2
9231090
941
784
983
844
786
9661087
815
711
P 3
864
986
842
1032
1083
868
712
10021000
858
996
696
P 5
1039
1037
1030
1053
826
933
P 6
985
807
P 7
894
963
P 10
829
781828
b1, . . . bn ! h1 : p1 " . . . " hm : pm
GreeksArchilleus
Menelaos
Paris
Troys
Troy
Sparta
Mykena
Pelion
conquer a city which is closecity(C, Owner), city(C2, Attacker), close(C, C2)
EC
ML
- 08
CPT-Rules
border
border
border
border
Alliance 2
Alliance 4
Alliance 6
P 2
9231090
941
784
983
844
786
9661087
815
711
P 3
864
986
842
1032
1083
868
712
10021000
858
996
696
P 5
1039
1037
1030
1053
826
933
P 6
985
807
P 7
894
963
P 10
829
781828
cause (past)b1, . . . bn ! h1 : p1 " . . . " hm : pm
GreeksArchilleus
Menelaos
Paris
Troys
Troy
Sparta
Mykena
Pelion
Close
GreeksArchilleus
Menelaos
Paris
Troys
Troy
Sparta
Mykena
Pelion
C
C2
Owner Attacker
conquer a city which is close
EC
ML
- 08
CPT-Rules
border
border
border
border
Alliance 2
Alliance 4
Alliance 6
P 2
9231090
941
784
983
844
786
9661087
815
711
P 3
864
986
842
1032
1083
868
712
10021000
858
996
696
P 5
1039
1037
1030
1053
826
933
P 6
985
807
P 7
894
963
P 10
829
781828
cause (past) effect (future)
b1, . . . bn ! h1 : p1 " . . . " hm : pm
city(C, Owner), city(C2, Attacker), close(C, C2)!conquest(Attacker, C) : p " nil : (1# p)
Close
GreeksArchilleus
Menelaos
Paris
Troys
Troy
Sparta
Mykena
Pelion
C
C2
Owner Attacker
GreeksArchilleus
Menelaos
Paris
Troys
Troy
Sparta
Mykena
Pelion
EC
ML
- 08
Tasks
EC
ML
- 08
Tasks
Sampling: Random sequence of states given a theory
Inference: Probability of sequence of states
Parameter Estimation: Given a set of sequences of states, what is the most likely parameterization of a theory
Prediction: Given a sequence of states and a formula the probability of coming true in steps
I0, . . . , IT TP (I0, ..., IT | T )
I0, . . . , IT
I0, . . . , IT
P (It+d ! F |I0, . . . , IT , T )F
F d
EC
ML
- 08
Toy Example + Sampling
Toy Example: r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b
EC
ML
- 08
Toy Example + Sampling
Toy Example:
Sampling successor of state {a,b}:
r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b
EC
ML
- 08
Toy Example + Sampling
Toy Example:
Sampling successor of state {a,b}:1. Select rules applicable {r1,r2,r33}
r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b
EC
ML
- 08
Toy Example + Sampling
Toy Example:
Sampling successor of state {a,b}:1. Select rules applicable {r1,r2,r33}
2. For each rule select one successor {r1:a,r2:nil,r33:a}
r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b
EC
ML
- 08
Toy Example + Sampling
Toy Example:
Sampling successor of state {a,b}:1. Select rules applicable {r1,r2,r33}
2. For each rule select one successor {r1:a,r2:nil,r33:a}
3. yield successor state {a}
r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b
Model:
Naive Implementation of e.g..
EC
ML
- 08
Probability Calculation: Naive
r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b
P({a,b}|{a})P (St+1|St)
Model:
Naive Implementation of e.g..Enumerate all rules with
EC
ML
- 08
Probability Calculation: Naive
r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b
P({a,b}|{a})P (St+1|St)
St ! b1, . . . , bn
b1, . . . , bn ! h1 : p1 " . . . " hm : pm
Model:
Naive Implementation of e.g..Enumerate all rules with
Generate all “selections” s.t. for each rule one is select
Probability of selection is product of ‘s
{r1:a,r31:a}:0.48,{r1:a,r31:b}:0.16,{r1:a}:0.16, {r31:a}:0.12,{r31:b}:0.04,{}:0.02
EC
ML
- 08
Probability Calculation: Naive
r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b
P({a,b}|{a})P (St+1|St)
St ! b1, . . . , bn
b1, . . . , bn ! h1 : p1 " . . . " hm : pm
hi
pi
Model:
Naive Implementation of e.g..Enumerate all rules with
Generate all “selections” s.t. for each rule one is select
Probability of selection is product of ‘s
Probability of state is sum of probability of selections generating
{r1:a,r31:a}:0.48,{r1:a,r31:b}:0.16,{r1:a}:0.16, {r31:a}:0.12,{r31:b}:0.04,{}:0.02
EC
ML
- 08
Probability Calculation: Naive
r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b
P({a,b}|{a})P (St+1|St)
St ! b1, . . . , bn
b1, . . . , bn ! h1 : p1 " . . . " hm : pm
hi
pi
St+1
{a,b}:0.16,{a}:0.76,{b}:0.04,{}:0.04
Model:
Query: applicable rules r1, r31
Min. one:
Only one:
Generate:
Calculate probability by translating into BDD
EC
ML
- 08
Probability Calculation: Efficient Algorithm
r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b
(r1 : a ! r1 : nil) " (r31 : a ! r31 : b ! r31 : nil)"(r1 : a ! r1 : b) " (r31 : a ! r31 : b) " (r31 : a ! . . .
(r1 : a ! r31 : a) " r31 : b
P({a,b}|{a})
!1 = {r1:a, r2:b, r3:b}!2 = {r1:a, r2:b, r3:nil}
!4 = {r1:nil, r2:a, r3:b}!3 = {r1:a, r2:a, r3:b}
possible selections:
EC
ML
- 08
Efficient calculation
Efficient calculation using Dynamic Programing
r1:a
r1:
nil
r2:a
r3:b
r3:
nil
01
r2:b
r1:
nil
r2:a
r2:b r2:b
r3:b
r3:
nil
r3:
nil
!1 = {r1:a, r2:b, r3:b}!2 = {r1:a, r2:b, r3:nil}
!4 = {r1:nil, r2:a, r3:b}!3 = {r1:a, r2:a, r3:b}
possible selections:
EC
ML
- 08
Efficient calculation
Efficient calculation using Dynamic Programing
11
p3:b
p3:b + p3:nil
= 1
pr2:b +pr2:ap3:b
P ({a, b}|true) =pr1:a(pr2:b + pr2:ap3:b)+pr1:nilpr2:apr3:b
r1:a
r1:
nil
r2:a
r3:b
r3:
nil
01
r2:b
r1:
nil
r2:a
r2:b r2:b
r3:b
r3:
nil
r3:
nil
pr3:nil
EC
ML
- 08
Learning
Problem: Given observation e.g. {a},{a}Hidden Parameter: Which Rule generated which Element
EM-AlgorithmExpectation: Calculate Probability for each Selection, turning it into fully Observable Problem
Maximization: Set Parameters according to Probability of Selections
r1 : (p1,a :: a ! p1,nil :: nil) " ar31 : (p31,a :: a ! p31,b :: b ! p31,nil :: nil) " a, b
!1 = {r1 : a, r31 : a} p1,a · p31,a
!2 = {r1 : a, r31 : nil} p1,a · p31,nil
!3 = {r1 : nil, r31 : a} p1,nil · p31,nil
p!1,a =
p1,a · p31,a + p1,a · p31,nil
p1,a · p31,a + p1,a · p31,nil + p1,nil · p31,nil=
p1,a
p
EC
ML
- 08
Experiment Blocks-World: Setup
Goal: show that model scales with world complexity
Stochastic Blocks-World (stacks can collapse, stacking can fail,...)
Complexity increases with number of blocks
EC
ML
- 08
Experiment Blocks-World: Setup
Goal: show that model scales with world complexity
Stochastic Blocks-World (stacks can collapse, stacking can fail,...)
Complexity increases with number of blocks
c b
a
c ba a c b
-10000
-9000
-8000
-7000
-6000
-5000
-4000
-3000
-2000
-1000
0 2 4 6 8 10
Log-L
ikelih
ood
Iterations of the EM-Algorithm
10 blocks25 blocks50 blocks
0
2
4
6
8
10
12
10 20 30 40 50
runtim
e [m
inute
s] fo
r 10 ite
rations
Number of blocks
runtime
Blocks-World: convergence behaviour
-10000
-9000
-8000
-7000
-6000
-5000
-4000
-3000
-2000
-1000
0 2 4 6 8 10
Lo
g-L
ike
liho
od
Iterations of the EM-Algorithm
10 blocks25 blocks50 blocks
EC
ML
- 08
Experimental Evaluation I
Blocks-World: runtime behavior
EC
ML
- 08
Experimental Evaluation I
0
2
4
6
8
10
12
10 20 30 40 50
run
tim
e [
min
ute
s]
for
10
ite
ratio
ns
Number of blocks
runtime
EC
ML
- 08
Experiment Travian: Setup
Goal: show that model works on real world problems.
Logged the state of a “live” game server:~30.000 players / ~3 months /snapshot of the world every 24h
Projecting down to a subset of players/cities/alliancesfollowing 10 players (~50 cities) over a period of one month
30 such sequences extracted from distinct parts of the map
Learn a model from December data
Want to predict conquests(P,C) in given... for January and February data (up to five steps in future)
It+5 It
EC
ML
- 08
Experimental Evaluation II
Travian: predict player behavior (attack of a city)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
tru
e p
ositiv
e r
ate
false positive rate
CPT-L, k = 1CPT-L, k = 2CPT-L, k = 3CPT-L, k = 4CPT-L, k = 5
EC
ML
- 08
Experimental Evaluation II
Travian: predict player behavior (attack of a city)
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
1 2 3 4 5
Are
a u
nder
RO
C c
urv
e
Number of steps predicted (k)
January 2008February 2008
EC
ML
- 08
Conclusions
Sequences of relational state descriptions
CPT-L: simple model for sequences of relational state descriptions addresses:1.efficiency
2.rather than maximal expressivity
Experimental evaluation of complex relational sequences 1.able to capture domain dynamics
2.scalability
EC
ML
- 08
Thanks!