© 2005 Erlang Training and Consulting Ltd

A Virtual World Distributed ServerA Virtual World Distributed Serverdeveloped indeveloped in Erlang Erlang as a Tool for analysing Needs as a Tool for analysing Needs

of Massivelyof Massively Multiplayer Multiplayer Online Game Servers Online Game Servers

ErlangErlang/OTP User Conference/OTP User ConferenceStockholm on November 10, 2005Stockholm on November 10, 2005

Michał Ślaskimichal@erlang-consulting.com

Page 2 © 2005 Erlang Training and Consulting Ltd

ContentsContents

•• MassivelyMassively Multiplayer Multiplayer Online Games Online Games•• Problems to solveProblems to solve•• PrototypingPrototyping•• UsingUsing Erlang Erlang/OTP for the prototype/OTP for the prototype•• Client and server applicationsClient and server applications•• Determining the maximum number of playersDetermining the maximum number of players•• Architectures with single and double-function nodesArchitectures with single and double-function nodes•• Synchronisation of the game’s stateSynchronisation of the game’s state•• ConclusionsConclusions

Page 3 © 2005 Erlang Training and Consulting Ltd

MassivelyMassively Multiplayer Multiplayer Online Games Online Games

•• Significant increasing in number of active subscribersSignificant increasing in number of active subscribers•• Fast growing online games marketFast growing online games market

Tota

l act

ive s

ub

scri

bers

1997 1998 1999 2000 2001 2002 2003 2004 20050

1 000 000

2 000 000

3 000 000

4 000 000

5 000 000

6 000 000

7 000 000

8 000 000

9 000 000

Source: IGDA, "2004 Persistent Worlds White Paper"Source: MMOGCHART.COM

Page 4 © 2005 Erlang Training and Consulting Ltd

Problems to solveProblems to solve

•• Big amount of data to processBig amount of data to process

•• Frequent synchronisationFrequent synchronisation– acceptable delay 150 - 500 ms

•• Adaptation to the connection qualityAdaptation to the connection quality

•• Limited Limited opportunity for full topportunity for full testing with thousands ofesting with thousands ofplayers during early phase of developmentplayers during early phase of development

Page 5 © 2005 Erlang Training and Consulting Ltd

PrototypingPrototyping

•• Experimenting with the functionalExperimenting with the functionaland technical aspectsand technical aspects

•• Determining the scalability of the architectureDetermining the scalability of the architecture

•• Testing new solutionsTesting new solutions

Page 6 © 2005 Erlang Training and Consulting Ltd

UsingUsing Erlang Erlang/OTP for the prototype/OTP for the prototype

•• Open sourceOpen source•• Compatible with various platformsCompatible with various platforms•• Concurrent oriented programmingConcurrent oriented programming

– one process for every connected player

•• Efficient internal communicationEfficient internal communication•• Supporting distributed applicationsSupporting distributed applications•• Distributed databaseDistributed database Mnesia Mnesia•• Distributed load testing tool IDX-TsunamiDistributed load testing tool IDX-Tsunami

Page 7 © 2005 Erlang Training and Consulting Ltd

Client applicationClient application

•• Java + Java3DJava + Java3D•• Actions:Actions:

– moving– collecting objects– magic– chat

•• Player’s statePlayer’s statekept on the serverkept on the server

Page 8 © 2005 Erlang Training and Consulting Ltd

Server applicationServer application

•• The game terrainThe game terraindivided into zonesdivided into zones

•• Players placed within thePlayers placed within thesame zone are processedsame zone are processedon the same nodeon the same node

•• Server distributed on aServer distributed on acluster of machinescluster of machines

Page 9 © 2005 Erlang Training and Consulting Ltd

Determining the maximum number of usersDetermining the maximum number of users

time of the test [s]

net

wor

k tr

affi

c [k

bits

/ s

ec]

0 20 40 60 80 100 120 140 1600

100

200

300

400

500

600

700

800overload

LegendOutgoing network traffic

time of the test [s]

tran

sact

ion

tim

e [m

sec]

0 20 40 60 80 100 120 140 1600

150

300

450

600

750

900

1 050

1 200

overload

LegendCollecting object transaction

time of the test [s]

nu

mbe

r of

pla

yers

0 20 40 60 80 100 120 140 1600

20406080

100120140160180

122 playersThe serverwasoverloadedin the 60thsecond ofthe test

Page 10 © 2005 Erlang Training and Consulting Ltd

Architectures with double-function nodesArchitectures with double-function nodes

number of nodes processing network traffic and player's actions

nu

mb

er

of

pla

yers

0 1 2 3 4 5 6 7 8 9 10100

200

300

400

500

600

700

800

900

Legendmaximum number of players that can be handled by the architecture

Page 11 © 2005 Erlang Training and Consulting Ltd

Architectures with single-function nodesArchitectures with single-function nodes

number of nodes processing player's actions (2 network nodes)

nu

mb

er

of

pla

yers

CP

U u

tilisa

tion

[%

]

0 1 2 3 4 5 6100 0

200 20

300 40

400 60

500 80

600 100

700 120

Legendmaximum number of playersaction processing nodes' CPU utilisationnetwork processing nodes' CPU utilisation

Page 12 © 2005 Erlang Training and Consulting Ltd

Synchronising issueSynchronising issueClient A Client BServer

Try to pick up a mushroom

Try to pick up a mushroom

Action status - OK

mushroom collectedby client A

Action status - FAIL

Page 13 © 2005 Erlang Training and Consulting Ltd

DesynchronisationDesynchronisation of the game’s state of the game’s state

•• ServerServeris notis notoverloadedoverloaded

•• MoreMoresuccessfulsuccessfulactionsactions

time of the test [s]

nu

mb

er

of

tran

sact

ion

s /

nu

mb

er

of

pla

yers

0 100 200 300 400 500 600 700 800 900 1 0000

0,01

0,02

0,03

0,04

0,05

0,06

0,07

0,08

0,09

0,1Legend

successful actionsnot successful actionsy=0.04855 (successful)y=0.0009119 (not successful)

Page 14 © 2005 Erlang Training and Consulting Ltd

DesynchronisationDesynchronisation of the game’s state of the game’s state

•• Server isServer isoverloadedoverloaded

•• MoreMoreactionsactionsfailfail

time of the test [s]

nu

mb

er

of

tran

sact

ion

s /

nu

mb

er

of

pla

yers

0 100 200 300 400 500 600 700 800 900 1 0000

0,005

0,01

0,015

0,02

0,025

0,03

0,035

0,04

0,045

0,05Legend

successful actionsnot successful actionsy=0.04855 (successful)y=0.0009119 (not successful)

Page 15 © 2005 Erlang Training and Consulting Ltd

ConclusionsConclusions

•• ErlangErlang/OTP is appropriate for developing prototypes/OTP is appropriate for developing prototypesof the distributed MMOG systemsof the distributed MMOG systems

•• Rapid development of the prototype allowsRapid development of the prototype allowsexamining different algorithms in a relatively shortexamining different algorithms in a relatively shortperiod of timeperiod of time

•• Future workFuture work– algorithms not dependent on the player’s position

Page 16 © 2005 Erlang Training and Consulting Ltd

ContentsContents

•• MassivelyMassively Multiplayer Multiplayer Online Games Online Games•• Problems to solveProblems to solve•• PrototypingPrototyping•• UsingUsing Erlang Erlang/OTP for the prototype/OTP for the prototype•• Client and server applicationsClient and server applications•• Determining the maximum number of playersDetermining the maximum number of players•• Architectures with single and double-function nodesArchitectures with single and double-function nodes•• Synchronisation of the game’s stateSynchronisation of the game’s state•• ConclusionsConclusions