Date post: | 23-Jan-2015 |
Category: |
Technology |
Upload: | - |
View: | 715 times |
Download: | 0 times |
Thread Blocking Is Evil
Game Network Component Team
Kim, J.H.
CONCURRENCY
Why is Thread Blocking
a Concern?
“The Free Lunch is OVER.”
Herb SutterDecember 2004
Clock speedExecution optimizationCache
Cache
HyperthreadingMulticoreCache
Write Multithreaded Application
CONCURRENCY
Waiting in Line
Thread Blocking
Starvation
Deadlock
Starvation
Task A Task B
1: Lock 1: Sleep 0 msec
2: Sleep 1msec
3: Unlock
Pro
cess
ti
me(m
s)
# of Thread
1 2 3 4 5 6 7 8 9984986988990992994996998
1000
Task BTask A
DeadlockHold & Wait
Thread A Thread B
1A: Lock 1B: Lock
2A: Request 2B: Send Response
3A: Wait for Response 3B: Unlock
4A: Do something with Response
5A: Unlock
Thread A Thread B
1A: Lock 1B: Lock
2A: Request 2B: Send Response
3A: Wait for Response 3B: Unlock
4A: Do something with Response
5A: Unlock
My Prob-lem
Framework for Lobby-Room style
Game
Remote Procedure Call
Easy to write code be-cause it’s all contained
inside one function.
Game Client
LobbyRoom‘Enter Room’
RPC‘Authentication’
RPC
Game Client
LobbyRoom
‘Enter Room’
RPC func-tion body
‘Authenti-cation RPC
‘Function
body
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
Game Client
LobbyRoom
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
‘Enter Room’
RPC func-tion body
‘Authenti-cation RPC
‘Function
body
Game Client
LobbyRoom
‘Enter Room’ RPC
function body
‘Authenti-cation RPC
‘Function
body
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
Response DelayFrom Lobby Server
Game Client
LobbyRoom
‘Enter Room’ RPC
function body
‘Authenti-cation RPC
‘Function
body
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
Room Serverthroughput Decline
Response DelayFrom Lobby Server
What is theAxis of Evil?
Thread Block-
ing
Game Client
LobbyRoom
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
Inherent Limitation of RPC
Easy to write code be-cause it’s all contained
inside one function.
Can not split code be-cause it’s all contained
inside one function.
RPC::Result* JoinRoom(…){ … RPC::Result answer = RPC::SyncCall(“VerifyLobbyUserToken”, …); …. return result;}
What We Want
RPC Function Body
Non-Blocking
What is the Essence of Problems?
Request & Wait
Solution
Asynchronous
Programming
Request(callback, …)
Pros:Non-Blocking
Cons:Can’t Write Sequential CodeCan’t Use Stack VariableCan’t Split Programming Construct…
Too Difficult!!
Coroutine
Similar to Thread
Line of Execution
Stack&
Local Variable
But
Non-preemptive
Instruction:YieldYield BreakResume
Who does provide Coroutine?
C#ErlangHaskell
JavaScript(since 1.7)LuaPerl
Python(since 2.5)Ruby
…
What about C++?
On Your Own!
Windows: Fiber
Linux: getcontext/setcontextmakecontextswapcontext
Asynchronous Pro-gramming
+Coroutine
Non-Blocking&
Writing Sequential Code
Solution for Starva-
tion
Task A Task B
1: Lock 1: Sleep 0msec
2: Sleep 2msec
3: Unlock
Before
Task A Task B
Thread A 1: Sleep 0msec
1: Create & Resume Coroutine
2: Request Lock
3: Yield
Thread A’
4: Resume
5: Sleep 2msec
6: Yield Break
7: Unlock
Non-Blocking
Sequential Code
After
BeforePro
cess
ti
me(m
s)
# of Thread
1 2 3 4 5 6 7 8 9984986988990992994996998
1000
Task BTask A
AfterPro
cess
ti
me(m
s)
1 2 3 4 5 6 7 8 90
200
400
600
800
1000
1200
Task BTask A
# of Thread
Solution for
DeadlockHold & Wait
Thread A Thread B
1A: Lock 1B: Lock
2A: Request 2B: Send Response
3A: Wait for Response 3B: Unlock
4A: Do something with Response
5A: Unlock
Before
Thread A Thread B
1A: Lock 1B: Lock
2A: Create & Resume Coroutine 2B: Send Response
3A: Request 3B: Unlock
4A: Yield
5A: Unlock
Thread A’ (invoked by response)
6A’: Lock
7A’: Resume
8A’: Do something with response
9A’: Yield Break
10A’: Unlock
Non-Blocking
Sequential Code
After
Solutionfor
My Problem
Thread Block-
ing
Game Client
LobbyRoom
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
Room Thread Lobby Service
1R: Call ‘Authentication’ RPC function 1L: Process the request
2R: Wait for the response 2L: Send back the response
3R: Process
Before
Room Thread Lobby Service
1R: Create & Resume Coroutine 1L: Process the request
2R: Request 2L: Send back the response
3R: Yield
Room Thread’(invoked by response)
4R’: Resume
5R’: Process
6R’: Yield Break
Non-Blocking
Sequential Code
After
Request & Wait
Request & Yield
Non-Blocking&
Writing Sequential Code
Coroutine(Fiber)
Thread
CPU CPU CPU CPU
User-mode SchedulingKernel-mode Scheduling
Corou-tine Pool
Network I/O Thread
Pool
RPC Task Queue
RPC Exe-cution
Thread Pool
RPC Task Queue
RPC Execu-tion Thread
Network I/O Thread
enqueuetask
dequeuetask
acquire corou-tine
enqueue
re-sumetask
dequeue
re-sumetask
resume
yield break
release corou-tine
Network Event
Network Event
Coroutine Pool
pool-ing
corou-tine
RPC Execu-tion
yield return
start
RPC Task Queue
RPC Execu-tion Thread
1
Coroutine Pool
RPC Execu-tion
enqueuetask
dequeuetask
acquire corou-tine corou-
tine
yield return
enqueue
re-sumetask
dequeue
re-sumetask
release corou-tine
Network Event
Network Event
pool-ing
RPC Execu-tion Thread
2
resume
yield break
resume
Network I/O Thread
RPC::Result* JoinRoom(…){ … RPC::Result answer = RPC::SyncCall(“VerifyLobbyUserToken”, …); …. return result;}
RPC::Result* JoinRoom(…){ … RPC::Result answer = RPC::SyncCallYield(“VerifyLobbyUserToken”, …); …. return result;}
TPS
# of Thread
Blocking
Yield
Wrap-up
“The free lunch is OVER.”Write Multithreaded
Application
CONCURRENCY
Thread Blocking is Evil
Starvation Deadlock
Asynchronous Programming
+Coroutine
Non-Blocking+
Writing Sequen-tial Code
Request & Wait
Request & Yield
Inspired by Jeffrey Richter
‘Simplified APM With The Asyn-cEnumerator’
Pictures fromwww.Istockphoto.comwww.Flickr.com