A Technique for improving the scheduling of network communicating processes in MOSIX
Rengakrishnan SubramanianMasters Report, Final Defense
Guidance by Prof. Dan Andresen
Agenda
n MOSIXn Network communicating processesn Breaking it downn Timingn Implementationn Testn Results n Conclusion
MOSIX - purpose
n Software tool for cluster computing
n Multiple servers work as if single to achieve high performance
n Automatic work distribution
n Load balancing
n Adaptive management (processes v/sresources)
MOSIX – tools
n Create a process (or more processes)
n Distribute (and redistribute)
n Algorithms respond to variation
n Works on Linux x86 platforms
n Kernel patch
n System Admin tools
Agenda
n MOSIX
n Network communicating processesn Breaking it downn Timingn Implementationn Testn Results n Conclusion
Network Processes
n Preemptive process migrationn Interaction with environment after
migrationn User context, System / UHN contextn Remote migrates, deputy stays at
UHNn Deputy has kernel resources (sockets
too!)
Network Processes - origin
Cluster of nodes
Node aProcess A & B
Node b
Node aProcess A
Node bProcess BCluster of
nodes
Network Process - migrated
Node b
Node aProcess AProcess B
B to A
B to A
Cluster of nodes
Network Process - migrated
Node bProcess B
Node cNode a
Process A
Node bNode c
Process B
Node aProcess A
Agenda
n MOSIXn Network communicating processes
n Breaking it downn Timingn Implementationn Testn Results n Conclusion
Breaking down – Microscopic User Space
Kernel / Socket Layer
TCP
Firewall
Lower Layers
MOSIX
User Space
Kernel / Socket Layer
TCP
Firewall
Lower Layers
MOSIX
User Space
Kernel / Socket Layer
TCP
Firewall
Lower Layers
MOSIX
Agenda
n MOSIXn Network communicating processesn Breaking it down
n Timingn Implementation n Testn Results n Conclusion
Timing - picture
T1
User Space
Kernel / Socket Layer
TCP/ IP
Firewall
Lower Layers
MOSIX
T2
T4
T5
T6
T7T3
T8
T911.94 µsec
8.59 µsec
24.22 µsec
2.9 µsec
Timing - equation
n The amount of time that can be saved if the
packets are redirected at the firewall layer =
(Time taken at the TCP layer, 24.22 µsec)
+ (Time taken at the socket layer, 2.9 µsec)
+ (Time taken by MOSIX to decide on the
fate of the packet, M µsec).
Timing – equation 2
n Recalculated Time = 2 X { (Time taken at the TCP layer, 24.22 µsec) + (Time taken at the socket layer, 2.9 µsec)} + (Time taken by MOSIX to decide on the fate of the packet, M µsec).
Timing - Inference
n Yes, time can be saved
n Yes, time can be saved at the firewall layer
n The packet can be identified
n The packet can be redirected
n Because, the firewall can do NAT
Agenda
n MOSIXn Network communicating processesn Breaking it downn Timing
n Implementation n Testn Results n Conclusion
Implementation - IPTables
n Built-in firewall tool in Linux kernel
n Successor of IPChains and IPFwadm
n MOSIX works on Linux x86
n Timing available for IPTables
Implementation - NAT
PRE-ROUTING
DNAT
POST-ROUTING
SNAT
[Routing Decision]
Local Process
IN
OUT
Implementation - picture
Node bFirewall
Node cServer
Node aClient
Implementation - rules
n # iptables -t nat -A PREROUTING -s $CLIENT -d $FIREWALL _SYSTEM -p tcp –dport $SERVER_PORT -i eth0 -j DNAT --to-destination $NEW_DESTINATION
n # iptables -t nat -A POSTROUTING -s $CLIENT -p tcp --dport $SERVER_PORT -o eth0 -j SNAT --to-source
$FIREWALL_SYSTEM
n # iptables -t nat -A PREROUTING -s $SERVER -d $FIREWALL_SYSTEM -p tcp --sport $SERVER_PORT -i eth0 -j DNAT --to-destination $CLIENT
Agenda
n MOSIXn Network communicating processesn Breaking it downn Timing
n Implementation
n Testn Results n Conclusion
Testing – what ?
n MOSIX communication through
redirection
n IPTables communication through
redirection
n Direct communication
Testing - Environment
n Pentium P4 CPUn 1.6 GHz Processor speedn Intel Ether express Network cardn 100 Mbps LANn Two Red Hat 7.2 Linux boxes with Kernel
2.4.19 n One Debian Linux box with Kernel 2.4.18n All nodes were connected on to the same
LAN switch
Testing – how?
n Server-Client communicating pairn Parameterizedn Buffer sizen Amount of data à time n Number of such communicating pairsn Port numbers n Prints out time taken for data transfer
Testing – MOSIX 1
Node bServer is created here. Server waits on a port number.
Node cNode aClient will reside hereClient is not yet created.
Testing – MOSIX 2
Node bServer is migrated manually using MOSIX admin tools to node cAll processes think server is still in Node b
Node cServer is now here.But, it goes to Node b for system calls. Node b is its UHN
Node aClient is created. Contacts server in Node b.Is unaware that server is in Node c
Testing - IPTables
Node bIPTables rules are written here.Forwards packets from Node a to Node c and vice-versa
Node aClient is created hereClient contacts Node b requesting for service
Node cServer is created here.It will get request from Node b (which is in reality from Node a). It will reply back to Node b
Testing - Direct
Node cServer is created here.It will get request from Node aIt will reply back to Node a
Node b
Node aClient is created hereClient contacts Node c requesting for service
Agenda
n MOSIXn Network communicating processesn Breaking it downn Timing
n Implementationn Test
n Resultsn Conclusion
Results – Execution time
Results - Bandwidth
Results – CPU Utilization
Results – Load Average
Results – Execution Graph
Execution Time Comparison
0
100
200
300
400
500
600
700
0 2 4 6 8 10 12 14 16
No of Connections
Tim
e (s
ecs)
mosix iptables direct
Results – Bandwidth Graph
Bandwidth Comparison
0
5
10
15
20
25
30
35
0 2 4 6 8 10 12 14 16
No of connections
Ban
dw
idth
(Mbps)
mosix iptables direct
Results – CPU Graph
%CPU Utlization Comparison
0
20
40
60
80
100
0 10 20 30 40 50 60
No of connections
%C
PU
Uti
lizat
ion
mosix iptables
Results – Load Graph
Load Average Comparison
0
1
2
3
4
5
0 10 20 30 40 50 60
No of Connections
Load
Ave
rage
mosix iptables
Agenda
n MOSIXn Network communicating processesn Breaking it downn Timing
n Implementationn Testn Results
n Conclusion
Conclusion - 1
n MOSIX takes 33% more time on
execution than direct communication
n IPTables takes only 4% more
n MOSIX takes 28% more time on
execution than IPTables
Conclusion - 2
n Bandwidth of MOSIX is 20% less than
IPTables
n MOSIX, on an average, takes 212% more CPU utilization
n MOSIX takes at least 138 times more load average than IPTables
Conclusion – 3
n Yes, better performance can be achieved
n Better Execution time
n Better Bandwidth utilization
n Better load average
n Better CPU utilization
Questions