SocketsDistributed Systems
Philipp Kupferschmied
Universitat Karlsruhe, System Architecture Group
May 6th, 2009
Philipp Kupferschmied Sockets 1/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
1 The Socket AbstractionSocket BasicsCharacteristics
2 User’s Point of ViewConnectionsSingle-Threaded ServersSummary
3 Sockets in LinuxData StructuresTraversing LayersConclusion
Philipp Kupferschmied Sockets 2/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
What are Sockets?
Socket Socket
Kernel
ProcessA
ProcessB
Connection oriented IPC
“Phone connection” between processes
Protocol determines how the connection is used
Philipp Kupferschmied Sockets 3/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
Socket Types
Socket
File descriptor (sock fd)
Protocol family
Unix-domain (AF UNIX)Internet-domain (AF INET)
Socket type
Stream (SOCK STREAM)Datagram (SOCK DGRAM)
Protocol
UNIX STREAM, UNIX DGRAMTCP, UDP
Philipp Kupferschmied Sockets 4/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
Socket Types
Socket
File descriptor (sock fd)
Protocol family
Unix-domain (AF UNIX)Internet-domain (AF INET)
Socket type
Stream (SOCK STREAM)Datagram (SOCK DGRAM)
Protocol
UNIX STREAM, UNIX DGRAMTCP, UDP
Philipp Kupferschmied Sockets 4/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
Socket Types
Socket
File descriptor (sock fd)
Protocol family
Unix-domain (AF UNIX)Internet-domain (AF INET)
Socket type
Stream (SOCK STREAM)Datagram (SOCK DGRAM)
Protocol
UNIX STREAM, UNIX DGRAMTCP, UDP
Philipp Kupferschmied Sockets 4/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
Socket Types
Socket
File descriptor (sock fd)
Protocol family
Unix-domain (AF UNIX)Internet-domain (AF INET)
Socket type
Stream (SOCK STREAM)Datagram (SOCK DGRAM)
Protocol
UNIX STREAM, UNIX DGRAMTCP, UDP
Philipp Kupferschmied Sockets 4/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
Unix-Domain Sockets
Implemented purely in memory
Users can bind a socket to an “address”
Socket becomes accessible at this “address”
Unix-domain addresses are file names
/tmp/server
sockaddr un
sun family = AF UNIXsun path = /tmp/server Socket Socket
Client Server
Philipp Kupferschmied Sockets 5/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
Unix-Domain Sockets
Implemented purely in memory
Users can bind a socket to an “address”
Socket becomes accessible at this “address”
Unix-domain addresses are file names
/tmp/server
sockaddr un
sun family = AF UNIXsun path = /tmp/server Socket Socket
Client Server
/ tmp/server
Philipp Kupferschmied Sockets 5/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
Unix-Domain Sockets
Implemented purely in memory
Users can bind a socket to an “address”
Socket becomes accessible at this “address”
Unix-domain addresses are file names
/tmp/server
sockaddr un
sun family = AF UNIXsun path = /tmp/server Socket Socket
Client Server
/ tmp/server
request
Philipp Kupferschmied Sockets 5/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
Unix-Domain Sockets
Implemented purely in memory
Users can bind a socket to an “address”
Socket becomes accessible at this “address”
Unix-domain addresses are file names
/tmp/server
sockaddr un
sun family = AF UNIXsun path = /tmp/server Socket Socket
Client Server
/ tmp/server
request
reply
Philipp Kupferschmied Sockets 5/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
Internet-Domain Sockets
Implemented via the network
Users can bind a socket to an “address”
Socket becomes accessible at this “address”
Internet-domain addresses consist of IP and port number
192.168.0.1:80
sockaddr in
sin family = AF INETsin port = 80sin addr = 0x0100A8C0
Socket Socket
Client Server
Philipp Kupferschmied Sockets 6/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
Internet-Domain Sockets
Implemented via the network
Users can bind a socket to an “address”
Socket becomes accessible at this “address”
Internet-domain addresses consist of IP and port number
192.168.0.1:80
sockaddr in
sin family = AF INETsin port = 80sin addr = 0x0100A8C0
Socket Socket
Client Server
192.168.0.1:80192.168.0.1:3245
Philipp Kupferschmied Sockets 6/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
Internet-Domain Sockets
Implemented via the network
Users can bind a socket to an “address”
Socket becomes accessible at this “address”
Internet-domain addresses consist of IP and port number
192.168.0.1:80
sockaddr in
sin family = AF INETsin port = 80sin addr = 0x0100A8C0
Socket Socket
Client Server
192.168.0.1:80
request
192.168.0.1:3245
Philipp Kupferschmied Sockets 6/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
Internet-Domain Sockets
Implemented via the network
Users can bind a socket to an “address”
Socket becomes accessible at this “address”
Internet-domain addresses consist of IP and port number
192.168.0.1:80
sockaddr in
sin family = AF INETsin port = 80sin addr = 0x0100A8C0
Socket Socket
Client Server
192.168.0.1:80
request
reply
192.168.0.1:3245
Philipp Kupferschmied Sockets 6/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Socket BasicsCharacteristics
Characteristics
Communication between two sockets
Connection-oriented orPacket-based
Allow for error control (TCP, UNIX STREAM)
Network-capable IPC
Flexible
AX 25, APPLETALK, IPX, HTTP, RPC, . . .
Philipp Kupferschmied Sockets 7/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Establishing a Connection
socket() socket
Philipp Kupferschmied Sockets 8/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Establishing a Connection
bind() socket
Philipp Kupferschmied Sockets 8/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Establishing a Connection
l isten() socket
Philipp Kupferschmied Sockets 8/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Establishing a Connection
accept() socket
Philipp Kupferschmied Sockets 8/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Establishing a Connection
accept()
socket()socket
socket
Philipp Kupferschmied Sockets 8/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Establishing a Connection
accept() connect()socketsocket
Philipp Kupferschmied Sockets 8/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Establishing a Connection
socket’
socket
socketread()wri te()close()
wri te()read()close()
Philipp Kupferschmied Sockets 8/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Establishing a Connection
accept() socket
Philipp Kupferschmied Sockets 8/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Establishing a Connection
socket() socket
bind()
l isten()
accept() connect()
socket()socket
socket
socket
socket’
socket
socketread()wri te()close()
wri te()read()close()
socket
Philipp Kupferschmied Sockets 8/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Data Transfer
socket’ socketread()wri te()
wri te()read()
socket
socket’
socket’’ socket
socket
socket connect()
wri te()
wri te()read()
read()
accept()
Blocking syscalls ‘accept’ and ‘read’ require a thread per socket.Philipp Kupferschmied Sockets 9/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Data Transfer
socket’ socketread()wri te()
wri te()read()
socket
socket’
socket’’ socket
socket
socket connect()
wri te()
wri te()
fd_set
select()or poll()
Use ‘select’ or ‘poll’ to watch many sockets in one thread.Philipp Kupferschmied Sockets 9/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Strengths and Weaknesses
+ Encapsulation of data transfer
+ Extensibility with new protocols
+ Flexibility (e.g., X server)
Unix-domain sockets for local clientsInternet-domain sockets for remote clientsSame code, except connect()
– Inconvenient programming interface
– Each process has to close its local socket
Philipp Kupferschmied Sockets 10/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Example (Unix-Domain)
Client:
int s = socket ( AF_UNIX, SOCK_STREAM, 0 );
struct sockaddr_un server_addr;
server_addr.sun_family = AF_UNIX;
strcpy( server_addr.sun_path, "/tmp/serversock" );
connect( s, (struct sockaddr *) &server_addr, sizeof(server_addr) );
Server:
s = socket( AF_UNIX, SOCK_STREAM, 0 );
struct sockaddr_un server_addr;
server_addr.sun_family = AF_UNIX;
strcpy( server_addr.sun_path, "/tmp/serversock" );
bind( s, (struct sockaddr *)&server_addr, sizeof(server_addr) );
listen( s, 5 );
Philipp Kupferschmied Sockets 11/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
ConnectionsSingle-Threaded ServersSummary
Example (Internet-Domain)
Client:
int s = socket( AF_INET, SOCK_STREAM, 0 );
struct sockaddr_in server_addr;
server_addr.sin_family = AF_INET;
server_addr.sin_port = htons(15001);
inet_aton( "127.0.0.1", &server_addr.sin_addr );
connect( s, (struct sockaddr *)&server_addr, sizeof(server_addr) );
Server:
int s = socket( AF_INET, SOCK_STREAM, 0 );
struct sockaddr_in server_addr;
server_addr.sin_family = AF_INET;
server_addr.sin_port = htons(15001);
inet_aton( "127.0.0.1", &server_addr.sin_addr );
bind( s, (struct sockaddr *)&server_addr, sizeof(server_addr) );
listen( s, 5 );
Philipp Kupferschmied Sockets 12/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Data StructuresTraversing LayersConclusion
struct socket
struct socket
stateflagsproto opsfasync listfileskwaittype
struct proto ops
inet writeinet read. . .
struct sock
sock commonsk protocolsk type. . .headtail
struct sk buff
nextprevskdevdata. . .
Philipp Kupferschmied Sockets 13/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Data StructuresTraversing LayersConclusion
struct socket
struct socket
stateflagsproto opsfasync listfileskwaittype
struct proto ops
inet writeinet read. . .
struct sock
sock commonsk protocolsk type. . .headtail
struct sk buff
nextprevskdevdata. . .
Philipp Kupferschmied Sockets 13/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Data StructuresTraversing LayersConclusion
struct socket
struct socket
stateflagsproto opsfasync listfileskwaittype
struct proto ops
inet writeinet read. . .
struct sock
sock commonsk protocolsk type. . .headtail
struct sk buff
nextprevskdevdata. . .
Philipp Kupferschmied Sockets 13/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Data StructuresTraversing LayersConclusion
struct socket
struct socket
stateflagsproto opsfasync listfileskwaittype
struct proto ops
inet writeinet read. . .
struct sock
sock commonsk protocolsk type. . .headtail
struct sk buff
nextprevskdevdata. . .
Philipp Kupferschmied Sockets 13/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Data StructuresTraversing LayersConclusion
Linux Sockets in Action
write(fd)
sys_write(fd)
sock_write()
inet_sendmsg()
tcp_sendmsg()
tcp_send_skb()
tcp_transmit_skb()
ip_queue_xmit()
ei_start_xmit()
read()
sys_read()
sock_read()
inet_recvmsg()
tcp_recvmsg()
tcp_data()
tcp_rcv()
ip_recv()
ei_recv
ei_interrupt
proto_ops
data USER
VFS
TRANSPORT
NETWORK
DATA LINK
PHYSICAL
socket
sock
data
data
Philipp Kupferschmied Sockets 14/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Data StructuresTraversing LayersConclusion
Linux Sockets in Action
write(fd)
sys_write(fd)
sock_write()
inet_sendmsg()
tcp_sendmsg()
tcp_send_skb()
tcp_transmit_skb()
ip_queue_xmit()
ei_start_xmit()
read(fd)
sys_read(fd)
sock_read()
inet_recvmsg()
tcp_recvmsg()
tcp_data()
tcp_rcv()
ip_recv()
ei_recv
ei_interrupt
proto_ops
data USER
VFS
TRANSPORT
NETWORK
DATA LINK
PHYSICAL
socket
sock
data
Philipp Kupferschmied Sockets 14/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Data StructuresTraversing LayersConclusion
Linux Sockets in Action
write(fd)
sys_write(fd)
sock_write()
inet_sendmsg()
tcp_sendmsg()
tcp_send_skb()
tcp_transmit_skb()
ip_queue_xmit()
ei_start_xmit()
read(fd)
sys_read(fd)
sock_read()
inet_recvmsg()
tcp_recvmsg()
tcp_data()
tcp_rcv()
ip_recv()
ei_recv
ei_interrupt
proto_ops
data USER
VFS
TRANSPORT
NETWORK
DATA LINK
PHYSICAL
socket
sock
data
data
Philipp Kupferschmied Sockets 14/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Data StructuresTraversing LayersConclusion
Linux Sockets in Action
write(fd)
sys_write(fd)
sock_write()
inet_sendmsg()
tcp_sendmsg()
tcp_send_skb()
tcp_transmit_skb()
ip_queue_xmit()
ei_start_xmit()
read(fd)
sys_read(fd)
sock_read()
inet_recvmsg()
tcp_recvmsg()
tcp_data()
tcp_rcv()
ip_recv()
ei_recv
ei_interrupt
proto_ops
data USER
VFS
TRANSPORT
NETWORK
DATA LINK
PHYSICAL
socket
sock
data
Philipp Kupferschmied Sockets 14/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Data StructuresTraversing LayersConclusion
Conclusions
Sockets are an abstract communication interface
Hide different protocols (Unix-domain vs. Internet-domain)Protocol dictates communication model
Transient vs. persistent
Unix’ish “everything is a file” paradigm
Linux implementations are similar for different protocolsRealized via function pointers and matching data structures
Homebrew inheritance, “object-oriented C”
(Questionable) solution to both performance and structuredsoftware
Philipp Kupferschmied Sockets 15/ 16
The Socket AbstractionUser’s Point of View
Sockets in Linux
Data StructuresTraversing LayersConclusion
Literature
Sources of the Linux kernel 2.4/2.6
Helmut Herold: Linux/Unix Systemprogrammierung
Jurgen Wolf: Linux-Unix-Programmierunghttp://www.pronix.de/pronix-6.html
Beck: Linux Kernelprogrammierung
Online Tutorial:http://www.cs.rpi.edu/academics/courses/netprog/
Programming Assignment
You can gain experience in socket programming by implementingRPC via sockets in the programming assignment!
Philipp Kupferschmied Sockets 16/ 16