1

Reliable Distributed Systems

Communication Basics ISlide set based on one by Professor Paul Francis,

Cornell UniversityModified by Bina Ramamurthy

2

Overview of Lecture

Introduction to the network layer Classic view of network layer

OSI stack Classic view no longer accurate End-to-end argument Internet components (hosts, routers, links,

etc.) Protocol layering fundamentals IP, UDP, TCP, pros and cons, SCTP Ethereal---nice protocol monitoring and

debugging tool

3

An Overview of Current State Client/server communication protocols Homogeneous system vs. interoperability of

heterogeneous systems Java run anywhere Interoperability standards (CORBA, web

services) Layered approach: SOAP/HTTP/TCP/IP Addressing:

provide unique identification of source and destination of a message,

ways of mapping resources to network addresses, and

obtain best route for sending messages. IP multicast (D class): under utilized, group

communications?

4

Socket based communication

int sockfd; struct sockaddr_in addr;

addr.sin_family = AF_INET; addr.sin_addr.s_addr =

inet_addr(SERV_HOST_ADDR); addr.sin_port = htons(SERV_TCP_PORT);

sockfd = socket(AF_INET, SOCK_STREAM, 0);connect(sockfd, (struct sockaddr *) &addr,

sizeof(serv_addr));do_stuff(stdin, sockfd);

5

Classic view of network API Start with host

name (maybe)

foo.bar.com

6

Classic view of network API Start with host

name Get an IP address

foo.bar.comgethostbyname()

10.5.4.3

7

Classic view of network API Start with host

name Get an IP address Make a socket

(protocol, address)

foo.bar.comgethostbyname()

10.5.4.3

sock_id

socket();connect();…

8

Classic view of network API Start with host

name Get an IP address Make a socket

(protocol, address)

Send byte stream (TCP) or packets (UDP)

foo.bar.comgethostbyname()

10.5.4.3

sock_id

socket();connect();…

TCP sock UDP sock

Network

1,2,3,4,5,6,7,8,9 . . . …

Eventually arrive in order

May or may not arrive

9

Classic approach “broken” in many ways IP address different depending on who asks

for it Address may be changed in the network IP address may not be reachable (even

though destination is up and attached) Or may be reachable by you but not another

host IP address may change in a few minutes or

hours Packets may not come from who you think

(network caches)

10

Classic OSI stack

11

Example Microsoft VPN stack

Application

TCP

PPP

L2TP

UDP

IPsec

IP

IP

PPP

PPPoE

Ethernet

12Ethernet

Example Microsoft VPN stack

Application

TCP

PPP

L2TP

UDP

IPsec

IP

IP

PPP

PPPoE The link layer

13Ethernet

Example Microsoft VPN stack

Application

TCP

PPP

L2TP

UDP

IPsec

IP

IP

PPP

PPPoE The link layer

A logical link layer

14Ethernet

Example Microsoft VPN stack

Application

TCP

PPP

L2TP

UDP

IPsec

IP

IP

PPP

PPPoE The link layer

A logical link layer

A tunnel

15Ethernet

Example Microsoft VPN stack

Application

TCP

PPP

L2TP

UDP

IPsec

IP

IP

PPP

PPPoE The link layer

A logical link layer

A security layer

A tunnel

16Ethernet

Example Microsoft VPN stack

Application

TCP

PPP

L2TP

UDP

IPsec

IP

IP

PPP

PPPoE The link layer

A logical link layer

A security layer

A network abstraction that Microsoft finds convenient

A tunnel

17Ethernet

Example Microsoft VPN stack

Application

TCP

PPP

L2TP

UDP

IPsec

IP

IP

PPP

PPPoE The link layer

A logical link layer

A security layer

A network abstraction that Microsoft finds convenient

The actual end-to-end network and transport layers

A tunnel

18Ethernet

Example Microsoft VPN stack

Application

TCP

PPP

L2TP

UDP

IPsec

IP

IP

PPP

PPPoE

TCP: Transport Control ProtocolIP: Internet ProtocolPPP: Point-to-Point ProtocolL2TP: Layer 2 Tunneling ProtocolUDP: User Datagram ProtocolIPsec: Secure IPPPPoE: PPP over Ethernet

19

What about the end-to-end argument?

In a nutshell:If you want something done right,

you gotta do it yourself

“End-To-End Arguments In System Design”, Saltzer, Reed, Clark, ACM Transactions on Computer Systems, 1984

20

End-to-end argument is mostly about reliability

Early 80’s: industry assumed that the network should do everything Guaranteed delivery, sequencing,

duplicate suppression If the network does it, the end system

doesn’t have to X.25, for example

21

The network doesn’t always work right

Applications had to check to see if the network really did its job… … and repair the problem if the

network didn’t do its job End-to-end insight:

If the application has to do it anyway, why do it in the network at all?

Keep the network simple

22

So when should the network do more? When you get performance gains

Link-level retransmissions over a lossy link are faster than E2E retransmissions

Also When the network doesn’t trust the end user

Corporation or military encrypt a link because the end user might not do it

Some things just can’t be done at the end Routing algorithms Billing User authentication

23

Network components

R

H H H H H H

R R

H H HH H H

Host: Source and sink of IP packets

Router: Forwards IP packets

Point to point link: link with two nodes (router or host)

Broadcast link: link with multiple nodes

24

Network components Network: Collection of hosts, links, and routers Site: Stub network, typically in one location and under control of

one administration Firewall/NAT: Box between the site and ISP that provides

filtering, security, and Network Address Translation ISP: Internet Service Provider. Transit network that provides IP

connectivity for sites Backbone ISP: Transit network for regional ISPs and large sites Inter-exchange (peering point): Broadcast link where

multiple ISPs connect and exchange routing information (peering) Hosting center: Stub network that supports lots of hosts (web

services), typically with high speed connections to many backbone ISPs.

Bilateral peering: Direct connection between two backbone ISPs

25

Internet topology

S

ISP

BackboneISP

IX IX

S S

Site

S

ISP

S S S

ISP

S S

BackboneISP

BackboneISP

HostingCenter

HostingCenter

IXs came first

IXs tend to be performance bottlenecks

Hosting centers and bilateral peering are a response to poor IXs

Sites

26

Protocol layering Communications stack consists of a set of

services, each providing a service to the layer above, and using services of the layer below Each service has a programming API, just like

any software module Each service has to convey information one

or more peers across the network This information is contained in a header

The headers are transmitted in the same order as the layered services

27

Protocol layering example

Browserprocess

HTTP

TCP

Link1

IP

Link1

IP

Link2

Web serverprocess

HTTP

TCP

Link1

IP

Physical Link 1 Physical Link 2

Router

28

HTTP

Protocol layering example

Browserprocess

TCP

Link1

IP

Link1

IP

Link2

Web serverprocess

HTTP

TCP

Link1

IP

Physical Link 1 Physical Link 2

Router

H

Browser wants to request a page. Calls HTTP with the web address (URL).HTTP’s job is to convey the URL to the web server.HTTP learns the IP address of the web server, adds its header, and calls TCP.

29

HTTP

Protocol layering example

Browserprocess

TCP

Link1

IP

Link1

IP

Link2

Web serverprocess

HTTP

TCP

Link1

IP

Physical Link 1 Physical Link 2

H

TCP’s job is to work with server to make sure bytes arrive reliably and in order.TCP adds its header and calls IP.(Before that, TCP establishes a connection with its peer.)

T Router

30

HTTP

Protocol layering example

Browserprocess

TCP

Link1

IP

Link1

IP

Link2

Web serverprocess

HTTP

TCP

Link1

IP

Physical Link 1 Physical Link 2

H

IP’s job is to get the packet routed to the peer through zero or more routers.IP determines the next hop from the destination IP address.IP adds its header and calls the link layer (i.e. Ethernet) with the next hop address.

T

Router

I

31

HTTP

Protocol layering example

Browserprocess

TCP

Link1

IP

Link1

IP

Link2

Web serverprocess

HTTP

TCP

Link1

IP

Physical Link 1 Physical Link 2

H

The link’s job is to get the packet to the next physical box (here a router).It adds its header and sends the resulting packet over the “wire”.

T

Router

I L1

32

HTTP

Protocol layering example

Browserprocess

TCP

Link1

IP

Link1

IP

Link2

Web serverprocess

HTTP

TCP

Link1

IP

Physical Link 1 Physical Link 2

H

The router’s link layer receives the packet, strips the link header, and hands the result to the IP forwarding process.

T

Router

I

33

HTTP

Protocol layering example

Browserprocess

TCP

Link1

IP

Link1

IP

Link2

Web serverprocess

HTTP

TCP

Link1

IP

Physical Link 1 Physical Link 2

H

The router’s IP forwarding process looks at the destination IP address, determines what the next hop is, and hands the packet to the appropriate link layer with the appropriate next hop link address.

T

Router

I

34

HTTP

Protocol layering example

Browserprocess

TCP

Link1

IP

Link1

IP

Link2

Web serverprocess

HTTP

TCP

Link1

IP

Physical Link 1 Physical Link 2

H

The packet goes over the link to the web server, after which each layer processes and strips its corresponding header.

T

Router

I L2

H T I

H T

H

35

Basic elements of any protocol header Demuxing field

Indicates which is the next higher layer (or process, or context, etc.)

Length field or header delimiter For the header, optionally for the

whole packet Header format may be text (HTTP,

SMTP (email)) or binary (IP, TCP, Ethernet)

36

Demuxing fields Ethernet: Protocol Number

Indicates IPv4, IPv6, (old: Appletalk, SNA, Decnet, etc.) IP: Protocol Number

Indicates TCP, UDP, SCTP TCP and UDP: Port Number

Well known ports indicate FTP, SMTP, HTTP, SIP, many others

Dynamically negotiated ports indicate specific processes (for these and other protocols)

HTTP: Host field Indicates “virtual web server” within a physical web

server

37

IP (Internet Protocol) Three services:

Unicast: transmits a packet to a specific host Multicast: transmits a packet to a group of hosts Anycast: transmits a packet to one of a group of

hosts (typically nearest) Destination and source identified by the IP

address (32 bits for IPv4, 128 bits for IPv6) All services are unreliable

Packet may be dropped, duplicated, and received in a different order

38

IP(v4) address format In binary, a 32-bit integer In text, this: “128.52.7.243”

Each decimal digit represents 8 bits (0 – 255) “Private” addresses are not globally unique:

Used behind NAT boxes 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16

Multicast addresses start with 1110 as the first 4 bits (Class D address)

224.0.0.0/4 Unicast and anycast addresses come from

the same space

39

UDP (User Datagram Protocol) Runs above IP Same unreliable service as IP

Packets can get lost anywhere: Outgoing buffer at source Router or link Incoming buffer at destination

But adds port numbers Used to identify “application layer”

protocols or processes Also a checksum, optional

40

TCP (Transmission Control Protocol) Runs above IP

Port number and checksum like UDP Service is in-order byte stream

Application does not absolutely know how the bytes are packaged in packets

Flow control and congestion control Connection setup and teardown phases Can be considerable delay between bytes in at

source and bytes out at destination Because of timeouts and retransmissions

Works only with unicast (not multicast or anycast)

41

UDP vs. TCP UDP is more real-time

Packet is sent or dropped, but is not delayed UDP has more of a “message” flavor

One packet = one message But must add reliability mechanisms over it

TCP is great for transferring a file or a bunch of email, but kind-of frustrating for messaging

Interrupts to application don’t conform to message boundaries

No “Application Layer Framing” TCP is vulnerable to DoS (Denial of Service)

attacks, because initial packet consumes resources at the receiver

42

Ethereal Great open-source tool for understanding

and debugging protocol behavior www.ethereal.com Features:

Trace packets over the wire Sophisticated filtering language Display contents of each protocol Dump contents into file Display TCP conversation

43

Captured Frames

44

TCP conversation

45

Supports these 340 protocols

802.11 MGT, AARP, AFP, AFS (RX), AH, AIM, AJP13, AODV, AODV6, ARCNET, ARP/RARP, ASAP, ASP, ATM, ATM LANE, ATP, AVS WLANCAP, Auto-RP, BACapp, BACnet, BEEP, BGP, BOOTP/DHCP, BOOTPARAMS, BOSSVR, BROWSER, BVLC, CDP, CDS_CLERK, CFLOW, CGMP, CHDLC, CLEARCASE, CLNP, CLTP, CONV, COPS, COTP, CPHA, CUPS, CoSine, DCCP, DCERPC, DCERPC_NT, DCE_DFS, DDP, DDTP, DEC_STP, DFS, DHCPv6, DLSw, DNS, DNSSERVER, DSI, DTSPROVIDER, DTSSTIME_REQ, DVMRP, Data, Diameter, EAP, EAPOL, EIGRP, EPM, ESIS, ESP, Ethernet, FC, FC ELS, FC-SWILS, FCIP, FCP, FDDI, FIX, FLDB, FR, FTP, FTP-DATA, FTSERVER, FW-1, Frame, GIOP, GMRP, GNUTELLA, GRE, GSS-API, GTP, GTPv0, GTPv1, GVRP, H.261, H1, HCLNFSD, HSRP, HTTP, HyperSCSI, IAPP, IB, ICAP, ICMP, ICMPv6, ICP, ICQ, IEEE 802.11, IGMP, IGRP, ILMI, IMAP, IP, IPComp, IPFC, IPP, IPX, IPX MSG, IPX RIP, IPX SAP, IPv6, IRC, ISAKMP, ISDN, ISIS, ISL, ISUP, IUA, KLM, KRB5, KRB5RPC, L2TP, LACP, LANMAN, LAPB, LAPBETHER, LAPD, LDAP, LDP, LLAP, LLC, LMI, LMP, LPD, LSA, LSA_DS, Lucent/Ascend, M2PA, M2TP, M2UA, M3UA, MAPI, MGMT, MMSE, MOUNT, MPEG1, MPLS, MRDISC, MS Proxy, MSDP, MSNIP, MTP2, MTP3, Mobile IP, Modbus/TCP, NBDS, NBIPX, NBNS, NBP, NBSS, NCP, NDMP, NDPS, NETLOGON, NFS, NFSACL, NFSAUTH, NIS+, NIS+ CB, NLM, NMPI, NNTP, NSPI, NTLMSSP, NTP, NetBIOS, Null, OSPF, OXID, PCNFSD, PFLOG, PGM, PIM, POP, PPP, PPP BACP, PPP BAP, PPP CBCP, PPP CCP, PPP CDPCP, PPP CHAP, PPP Comp, PPP IPCP, PPP IPV6CP, PPP LCP, PPP MP, PPP MPLSCP, PPP PAP, PPP PPPMux, PPP PPPMuxCP, PPP VJ, PPPoED, PPPoES, PPTP, Portmap, Prism, Q.2931, Q.931, QLLC, QUAKE, QUAKE2, QUAKE3, QUAKEWORLD, RADIUS, RANAP, REMACT, REP_PROC, RIP, RIPng, RMI, RPC, RPC_BROWSER, RPC_NETLOGON, RPL, RQUOTA, RSH, RSTAT, RSVP, RS_ACCT, RS_ATTR, RS_PGO, RS_REPADM, RS_REPLIST, RS_UNIX, RTCP, RTMP, RTP, RTSP, RWALL, RX, Raw, Rlogin, SADMIND, SAMR, SAP, SCCP, SCCPMG, SCSI, SCTP, SDP, SECIDMAP, SGI MOUNT, SIP, SKINNY, SLARP, SLL, SMB, SMB Mailslot, SMB Pipe, SMPP, SMTP, SMUX, SNA, SNAETH, SNMP, SPNEGO-KRB5, SPOOLSS, SPRAY, SPX, SRVLOC, SRVSVC, SSCOP, SSL, STAT, STAT-CB, STP, SUA, Serialization, SliMP3, Socks, Spnego, Syslog, TACACS, TACACS+, TAPI, TCP, TDS, TELNET, TFTP, TIME, TKN4Int, TNS, TPKT, TR MAC, TSP, Token-Ring, UBIKDISK, UBIKVOTE, UCP, UDP, V.120, VLAN, VRRP, VTP, Vines, Vines FRP, Vines SPP, WCCP, WCP, WHO, WINREG, WKSSVC, WSP, WTLS, WTP, X.25, X11, XDMCP, XOT, XYPLEX, YHOO, YPBIND, YPPASSWD, YPSERV, YPXFR, ZEBRA, ZIP, cds_solicit, cprpc_server, dce_update, iSCSI, roverride, rpriv, rs_misc, rsec_login,

46

Summary TCP, UDP, IP provide a nice set of basic tools

Key is to understand concept of protocol layering But problems/limitations exist

IP has been compromised by NAT, can’t be used as a stable identifier

Firewalls can block communications TCP has vulnerabilities Network performance highly variable

Next lecture we’ll look at other forms of naming and identification

Help overcome limitations of IP