Request/Reply Communication

Mary Hudachek-BuswellGSU CSC 8320, Section 4.2Fall 2011

Outline of Background Introduction

Request /Reply CommunicationRemote Procedure Call (RPC)

RPC OperationsParameter PassingData ConversionBindingRPC Compilation

RPC Errors and FailureException Failure Handling

RPC Security2

Introduction– Request/Reply

Request/Reply Communication Send a single block of data/packet, then wait for a reply before sending another packetRequest/Reply protocols include FTP, HTTP, SMTP, POP, IMAP, etc…Remote Procedure Call (RPC); the Internet's most widely used request/reply communication model

3

Introduction– Request/Reply Communication

Requestor Replier

Request Queue

Reply Queue

4

Client Process

Introduction– Remote Procedure Call Client

(request initiated)

Client Stub(pack request for

transfer)

RPC Runtime Library

call return

returncall

Server Process

Manager Procedures

Server Stub

(unpack request )

RPC Runtime Library

callreturn

return call

Interface

Apparent Flow

Transport NetworkMessages

call

return

return

call

http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=%2Fcom.ibm.aix.progcomm%2Fdoc%2Fprogcomc%2Frpc_mod.

5

RPC Operations

Parameter Passing - what data can be passedData Conversion - how is the data representedBinding - locating server and registering the serviceRPC Compilation - origination of stub procedures and linking

6

RPC Operations - Parameter Passing Single process: via parameters and/or global variablesMultiple processes on the same host: via message passingPassing parameters is typically the only way that RPC-based clients and servers share informationParameter marshaling: Rules for parameter passing and data/message conversion is the primary responsibility of Stub procedure Parameter-passing modes: IN: pass info from caller to receiver OUT: receiver writes a value in the caller IN/OUT: caller tells receiver value of variable, which may be

updated by receiver

7

RPC Operations - Parameter Passing Call-by-value: just copy data to network message where value passed to procedure is copied to local variable, client stub performs copyCall-by-reference: pass pointer address and same memory location is referred to in passing, hard to implement in distributed systems with non-shared memory.Call-by-copy/restore: combination of call-by-value and call-by-reference. Call-by-value at the entry of call and call-by-reference to the exit of the callCall-by-name: requires dynamic run-time evaluation of symbolic expression.

Most RPC implementations assume that parameters are passed by call-by-value and call-by-copy/restore.

8

RPC Operations - Data Conversion Three areas requiring conversion between

data and messageData typingData representationData transfer syntax

Incompatibility issues of data passed from local machine to remote machine

Different byte orderingDifferent sizes of integers and other typesDifferent floating point representationsDifferent character setsAlignment requirements

9

RPC Operations - Data Conversion Needed standard encoding to enable communication between heterogeneous systems using symmetric conversion

ASN.1 (ISO Abstract Syntax Notation)Most important developments in standards.Used to define data structures.Used for specifying formats of protocol data units in network communications.Data types are checked during stub generation and compilation.

Sun’s RPC uses XDR (eXternal Data Representation)

10

RPC Operations - Data Conversion For example: IP (headers) forced all to use ‘big endian’ byte ordering

for 16 and 32 bit valuesMost significant byte in low memorySparc, 680x0, MIPS, PowerPC G5x86/Pentiums use little endian

Symmetric Conversion

Client data

big endian

Server data

little endian

Convert dataConvert data

htonl()… (host to network)

11

ntohl()… (network to host)

RPC Operations - Data Conversion Network Data Representation (NDR) is used in the Distributed Computing Environment (DCE)

Uses asymmetric “receiver-makes-right” Open Source Foundation’s Distributed Computing Environment RPC uses IDL (Interface Definition Language)

Client data

big endian

Server data

little endian

Convert data

12

Asymmetric Conversion

Binding is the process of connecting the client to the server

Services are specified by a server interface with interface definition language such as XDR.

RPC Operations - Binding

13

1. The server starts upRegister its communication endpoint by sending a request (program, version number, port number) to the port mapper .Port mapper manages the mapping.

2. Before RPC, client call RPC run-time library routine create, which contacts the port mapper to obtain a handle for accessing.

Create message contains the server name, program, version number, transport protocol.

3. Port mapper verifies the program and version numbers, returns the port number of the server to the client.

4. Client builds a client handle for subsequent use in RPC. This establishes socket connections between clients and server.

RPC Operations - Binding

14

client

port mapper

server

directory server

1. register2. create

3. port #

4. handle

Register service (if server is unknown)

Serv

er m

achi

ne a

ddre

ss o

r ha

ndle

to s

erve

r

15

RPC Operations - Binding

Compilation of RPC requires the followings:

1. An interface specification file2. An RPC generator : input is the interface

specification file and output is the client and server stub procedure source codes.

3. A run-time library for supporting execution of an RPC, including support for binding, data conversion, and communication

RPC Operations - Compilation

16

RPC Operations - Compilation

17

Interface Specification File

Server Stub

Server Code

RPC Generator

Language Compiler

Client Stub

Header File

Server Application

Language Compiler

Client Code

Client Application

RPC Errors and Failure – Exception HandlingExceptions are abnormal conditions raised by the execution of server stub and procedures.Exceptions that occur in server application, server stub, and server run-time library (above the transport layer) are propagated to the clientIf a server routine encounters an error that prevents it from executing the RPC, it raises an exception after cleaning up and before returning to the RPC run time, Server routine doesn’t return a value to RPC that only the server routine recognizes as an errorExamples of exceptions: Overflow/underflow, protection violation

18

RPC Errors and Failure – Exception Handling

19

Exceptions must be reported to the clients.Question: how the server report status information to clients?

A client may have to stop the execution of a server procedure.

Question: how does a client send control information to a server?

In local procedure call: global variables and signals.In computer network, the exchange of control and status information must rely on a data channel.

In-band signaling, or out-band signaling (flag).Separate channel (socket connection) – more flexible for RPCIt is implemented as part of the stub library support and should be transparent.

RPC Errors and Failure – Failure Handling

20

Problems caused by crashes of clients and servers, or the communication network.

Cannot locate the server Nonexistent server, or outdated programHandle like an exception.

Messages can be delayed or lostEventually detected by a time-out or by no response from the server

The messages can be retransmitted

RPC Errors and Failure – Failure Handling

21

Problem with Retransmission of requests. Delay case: server gets multiple requests Idempotent case: request executed multiple times with no change in resultIdempotent case of impossible (lock servers), each request has sequence number.

Typically RPC do not use sequence numbers – only requests-based.

RPC Errors and Failure – Failure Handling

22

Crash of a serverClient attempts to reestablish a connection, and retransmits its request.If server not fail, but TCP connection fail: examine the cache table for duplicated message.If server failed, then cache table lost. Then raise exception.

Three assumptions for RPC semantics in failures. Server raise exception, client retries. At least onceServer raise exception, client give up immediately At most onceNo error report from server, client resubmits until it gets or give up Maybe

RPC Errors and Failure – Failure Handling

23

Most desirable RPC semantics occur exactly onceBut hard to implement.Loss of cache table: at least once and log the cache table to storage.Reload the cache table when server recovers.Overhead since each service must be executed as a transaction at the server

RPC Errors and Failure – Failure Handling

24

Crash of a client processServer has an orphan communication and its reply is undeliverable. Orphan computation waste server resources, may confuse the client with invalid replies from previous connections.How to eliminate orphan communication? But hard to implement

Client: On reboot, cleans up all previous requests. Server: Occasionally locate owners of requests. Expiration: Each remote operation is given a maximum lifetime.

RPC Errors and Failure – Failure Handling

25http://www.cherrypy.org/chrome/common/2.2/docs/book/chunk/ch03.html

Security areas in RPC resulting fromRPC introduces vulnerability because it opens doors for attacks.RPC is a cornerstone of client/server communication

Primary security issuesAuthentication of processes.Confidentiality of messages.Access control authorization from client to server.

RPC Security

26

Authentication protocol for RPC should establish:

Mutual authentication.Message integrity, confidentiality, and originality.

Design of a secure authentication protocolHow strong the security goals.What possible attacksSome inherent limitations of the system.Short-term solution: additional security features.

RPC Security

27

Sun’s Secure RPCBuilt into Sun’s basic RPC.Assume a trusted Network Information Service (NIS), which keeps a database and secret keys.The keys are for generating a true crypto graphical session key.When user login, NIS gives the key. With user password, the key used to decrypt the secret key, discard password.Passwords are not transmitted in the network.

RPC Security – Sun Secure RPC

28

RPC Security – Sun Secure RPC

Sun secure RPC – RPC message may contain more

Timestamp : check message expirationNonce : protect against the replay of a messageMessage digest: detect any tampering.Sun secure RPC is simple, using existing NIS.

30

RPC Security – Sun Secure RPC

HistoryExamples of somewhat recent designs

GridRPCICE

Recent research activityMultiple GridFlexRPCSpartanRPCS-RPCSerial Multicast

Outline of RPC Current & Future

31

History of RPC70s Remote Procedure Call (RPC)80s Communication protocol models focused on the network layer

Microsoft's Distributed Computing Environment (DCE)Network File System (NFS) by Sun Microsystems

90s Object RPC (ORPC) codified the mapping of a communication endpoint to a language-level object

DCOM distributed extension builds an ORPC)layer on top of DCE RPC (Microsoft)Common Object Request Broker Architecture (CORBA) does the same on Unix systemsAny solution built on these protocols will be dependent on a single vendors implementation, Both these protocols depend on a closely administered environmentLate in 90s Doors (Solaris) for UNIX operating system

32

00s improved object and component Java’s Remote Method Invocation (RMI) is their traditional method of performing remote communications and uses a non-standardized protocolMicrosoft Transaction Server (MTS) component-based transaction processing systemDDObjects for Borland Delphi and C++ Builder based on DCOM, RMI, Corba

Late 00s Grid computing is composed of many networked loosely coupled computers acting together to perform very large tasks.

GridRPC distributes individual applications and serves as a base for higher –level substrates such as distributed scientific components, Ninf and NetSolveICE (Internet Communications Engine) for grid computing

History of RPC

33

GridRPCGrid computing distributes individual applications and serves as a base for higher–level substrates such as distributed scientific components, RPC much more complicated

Two different grid computing systems to look at: NetSolve and NinfBoth support remote procedure calls (RPCs) on the Grid

Standardized at GridRPCStandardizing only minimal set of APIsHigher-level features can be built on topProviding several reference implementations

Usage scenariosRemote library calls for executing compute-intensive tasksExecuting large scale task-parallel processing

34

GridRPC Ninf-G2

http://www.globus.org/grid_software/computation/ninf-g.php

Implementation of GridRPCConstructed on top of Globus, online grid computingGlobus is introduced in many sitesNo more security halls are required in introducing Ninf-G2Ninfied applications can be executed on many sites

Globus provides primitive functions/tools for grid computing

Concentrating on the development of RPC specific functionsEasy to use other tools on GlobusNinf-G2 hides complicated mechanisms of GlobusGrid applications/middleware can be easily constructed

35

ICE is object-oriented middleware providing RPC, grid computing, and publish/subscribe functionality.Influenced by CORBA (Common Object Request Broker Architecture) in its design, and developed by ZeroC, Inc. Supports C++, Java, .NET-languages, Objective-C, Python, PHP, and Ruby on most major operating systems.

36

ICE (Internet Communications Engine)

ICE (Internet Communications Engine)

37http://en.wikipedia.org/wiki/Internet_Communications_Engine

ICEIce middleware in the New Solar Telescope’s Telescope Control System, 2008 [5]

NST (new solar telescope) is an off-axis solar telescope with the world largest aperture.Develop TCS (telescope control system) to control all aspects of the telescope

Ice advantagesProvides Simple ,fast and scalable communications.Ice Embedded (Ice-E) supports Microsoft Windows Mobile operating system for handheld devices.Source code of Ice is provided under the GNU (General Public License)Continuously updated.

38

ICETCS Implementation (sample of RPC)

Star-like structure: all subsystems through HQ (headquarters).Each subsystem acts as a server and a client.Each subsystem use the same ICE interface.Interface for every object includes seven operations; Register, Unregister, SendError, SendCommand, RequestInformation, SendNotification, SendReply, ErrorSubsystems can be started in any order, only need to register with HQ and IcePack registry.

39

Integrating Computing Resources on Multiple Grid-enabled Job Scheduling Systems Through a Grid RPC System 2006

Framework for parallel programming by RPC Bridge created for large scale computing resource pools managed by multiple grid enabled job scheduling systems User can exploit remote servers, clusters, and computing resources on grid-enabled job scheduling systems located on different sites

New Research: Multiple Grid (2006)

40

Nakajima, Y.; Sato, M.; Aida, Y.; Boku, T.; Cappello, F.; , "Integrating Computing Resources on Multiple Grid-enabled Job Scheduling Systems Through a Grid RPC System," Cluster Computing and the Grid, 2006. CCGRID 06. Sixth IEEE International Symposium on , vol.1, no., pp. 296- 300, 16-19 May 2006

New Research: FlexRPC (2007)FlexRPC, a flexible user-level RPC system that enables to develop high-performance cluster file systems easily.

Ensures client-side thread-safeness and fully supports multithreaded RPC servers. Parallel and serial multicasting mechanisms Concurrent call requests handled by worker threads on client and server sideRemote procedures are designed to be similar to SunRPCImproves latency and bandwidthBuilding a working prototype of cluster file system called Kadoop on top of FlexRPC.

41

Sang-Hoon Kim; Youngjae Lee; Jin-Soo Kim; , "FlexRPC: A flexible Remote Procedure Call facility for modern cluster file systems," Cluster Computing, 2007 IEEE International Conference on , vol., no., pp.275-284, 17-20 Sept. 2007

RoutingControl plane, part of the router architecture, is concerned with drawing the network map, or the information in a routing table that defines what to do with incoming packets.

Multicast Communication between a single sender and multiple receivers over a networkRouting requires an additional table for multicast routes.

Serial MulticastingInstead of sending a RPC request to multiple receivers, each receiver forwards the RPC request to the “closest” receiver in the network topology that has not received the request

New Research: Serial Multicast Info

42

Modeling and evaluation of serial multicast remote procedure calls (RPCs)

Serial multicast RPC are needed in many distributed systems to avoid network bottlenecks and high-latency linksPresent a mathematical model of serial multicast calls to better understand performance behaviorModel is very effective in predicting the performance of serial multicast RPCs.

New Research: Serial Multicast (2009)

43

Sang-hoon Kim; Jin-Soo Kim; Seungryoul Maeng; , "Modeling and evaluation of serial multicast remote procedure calls (RPCs)," Communications Letters, IEEE , vol.13, no.4, pp.283-285, April 2009

Wireless sensor network (WSN) consists of spatially distributed autonomous sensors to monitor physical conditions

New Research: WSN Info

44http://en.wikipedia.org/wiki/Wireless_sensor_networkhttp://embedsoftdev.com/embedded/wireless-sensor-network-wsn/

Spartan RPC: Secure WSN middleware for cooperating domains

Middleware technology for wireless sensor network (WSN) applications supporting cooperation between distinct protection domains SpartanRPC extends the nesC programming languageProvides a link-layer remote procedure call (RPC) mechanismIncorporates capability-based security architecture for protection of RPC resources in heterogeneous environments

New Research: SpartanRPC (2010)

Chapin, P.; Skalka, C.; , "SpartanRPC: Secure WSN middleware for cooperating domains," Mobile Adhoc and Sensor Systems (MASS), 2010 IEEE 7th International Conference on , vol., no., pp.61-70, 8-12 Nov. 2010

45

S-RPC, lightweight framework for remote procedure calls on wireless sensor and actuator nodes

Enables seamless interoperability between nodes with common interface to remote method invocations Previous communication between embedded devices took place over proprietary protocols tailored for the given use case which limits integration of new functionsS-RPC allows unified access to functions on remote devicesPermits dynamic integration of new devices with different functions into the network

New Research: S-RPC (2011)

46

Reinhardt, A.; Mogre, P.S.; Steinmetz, R.; , "Lightweight remote procedure calls for wireless sensor and actuator networks," Pervasive Computing and Communications Workshops (PERCOM Workshops), 2011 IEEE International Conference on , vol., no., pp.172-177, 21-25 March 2011

1. Ice in Wikipedia, http://en.wikipedia.org/wiki/Internet_Communications_Engine

2. Interprocess Commnications, http://en.wikipedia.org/wiki/Interprocess_communication

3. Randy Chow, Theodore Johnson, “Distributed Operating Systems & Algorithms”, 1997.

4. Shumko, Sergij. "Ice middleware in the New Solar Telescope's Telescope Control System". Astronomical Data Analysis Software and Systems XVII, ASP Conference Series, Vol. XXX, 2008., Canada.

5. Sun RPC, www.cdk4.net/additional/rmi/Ed2/SunRPC.pdf 6. Zeros, Inc. http://zeroc.com/ice.html

References