Post on 27-Mar-2015
transcript
1
Chapter 4: Client server Programming
With RPC
Rufin Soh
INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
2
Content
1. RPC fields of applications
2. Distributed systems challenges
3. Alternatives to DS challenges ?
4. RPC
a. Why RPC?
b. RPC: definitions
c. RPC working mode
d. RPC model
e. RPC architecture
f. Execution mode
g. Calling Mode
11. SUN RPCa. The remote program b. Programs Identificationc. The port mapperd. XDR
e. Parameters exchanges
f. Passing parameters
g. Messages format
h. Rpcgen
12. Development with RPC
13. RPC server types
14. Remote PC vs Local PC
15. Failures with RPC
16. RPC lacks
17. RMI
18. Idempotent operations
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
3
RPC fields of applications
Transport and delivery Inventories
Cash and billing
Accounting
manufacturing
EngineeringSales
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
4
Content (1)
1. RPC possible fields of applications
2. Distributed systems challenges
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
5
Distributed systems challenges
Distributed systems are applications that run in different namespaces or
hosting computers. Their Inherent problems are:
Communication:
• Transmission of information between softwares.
Heterogenous environment:
• Manage the diversity among hardwares and softwares that interact.
Intégration
• Preserv the patrimony for legacy applications
Interoperability
• Exchange data between distributed applications from different vendors.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
6
Content (2)
1. RPC possible fields of applications
2. Distributed systems challenges
3. Alternatives to DS challenges ?
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
7
Alternatives to DS challenges ?
To overcome the following challenges we need:
A normalized network infrastructure
• OSI, TCP IP model.
Normalized communication protocols
• IP (for packets transport), TCP IP connexions(for streaming e.g. sockets).
• HTTP, FTP, CGI, at the application layer.
Normalized mechanism for components (applications) interaction
• Remote procedure call is the fundamental paradigm for distributed
programming as well as procedure call is for structured programming.
Normalized conversion functions
All this implemented on different hardware and software platforms, with the
illusion of a unique platform
• The distributed platform: the middleware, the RPC middleware.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
8
Content (3)
1. RPC possible fields of applications
2. Distributed systems challenges
3. Alternatives to DS challenges ?4. RPC
a. Why RPC?
b. RPC: definitions
c. RPC working mode
d. RPC model
e. RPC architecture
f. Execution mode
g. Calling Mode
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
9
Why RPC?
Advantages of RPC:
• The goal is to inherit as far as possible the conventional procedure call
semantic in a different environment.
• Encapsulates the communication details, making no difference between local
and a remote calls (or nearly…).
• Widely used by programmers.
• Lets you achieve nearly all the following distributed systems challenges
without using many different development tools.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
10
Introduction to RPC
Remote Procedure Call (RPC) : is the technology used to execute procedures
on remote environments.
A distributed application designer may work following two approaches:
Communication oriented design:
• Define the application protocol(messages format and syntax ), that
interwork between the client and the server
• Design the client and server components and describe how they react
to input messages and emit output.
• e.g: HTTP, FTP.
Application oriented design:
• Build a mono-computer conventional application.
• Then divide it into many modules that will run on different machines.
• e.g: RPC.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
11
RPC working modeClient Request
1. The call is detected as an RPC call.2. On A, the Call parameters are passed and placed in a data structure
(Marshalling) waiting to be transferred on the network3. An RPC identifier is generated for this Call4. A timer is initialized at A and set5. Data are transferred on the network to B, through the Operating System
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
Caller program
Call
RPC Service
A
E
Low level protocols
NETWORK
Low level protocols
Call
RPC Service
B
D
Called procedure
return
C
Caller System (Client) Called System (Server)
1
5
1
2 Marshalling3 RPC identifier4 Timer
12
RPC working mode (2)
Server process and response6. Prameters are extracted from the network buffer in a useful
format(Unmarshalling) for Call to be made7. RPC identifier is recorded8. The Call is processed at C and return to the caller at D9. The results to return are placed in a special data structure (Marshalling)
waiting for transfer10. The timer is initialized
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
Caller program
Call
RPC Service
A
E
Low level protocols
NETWORK
Low level protocols
Call
RPC Service
B
D
Called procedure
return
C
Caller System (Client) Called System (Server)
1
5
1
6 Unmarshalling7 Record RPC identifier
89 Marshalling10 Set Timer
13
RPC working mode (3)
Client ACK11. Data are sent on the network (through the server operating system APIs) and
transferred to E12. Results are extracted from the network buffer in a data structure
(Unmarshalling)13. The Client-side timer is desactivated14. An acknowledgement is sent to the server with the corresponding RPC identifier.15. The server side timer is desactivated on D
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
Caller program
Call
RPC Service
A
E
Low level protocols
NETWORK
Low level protocols
Call
RPC Service
B
D
Called procedure
return
C
Caller System (Client) Called System (Server)
1
5
12 Unmarshalling13 Desactivate timer
8
15 Desactivate Timer
11
14
14
RPC modelThe RPC model uses an «application oriented design» approach that enables many
procedures to run on remote sites.
The remote procedure call is the client request, and the procedure return is the server
response.
A procedure call follows a synchronous working mode(blocking tasks): the next
instruction after the procedure call can not be executed before the called procedure
ends.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
procA()
return
procB()
returnreturn
Procedure B (server)
Procedure A (server)Main Program
Machine 1 Machine 2Network Machine 3Network
15
RPC architecture
Marshalling Unmarshalling
SendRequest() ReceiveResponse()
Application
Procedure call
Procedure return
Marshalling Unmarshalling
SendResponse() ReceiveRequest()
Procedure
Procedure return
Run procedure
Client stub Server stub
Client RPC Server
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
16
Execution mode
Procedures calls are made on the following model:
local variables main() SP
local variables main()bytes
buff
fd
Return adress
local variablesread() SP
How to Pass parameters :• per value:initialize a procedure local variable• per reference: use a pointer, pointing on the parameter variable • by copy of reference: a copy at the call and a copy at the return
N = read(fd,buff,nbytes)
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
17
Calling Mode
Client
Client Stub
Marshalling parameters
Unmarshalling results
core
Server
ServerStub
Marshalling results
Unmarshalling parameters
core
Call Return Return Call
The machine hosting the procedure on the network remains transparent to the client
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
18
Content (4)
1. RPC possible fields of applications
2. Distributed systems challenges
3. Alternatives to DS challenges ?
4. RPC
11. SUN RPCa. The remote program b. Programs Identificationc. The port mapperd. XDR
e. Parameters exchanges
f. Passing parameters
g. Messages format
h. rpcgen
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
19
SUN RPC
Sun Microsystems has developed an RPC technology named the « Sun
RPC ». It is Todays de facto standard. NFS is built on top RPC.
Sun RPC defines:
The messages format that the caller(client) transmit to start the
remote procedure on the server.
The parameters format
The results format.
UDP and TCP (OSI layer …?) protocols are used for communication and
XDR presentation protcol (TCP-IP layer …? Give the answer before continuing) enables
RPC to work in an heterogenous environment.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
20
The remote program
(!!! not procedure)
The remote program is the logical software or unit, running on a remote machine(??? A machine different from the one in front of you).
A remote program is a server with its own procedures and its own data.
E.g a Database server
insert procedure delete procedure lookup procedure
Databases
Every remote program is identified by a unique 32-bits integer used by the caller.
A remote program procedures are sequecially identified by the integers 1,2,…,N
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
21
The remote program (2)
RPC communication mode « at least one transmission»:
If a remote procedure call running on UDP does not return a
value, the caller couldn’t know if the procedure has been
executed or if the answer is lost (why?… because UDP is best effort, it
never promises nothing as compare to who??? Its brother TCP).
Futhermore, nothing tells you that the procedure has not been
executed many times, due to duplicated requests.
The communication mode is a client sever mode(Blocking or not???).
The caller specified the server address(IP, port).
But which server? Of course the one hosting the remote program.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
22
The remote program (3)
A port number is dynamically allocated to any remote program, and the
corresponding RPC client is notified:
Every computer that offers RPC programs hosts a dynamic port association
service: The port mapper.
When an RPC program starts on the server, it dynamically allocate a local
port number, then inform the port mapper on the hosting machine, of the
RPC program identifier\port number association.
The port mapper will maintains a databases table of the different
associations.
Any client who want to contact an RPC program on a M machine will first
ask for its associated communication port to the M machine’s port mapper.
The port mapper always runs on communication port 111.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
23
Programs Identification
A remote procedure is identified by the trio (prog, vers,proc)
Prog identifies the remote program
Vers is the version (of what??? Of prog…of course)
Proc is the procedure (which one ??? The one you may want to call if you are an RPC client).
Address groups for remote programs:
Name identifier descriptionportmap 100000 port mapperrstat 100001 rstat, rup, perfmeterruserd 100002 remote usersnfs 100003 Network File Systemypserv 100004 Yellow pages (NIS)mountd 100005 mount, showmountdbxd 100006 debuggerypbind 100007 NIS binderetherstatd 100010 Ethernet snifferpcnfs 150001 NFS for PC
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
24
The port mapper
RPC server Program
Port Mapper
The program transmits
the trio (ident, RPC, port)
socket allocated to the RPC program
Port Mapper’s socket = 111
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
25
XDR
RPC messages format length is generally variable. Messages fields are
specified in XDR (eXternal Data Representation) language.
XDR : Data representation as defined by SUN Microsystems
Defines how data are going to be routed on the network
Enables data exchange between computers with differents internal data
representations.
E.g. a 32 bits integer with a value of 260 will have the 2 following
representations:
0014 for a « big endian » type machines, that is a machine with lower
addresses on MSB and higher addresses on LSB.
4100 for « little endian » type machines.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
26
XDR (2)
XDR data encoding format only contains represented data without any
information on the data type (as compared to ASN1).
If an application work with 32-bits integer, the encoding result will be stored
exactly on 32 bits with nothing to tell you its type (if it is an integer or …).
This encoding scheme implied that both the client and the server must agree on
the exact data format to be exchanged.
An XDR conversion fonctions library enables applications designers to use a
standard software on any type of computer.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
27
Communication between the client and the server is done through a message containing the name of the procedure, and grouped parameters (marshalling )
n=sum(4,7)
core
sum(I,j){ int I, j; return (i+j) ; }
sum
4
7
sum
4
7
core
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
Parameters exchanges
28
Passing parameters
For a per value parameters passing, the parameters copies are
sent in the message.
For a per reference parameters passing (e.g. a vector or table in
C programming language), there are many solutions among
which:
Tables may be sent as copies of their values. So the server
will replace each of those values with a pointer, work with the
pointer and return the table to the client who will use the
initial pointer (copy/restore).
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
29
Messages format
Message ID
Message type
RPC Version number
REMOTE Program
REMOTE program version
REMOTE Procedure
Authentication
Procedure arguments
The format is of variable length, because the number of arguments in the called procedure can not be pre determined.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
30
rpcgenRpcgen is a software generator tool, generating:
The client stub
The server skeleton
XDR procedures for results and parameters
A file containing common definitions
SUN provides a complete methodology with:
Conversion routines for simple types
XDR routines for formatting complex types (tables and structures), used
in RPC message definition.
Run time RPC functions that makes possible a remote procedure call by
a program, a service record on the port mapper, dispatching a request
for procedure call to the corresponding procedure inside the remote
program.
Example of run time function is
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
31
rpcgen (1)
Q.x
rpcgen
Q_clnt.c
Q.h
Q.xdr.c
Q.svc.c
compiler
Client application Client interface
client
compiler
remote procedures Server interface
server
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
32
rpcgen (2)
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
Rpcgen is a procompiler which generates many files from a synthetic description of
a program in an “Q.x” file
The description language used resemble C
The resulting files are:
o A file to include into both server and clients codes (Q.h)
o A server code skeletton (Q_svc.c)
o Procedures that actually handle client remote calls (Q_clnt.c)
o XDR filters (Q_xdr.c)
So the programmer will only have to write two files:
o A file containing the implementations of the procedures provided by the server,
that will complete the “Q_svc.c” file which already contains the main function.
o The client code that will be the “Q_clnt.c”, and in which the programmer will add
a main method.
33
rpcgen (3)
The methodology consist of developing a distributed application exactly as if you was
developing a local conventional application, then defines the procedures which are
going to be ran remotely.
This decoupling implied adding codes between the call of the procedure and the remote
procedure itself:
On the Client side the new code must: Encode parameters Create an RPC CALL message Transmit this message to the
remote program Wait for the results and
decodes them according to the internal representation of the local machine
On the server side the new code must: Accept an RPC request Decodes parameters according to the internal representation of the local machine
Dispatch the message towards the adequat procedure.
Build the response and encodes it. Transmit the encoded message to the client
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
34
rpcgen (4)The stubs procedures replace the following conventionnal procedures:
Calling procedure on the client side Procedure being called on the server side
Proc A1 Proc A2
client stub for B2
client stub for B2
Dispatcher
server stub for B1
server stub for B2
Proc B1 Proc B2
The « stub » procedure is exactly named as the original calling procedure. That helps to keep the same source in the distributed version. e.g. the « stub » procedure for B2 is named B2, and the « stub » procedure for B1 is named B1.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
35
rpcgen (5)
Rpcgen splits every « stub » in two parts:
An entity that is common to all the applications, and which use
to build the client server communication.
An entity belonging to the application, providing an interface
with the application.
This separation is justified because rpcgen uses « communication
package » procedure call convention whereas it allows the user to
remote procedures call conventions.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
36
rpcgen (6)
Proc A
client Interface
Clientcomm. Proc B
ServerInterface
server comm.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
37
Content (5)
1. RPC possible fields of applications
2. Distributed systems challenges
3. Alternatives to DS challenges ?
4. RPC5. SUN RPC
6. Development with RPC
7. RPC server types
8. Remote PC vs Local PC
9. Failures with RPC
10. RPC lacks
11. RMI
12. Idempotent operations
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
38
Development with RPC
interface Definition interface Definition
Client StubClient Stub Server StubServer StubHeader fileHeader fileClient CodeClient Code Server CodeServer Code
Client ObjetClient Objet Client Stub Objet Client Stub Objet Server ObjetServer ObjetServer Stub ObjetServer Stub Objet
LibrairyLibrairy
CC
Linker Linker
CC CC CC
rpcgen
ClientClient ServerServer
LibrairyLibrairy
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
39
RPC server types
Two types of RPC servers :
Stateful:
• The server keeps informations on the different clients
transactions. Those informations may serv to repeat
operations that have been already been executed once.
Stateless:
• There is no memory of information on client operations.
The client is responsible of that, it could makes it
operation idempotent itself.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
40
Remote PC vs local PC
Differences between remote procedures and local procedures calls are:
The waist of time in the network (network delay) may generate very
important processing delay.
A remote procedure call may contain pointer types parameters,
not easy to handle as on local procedure call.
Access to Input and output are denied to remote procedures, thus
they can not use local peripherals (e.g. unable to write error
messages)
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
41
Failures with RPC
Many failures types may occur:
1. The client is unable to locate the server
The server is not availaible
The Stubs versions are not adequat
2. The client request towards the server is lost:
After a timeout, the client retransmits the same request
3. The server response from the server to the client is lost:
The client will retransmit the same request.
No problem if it is an idempotent operation
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
42
Failures with RPC (2)
4. The server is down after receiving the client request (3 cases):
• At least once:
– When the machine restarts, retransmits the requests many times until the response arrives.
• At most once:
– Abandon the operation. The operation is done either once or never.
• Exactly once:
– Keep in memory the state of the operations that have been executed. Not easy to implement.
Receives
Executes
Respond
Req
Rep
Receives
Executes
Failure
Req
Receives
Failure
Req
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
43
RPC lacks
RPC lacks:
RPC is for structured programming
Parameters and return values are primitives types
Depends on the server localisation
No remote pointer
Evolutions
CORBA
Multilanguage, multi-platform (architecture+OS), MuliVendors (Chapter 11)
Java RMI
mono-language : Java, multiplatform : work on top JVM (Chapter 9)
DCOM / Object RPC
multi-languages, mainly Win32 platform, il existe des
There are some (non-MicroSoft) implementations for Unix, but proprietaries
SOAP (Simple Access Object Protocol)
multi-languages, multi-platform
Uses XML (with SOAP DTD) in request and response
Messages are transported on HTTP
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
44
RMI: Remote Method Invocation
The Object Oriented Programming RPC is Java Remote Method
Invocation (RMI).
It is an approach for executing code on other machines across
a network.
You can send a message to the remote object and get a result
as if the object lived on your local machine.
In Chapter 9 we will walk you through the steps necessary to
create your own RMI objects.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES
45
Idempotent operations
Idempotent operations are the one that can be repeated without
side effects:
Idempotent operation:
• Read the 1024 first records of a file
Non Idempotent operation:
• Transfer an amount of $2000 from an account to another.
A.Obaid-Wilfried Probst – Rufin Soh INE4481 DISTRIBUTED DATABASES & CLIENT-SERVER ARCHITECTURES