Post on 26-Jun-2018
transcript
A Logical Look at Recent Messaging Systemsand Standards, and a Look Beyond∗
Steven O. Kimbrough565 JMHH
kimbrough (a) wharton.upenn.edu215-898-5133
29 November 2006, Amsterdam, Free University
Acknowledgements: Andrew J.I. Jones, David Eyers, and previously, Yao-Hua Tan.
$Id: freeunl-29112006-foils.tex,v 1.5 2006/11/28 21:37:15 sok Exp $
∗File: freeunl-29112006-foils.tex/pdf.
Abstract
Messaging systems have come of age. Conceptual and technical advances have
led to the maturing of message oriented middleware (MOM) systems and to a leap
into the next generation of EDI (electronic data interchange), based upon improved
standards and conversion to XML. This talk briefly reviews the state of play and
considers the prospective convergence of MOM and EDI. Recent messaging standards
are examined through the example of ISO 20022 efforts for financial EDI. These efforts
have been impressively productive in detailing requirements and specifying usable XML
representations. Abstracting the results with logic, however, reveals both limitations in
the present approach and opportunities for improvements. The talk emphasizes one such
aspect: the support a logical point of view gives for the problem of the dynamic aspects
of message protocols, the fact that they will continue to require changes. In this regard,
the talk connects the design of message protocols with recent thinking on systems analysis
and design, which emphasizes IID, incremental and iterative development.
1
Current (Best) Practice
Two principles:
1. Proposals for change are (usually) more secure and rightly credible to theextent that they show us how to improve on current practice, especiallycurrent best practice.
2. New concepts and ideas are very often realized by abstracting andgeneralizing current practice.
It is in this way and with this spirit that the work described in this talk isundertaken. Here, focus on recent messaging standards in EDI, specificallyfinancial services messages, led by SWIFT. These represent the best ofcurrent practice and are important achievements on their own.
2
Two Developments
• MOM (Message Oriented Middleware) comes of age (beginning in the1980s)
Vendors/products include: Microsoft BizTalk, IBM WebSphere MQ, Sun,JMS, Java Message Service, TIBCO, WebMethods, SeeBeyond, Vitria,etc. (NB, JMS universal except for Microsoft)
• A new generation of messaging standards, particularly for EDI (broadlyconstrued)
Focus here: ISO, in particular ISO 20022 for financial services
3
MOM: Message Oriented Middleware
Useful, but not definitive:
http://en.wikipedia.org/wiki/Message oriented middleware
Thorough:
Enterprise Integration Patterns: Designing, Building, and DeployingMessaging Solutions (2004) by Gregor Hohpe, Bobby Woolf et al.
Relevant and useful:
The Power of Events: An Introduction to Complex Event Processingin Distributed Enterprise Systems (2002) by David Luckham
4
MOM (from the Wikipedia article)
“Message-oriented middleware comprises a category of inter-application communication
software that generally relies on asynchronous message-passing as opposed to a
request/response metaphor.
“Most message-oriented middleware (MOM) depends on a message queue system,
although some implementations rely on broadcast or on multicast messaging systems. . . .
“Middleware arrived on the computing landscape comparatively late. It emerged in the
1980s as a solution to the problem of how to link new applications to older legacy systems.
It also facilitated distributed processing the connection of multiple applications together
to create a larger application, usually over a network.. . .
“The primary advantage of a message-based communications protocol lies in its ability
to store, route or transform messages in the process of delivery.”
5
MOM (from Enterprise Integration Patterns)
“Messaging is a technology that enables high-speed, asynchronous, program-to-
program communication with reliable delivery. Programs communicate by sending packets
of data called messages to each other. Channels, also known as queues, are logical
pathways that connect the programs and convey messages. . . .
“The message itself is simply some sort of data structure—such as a string, a byte
array, a record, or an object. It can be interpreted simply as data, as the description of a
command to be invoked on the receiver, or as the description of an event that occurred in
the sender.” [page xxxi]
“A messaging system is needed to move messages from one computer to another
because computers and the networks that connect them are inherently unreliable.” [xxxii]
Message semantics (and other little details) are (properly) outside the scopeof MOMs.
6
MOM (from Enterprise Integration Patterns)
• Emphasize: Messaging systems are for asynchronous communication
• Aka: “send and forget”
NB. (not quite, but. . . user can track but does not wait)
• NB. MOMs do not track at the application level for the users
E.g., dialog management is outside the scope of MOMs
7
MOM (from Enterprise Integration Patterns)
On design tools and methods:
“To our knowledge, there is no widely used, comprehensive notation that is geared
toward the description of all aspects of an integration solution. The Unified Modeling
Language (UML) does a fine job of describing object-oriented systems with class and
interaction diagrams, but it does not contain semantics to describe messaging solutions.
The UML Profile for EAI . . . enriches the semantics of collaboration diagrams to describe
message flows between components. This notation is very useful as a precise visual
specification that can serve as the basis for code generation as part of a model-driven
architecture (MDA). We decided not to adopt this notation . . . ” [page xliv]
Remember: This pertains to MOMs and messaging systems, NOT(even!) to the semantics or constitution of the message protocols/contentsthemselves. There is much that needs doing.
8
Our Interest
• Is focused on the design of the messages themselves and on theirprocessing (creation, interpretation, dialogs).
• MOMs add value in practice, they enable and facilitate (potentially atleast) COAs (communication oriented applications— ;) ).
• EDI standardization is an instance par excellence
• We’ll look in particular at ISO 20022, UNIFI, “UNIversal FinancialIndustry message scheme”
http://www.iso20022.org/ and especially
http://www.iso20022.org/index.cfm?item id=42790
9
But first. . .
• “Introduction to ISO 20022 — UNIversal Financial Industry messagescheme” at www.iso20022.org,
also Scripted-wwwiso20022org-20061106.ppt
10
Comments
• Focus, wlog, on “Payments Standards – Clearing and Settlement”
http://www.iso20022.org/index.cfm?item id=60053
• Key elements: Diagramming of business contexts and processes; inmessage protocol (i) fixing reference, (ii) message structure (header +body), (iii) constraints on header-body combinations.
11
Critique
This in many ways represents solid progress and the codification of muchcareful thought. So what’s not to like?
Answer:
1. Change is cumbersome: “the dynamic point”
2. Limited scope of constraints: “the constraint generality point”
3. Need for a more robust and extensive account of message meaning: “thesemantics and inference point”
12
The dynamic point, part 1: fixing reference
See: “On Representing Special Languages with FLBC: Message Markersand Reference Fixing in SeaSpeak,” by Steven O. Kimbrough, Yinghui(Catherine) Yang, FMEC book, pp. 297f.
The 567 page specification document, “Payments Standards – Clearing andSettlement,” devotes about 55 pages to specifying how to make referenceto various sorts of things.
. . . And that’s not all.
13
Ways of referring
1. With names, whose meaning is conventionally assigned; proper names.
2. With data structures, whose components are combined. E.g., Surname,given name; dates and time stamps
3. By ostension, pointing. See http://en.wikipedia.org/wiki/Ostensive definition
4. With demonstratives: this, that, these, those
5. By description; Russell and definite description
(1) and (2) only in ISO 20022. NB. Importance of events and naming them.
14
ISODate, p. 447
An unexceptional case:
2.1 ISODate
Definition: Date within a particular calendar year represented byYYYY-MM-DD (ISO 8601).
Example: 2002-02-25
Note:
• Use of an exogenous convention
• Implicit agreement [apparently] in ISO 20022 to use that convention,even if its contents change
15
BICs, page 463: 2.1.3 BIC <BIC>
Presence: [1..1]This message item is part of choice 2.1.2 FinancialInstitutionIdentification.Definition: Bank Identifier Code. Code allocated to financial institutionsby the Registration Authority, under an international identification scheme,as described in the latest version of the standard ISO 9362 Banking(Banking telecommunication messages, Bank Identifier Codes).Data Type: BICIdentifierFormat: [A-Z]{6,6}[A-Z2-9][A-NP-Z0-9]([A-Z0-9]{3,3}){0,1}Rule(s): BICValid BICs are registered with the ISO 9362 Registration Authority, andconsist of eight (8) or eleven (11) contiguous characters comprising thefirst three or all four of the following components: BANK CODE,COUNTRY CODE, LOCATION CODE, BRANCH CODE. The bank code, country codeand location code are mandatory, while the branch code is optional.
16
Comments on BIC
Note:
• Use of an exogenous convention
• Implicit agreement [apparently] in ISO 20022 to use that convention,even it its contents change
So, this is also good. But, what if there is a need to communicate with orabout a financial institution that doesn’t have, or yet have, or can neverhave a BIC?
17
FinancialInstitutionIdentification, page 463
Definition: Unique and unambiguous identifier of a financial institution,as assigned under an internationally recognised or proprietary identificationscheme. Type: This message item is composed of one of the followingFinancialInstitutionIdentification5Choice element(s):
Ref Or Message Item <XML Tag> Mult. Represent./
Type
2.1.3 {Or BIC <BIC> [1..1] Identifier
2.1.4 Or ClearingSystemMember-
Identification <ClrSysMmbId> [1..1]
2.1.7 Or NameAndAddress <NmAndAdr> [1..1]
2.1.18 Or ProprietaryIdentification <PrtryId> [1..1]
2.1.21 Or} CombinedIdentification <CmbndId> [1..1]
18
ProprietaryIdentification, page 466
2.1.18 ProprietaryIdentification <PrtryId>Presence: [1..1]This message item is part of choice 2.1.2FinancialInstitutionIdentification.Definition: Unique and unambiguous identifier, as assigned to afinancial institution using a proprietary identification scheme.Type: This message item is composed of the followingGenericIdentification3 element(s):
Ref Or Message Item <XML Tag> Mult. Represent./
Type
2.1.19 Identification <Id> [1..1] Text
2.1.20 Issuer <Issr> [0..1] Text
19
Comments
• GenericIdentification3???!!! (Such numbering appears throughoutthe standards.)
• See also: Page 464, 2.1.6 Proprietary.
• Back to FinancialInstitutionIdentification. Notice it’s a disjunction. Whyinsist on identifying all of the disjuncts? Why insist on either (i) goingback to the standards process or (ii) using a proprietary code (and allthe mess that invites)?
But, what’s the alternative?
20
Abstract and extend the concept of a protocol
• What’s needed is a standard for extending the standard. Concept:
protocol (or convention) = kernel protocol + rules of extension
• How might this work? Lots of ways. Whenever two communicants wantto add a new ID, there is identified a standard way of publishing it, sayon the Web, and accessing it and its definition, e.g., with a URI. (Hello,XML!)
What would it take to make this work? (discuss)
21
Comments (con’t.)
• There are lots of examples like this in the naming portion of the ISO20022 standards (and even in the current document). Implementing theabstraction (kernel + rules of extension) for naming would have extensiveeffects.
• Now let’s look elsewhere in the standards, at the message formationrules.
Wlog, I’ll focus on just one message type: FIToFICustomerCreditTransferV01.
22
The dynamic point, part 2: changing structure
FIToFICustomerCreditTransferV01, pp. 42f.:
“The FinancialInstitutionToFinancialInstitutionCustomerCreditTransfermessage is sent by the debtor agent to the creditor agent, directlyor through other agents and/or a payment clearing and settlementsystem. It is used to move funds from a debtor account to a creditor.”
23
Structure of a FIToFICustomerCreditTransferV01message
Outline
The FIToFICustomerCreditTransfer message is composed of 2 building blocks:
A. Group Header
This building block is mandatory and present once. It contains elements such as
MessageIdentification, CreationDateAndTime.
B. Credit Transfer Transaction Information
This building block is mandatory and [may be] repetitive. It contains elements
related to the debit and credit side of the transaction such as Creditor, CreditorAgent,
Debtor, DebtorAgent.
24
Sizes of the 2 building blocks, as specified
• A. Group Header. 31 “message items”. Many optional. Some complex(containing multiple message items).
• B. Credit Transfer Transaction Information. 95 “message items”. Manyoptional. Some complex (containing multiple message items).
Known problems, based on past experience: (i) Even this many items isunlikely to prove complete, (ii) To avoid having to go back to the standardsbodies and having the burden of delay, users will typically use a messageitem (say one containing free text) for other than its intended purpose.XML doesn’t address this issue, and can’t.
25
There are naming issues as above, e.g. ClearingSystem,pp. 50f.
1.11 ClearingSystem <ClrSys>
Presence: [0..1]
Definition: Specification of a pre-agreed offering between clearingagents or the channel through which the payment instruction is processed.Type: This message item is composed of one of the followingClearingSystemIdentification1Choice element(s):
Ref Or Message Item <XML Tag> Mult. Represent./
Type
1.12 {Or ClearingSystemIdentification <ClrSysId> [1..1] Code
1.13 Or} Proprietary <Prtry> [1..1] Text
26
OK, ClearingSystemIdentification? Pp. 51f.
1.12 ClearingSystemIdentification <ClrSysId>
Presence: [1..1]
This message item is part of choice 1.11 ClearingSystem.
Definition: Infrastructure through which the payment instruction isprocessed.
Data Type: Code
One of the following CashClearingSystem3Code values must be used:
[And there follows a 3 page table of country-related codes. Absencesinclude Iraq, Lebanon, Israel. . . And if there are additions?]
27
Examples. InstructionPriorityInstructionPriority, p. 56.Why just 2? See 1.22–1.24 for repetition of the mistake.
1.21 InstructionPriority <InstrPrty>
Presence: [0..1]
Definition: Indicator of the urgency or order of importance that the instructing party
would like the instructed party to apply to the processing of the instruction.
Data Type: Code
When this message item is present, one of the following Priority2Code values must be
used:
Code Name Definition
HIGH High Priority level is high.
NORM Normal Priority level is normal.
28
What can be done?
• Much as before with fixing reference: abstract and define a broaderconcept of a protocol.
29
Abstract and extend the concept of a protocol
• What’s needed is a standard for extending the standard. Concept:
protocol (or convention) = kernel protocol + rules of extension
• How might this work? Lots of ways. Whenever two communicants want to add a new
message item [name], there is identified a standard way of publishing it, say on the
Web, and accessing it and its definition, e.g., with a URI. (Hello, XML!)
What would it take to make this work? (discuss)
Smallish practical point: No longer can we rely on strict positioning of message items.
(Why?). This is hardly an insurmountable problem (although it would make heavier
processing demands). One approach is to use “manifest types” or something like that:
a message item outside the kernel could be presented with its defining URI preceding
it. (Hello, XML and namespaces!)
30
The constraint generality point (our 2nd of 3)
The FIToFICustomerCreditTransferV01 message definition comes with“rules” that define syntactic constraints (pp. 42f.). Examples:
InstructedAgentRule
If GroupHeader/InstructedAgent is present, thenCreditTransferTransactionInformation/InstructedAgent is not allowed.
InstructingAgentRule
If GroupHeader/InstructingAgent is present, thenCreditTransferTransactionInformation/InstructingAgent is not allowed.
(There are dozens of these.)
31
Briefly. . . (this is a huge topic)
• All of the constraints are hard, exceptions are all “not allowed”. Why beso limited? Why not soft constraints, warnings that can accumulate?
• All of the constraints are syntactically oriented? Why not flag businessrules, diagnostics, etc.?
• All of the constraints are within-message only? Why be so limited?
• In short, why not provide for a general purpose constraint-expressing and-processing facility?
32
Finally and briefly, the semantics and inference point(3rd of 3)
• The protocol provides explicit semantic (meaning, interpretation)information primarily through its efforts at reference fixing.
• The basic message structure is:
H1,H2, . . . ,Hm[B1,1, B1,2, . . . , B1,B1; . . . ;Bm,1, Bm,2, . . . , Bm,Bm]
• Implicitly, this is our old friend F (C) from speech act theory.
Points arising: (i) FLBC should be able to represent these messages easily;(ii) Having in hand a logical representation (as in FLBC), articulating apowerful and general way of expressing constraints is a natural next step.
33