Websphere MQ Integrator

8/21/2019 Websphere MQ Integrator

1/483

ibm.com/redbooks

WebSphere MQ IntegratorDeployment and

Migration

Geert Van de Putte

Chris Griego

Brian McCarty

Peter Toogood

Ralf Ziegler

Migration possibilities for brokers and

NEON-based message flows

Multi-broker and multi-platform

configuration scenarios

New features, including

enhanced MRM and Java

nodes

Front cover


2/483


3/483

WebSphere MQ Integrator Deployment andMigration

April 2002

International Technical Support Organization

SG24-6509-00


4/483

Copyright International Business Machines Corporation 2002. All rights reserved.

Note to U.S Government Users Documentation related to restricted rights Use, duplication or disclosure is subject to

restrictions set forth in GSA ADP Schedule Contract with IBM Corp.

First Edition (April 2002)

This edition applies to Version 2, Release 1 of WebSphere MQ Integrator, for use with WindowsNT Version 4, Windows 2000, AIX and Sun Solaris, Program Numbers 5724-A82-00,5724-A82-01 and 5724-A82-02.

Comments may be addressed to:IBM Corporation, International Technical Support OrganizationDept. HZ8 Building 662

P.O. Box 12195Research Triangle Park, NC 27709-2195

When you send information to IBM, you grant IBM a non-exclusive right to use or distribute theinformation in any way it believes appropriate without incurring any obligation to you.

Take Note! Before using this information and the product it supports, be sure to read the

general information in Notices on page ix.


5/483

Copyright IBM Corp. 2002 iii

Contents

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Chapter 1. WebSphere MQ Integrator features overview . . . . . . . . . . . . . . 1

1.1 WebSphere MQ Integrator basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.1.1 Architecture of WebSphere MQ Integrator . . . . . . . . . . . . . . . . . . . . . 4

1.1.2 The role of the Configuration Manager . . . . . . . . . . . . . . . . . . . . . . . . 5

1.1.3 The functionality of the broker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.1.4 Publish/subscribe and the User Name Server . . . . . . . . . . . . . . . . . 10

1.1.5 The user interface: the Control Center . . . . . . . . . . . . . . . . . . . . . . . 11

1.2 Anatomy of a message flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.3 Enhanced Message Repository Manager . . . . . . . . . . . . . . . . . . . . . . . . . 18

1.4 XML support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

1.5 Database support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231.6 New ESQL features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

1.7 Summary of new features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Chapter 2. Migration considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.1 Considerations overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.2 Uninstalling the older version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.3 Message repository changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.4 Moving components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.5 Enhancement and schedule considerations . . . . . . . . . . . . . . . . . . . . . . . 312.6 DB2 database upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.7 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Chapter 3. Migration of an MQSeries Integrator environment . . . . . . . . . 37

3.1 Evolution of the MQSeries Integrator environment . . . . . . . . . . . . . . . . . . 38

3.2 The original environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.3 Adding a new broker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3.4 Adding a new application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

3.5 Adding a new broker using WebSphere MQ Integrator V2.1 . . . . . . . . . . 773.6 Introducing a V2.1 Configuration Manager . . . . . . . . . . . . . . . . . . . . . . . . 92

3.7 Using a V2.0.2 broker with a V2.1.0 Configuration Manager . . . . . . . . . 105

3.8 Product upgrade on AIX1, NT1 and NT2 . . . . . . . . . . . . . . . . . . . . . . . . 108


6/483

iv WebSphere MQ Integrator Deployment and Migration

3.8.1 Upgrade on AIX1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

3.8.2 Upgrade on NT1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

3.8.3 Upgrade on NT2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

3.9 Consolidating Configuration Managers . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Chapter 4. Deploying a broker on Sun Solaris using Oracle 8i . . . . . . . 131

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

4.2 Getting started. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

4.3 Software installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

4.4 Creating and configuring the broker . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

4.4.1 Connecting the brokers queue managers and the Configuration

Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

4.4.2 Configuring XA support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

4.4.3 Verification of the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

Chapter 5. Deploying a broker on Windows 2000 using SQL Server . . . 143

5.1 Why install on SQL Server? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

5.2 Getting started. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

5.3 Installation of SQL Server 7.0 SP2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

5.4 Installing a WebSphere MQ Integrator broker. . . . . . . . . . . . . . . . . . . . . 153

5.4.1 Broker only installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

5.4.2 Using the SQLServer broker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

5.4.3 Verification of the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

Chapter 6. New MRM features explored . . . . . . . . . . . . . . . . . . . . . . . . . . 163

6.1 Converting a delimited message to XML. . . . . . . . . . . . . . . . . . . . . . . . . 164

6.1.1 Creating a new message set. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

6.1.2 Adding the physical formats to the message set. . . . . . . . . . . . . . . 168

6.1.3 Creating message elements and types. . . . . . . . . . . . . . . . . . . . . . 170

6.1.4 Creating a message using the compound type. . . . . . . . . . . . . . . . 176

6.2 Creating a message flow to convert CSV to XML . . . . . . . . . . . . . . . . . . 179

6.2.1 Configuring the MQInput node . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1796.2.2 Configuring the Compute node . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

6.3 Deploying the message set and flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

6.4 Testing the message flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

6.5 Changing XML names in the output . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

6.6 Converting a tagged message to XML . . . . . . . . . . . . . . . . . . . . . . . . . . 191

6.6.1 Configuring a message set for tagged data . . . . . . . . . . . . . . . . . . 191

6.6.2 Testing the transformation of a tagged message . . . . . . . . . . . . . . 195

6.7 Using the debugger to examine the message tree . . . . . . . . . . . . . . . . . 198

6.7.1 Importing the Web materials message sets and flows . . . . . . . . . . 203

Chapter 7. Exploring the new XML features . . . . . . . . . . . . . . . . . . . . . . . 205

7.1 Importing an XML DTD and using it in a flow . . . . . . . . . . . . . . . . . . . . . 206


7/483

Contents v

7.1.1 Creating a DTD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206

7.1.2 Creating the message set and adding an XML format . . . . . . . . . . 208

7.1.3 Importing the DTD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

7.1.4 Examining the imported message set definitions . . . . . . . . . . . . . . 212

7.1.5 Correcting some of the XML names . . . . . . . . . . . . . . . . . . . . . . . . 2157.2 Using the message set in a flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

7.2.1 Configuring the Compute node . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

7.2.2 Testing the message flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

7.2.3 Modifying the message in the Compute node. . . . . . . . . . . . . . . . . 222

7.2.4 Problem solving with this flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

7.2.5 Some possible errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

7.3 Some more advanced features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

7.3.1 Processing multiple persons in a single message . . . . . . . . . . . . . 229

7.3.2 Using generic XML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2327.3.3 Using ESQL reference variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

7.3.4 Importing the Web materials message set and flows . . . . . . . . . . . 236

Chapter 8. Migration of solutions based on New Era Of Networks

functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

8.1 New Era Of Networks support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

8.1.1 MQSeries Integrator V1.0 and V1.1 . . . . . . . . . . . . . . . . . . . . . . . . 239

8.1.2 MQSeries Integrator V2.0 and V2.0.1 . . . . . . . . . . . . . . . . . . . . . . . 240

8.1.3 MQSeries Integrator V2.0.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2408.1.4 WebSphere MQ Integrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

8.2 Summary of changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

8.2.1 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

8.2.2 Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

8.2.3 Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

8.3 Migration of the New Era Of Networks environment . . . . . . . . . . . . . . . . 249

8.3.1 Migration steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

8.3.2 Example migration from MQSeries Integrator to WebSphere MQ

Integrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264

8.4 Working on New Era Of Networks messages . . . . . . . . . . . . . . . . . . . . . 289

8.4.1 Scenario 1: A simple reformat procedure . . . . . . . . . . . . . . . . . . . . 291

8.4.2 Scenario 2: Exploiting WebSphere MQ Integrator functionality for

messages in the NEONMSG domain . . . . . . . . . . . . . . . . . . . . . . . 299

8.4.3 Scenario 3: Transforming New Era Of Networks message to XML 307

8.4.4 Scenario 4: Routing a message using a NEONRules node . . . . . . 320

8.4.5 Scenario 5: Further exploitation of WebSphere MQ Integrator features

for NEONMSG messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3308.4.6 Scenario 6: Simulating the rules engine . . . . . . . . . . . . . . . . . . . . . 343

8.4.7 Other migration considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347


8/483

vi WebSphere MQ Integrator Deployment and Migration

Chapter 9. Developing and deploying custom nodes in Java. . . . . . . . . 351

9.1 What can you do with a Java plug-in?. . . . . . . . . . . . . . . . . . . . . . . . . . . 352

9.1.1 Plug-in nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352

9.1.2 MbInputNodeInterface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

9.1.3 MbNodeInterface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

9.2 Getting started. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354

9.2.1 System requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354

9.2.2 Programming requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354

9.2.3 Debugging and logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354

9.3 Implementing an input node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355

9.3.1 The Java code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355

9.3.2 Packaging and loading the input node . . . . . . . . . . . . . . . . . . . . . . 359

9.3.3 Simple message flow to test the input node . . . . . . . . . . . . . . . . . . 360

9.3.4 How to test the input node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3679.3.5 How to use the input node to read a telnet stream . . . . . . . . . . . . . 369

9.4 Implementing a plug-in node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371

9.4.1 Details of the example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371

9.4.2 The Java code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372

9.4.3 Packaging and loading the input node . . . . . . . . . . . . . . . . . . . . . . 374

9.4.4 A simple message flow to test the plug-in node . . . . . . . . . . . . . . . 374

9.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378

Chapter 10. Using the aggregation nodes . . . . . . . . . . . . . . . . . . . . . . . . 37910.1 Developing a message flow using aggregation . . . . . . . . . . . . . . . . . . . 380

10.2 Components of the example system. . . . . . . . . . . . . . . . . . . . . . . . . . . 380

10.3 The role of the aggregation nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

10.3.1 The AggregateControl node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

10.3.2 AggregateRequest node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382

10.3.3 AggregateReply node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383

10.4 The FAN_OUT flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384

10.5 The processing flows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395

10.5.1 PROC_A flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395

10.5.2 PROC B flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397

10.6 The FAN_IN flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401

10.7 A trace showing the compound message tree . . . . . . . . . . . . . . . . . . . 404

10.8 Best practices for aggregation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409

Appendix A. Hardware/software configuration . . . . . . . . . . . . . . . . . . . . 411

A.1 Hardware/software specification for brokers running on Windows 2000 412

A.2 Hardware/software specification for the AIX broker machine . . . . . . . . . 413A.3 Hardware/software specification for the Sun Solaris broker machine. . . 413

Appendix B. Source code for the Java extensions to WebSphere MQ

Integrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415


9/483

Contents vii

B.1 Java input node getting started example . . . . . . . . . . . . . . . . . . . . . . . . 416

B.1.1 SocketNode.java. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416

B.1.2 SocketNodeException.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420

B.1.3 SocketNodeMessages.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420

B.1.4 DirFilter.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421

B.1.5 Compilation and packaging instructions . . . . . . . . . . . . . . . . . . . . . 421

B.2 Java plug-in node getting started example . . . . . . . . . . . . . . . . . . . . . . . 422

B.2.1 SwitchNode.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422

B.2.2 TransformNode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424

B.2.3 Compilation and packaging instructions . . . . . . . . . . . . . . . . . . . . . 426

Appendix C. Oracle8i installation and configuration on Solaris. . . . . . . 429

Installation planning and design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430

Which versions are supported by the product? . . . . . . . . . . . . . . . . . . . . . . . 430Getting started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430

Preparing the Solaris machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431

Install Oracle8i server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431

Appendix D. Additional material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445

Locating the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445

Contents of the Web material. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445

System requirements for downloading the Web material . . . . . . . . . . . . . 447

How to use the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449

IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449

Other resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449

Referenced Web sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449

How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450

IBM Redbooks collections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451

Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457


10/483

viii WebSphere MQ Integrator Deployment and Migration


11/483

Copyright IBM Corp. 2002 ix

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consultyour local IBM representative for information on the products and services currently available in your area.Any reference to an IBM product, program, or service is not intended to state or imply that only that IBMproduct, program, or service may be used. Any functionally equivalent product, program, or service thatdoes not infringe any IBM intellectual property right may be used instead. However, it is the user'sresponsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document.The furnishing of this document does not give you any license to these patents. You can send license

inquiries, in writing, to:IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where suchprovisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATIONPROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS ORIMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimerof express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically madeto the information herein; these changes will be incorporated in new editions of the publication. IBM may

make improvements and/or changes in the product(s) and/or the program(s) described in this publication atany time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in anymanner serve as an endorsement of those Web sites. The materials at those Web sites are not part of thematerials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distr ibute any of the information you supply in any way it believes appropriate withoutincurring any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their publishedannouncements or other publicly available sources. IBM has not tested those products and cannot confirm

the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions onthe capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate themas completely as possible, the examples include the names of individuals, companies, brands, and products.All of these names are fictitious and any similarity to the names and addresses used by an actual businessenterprise is entirely coincidental.

COPYRIGHT LICENSE:This information contains sample application programs in source language, which illustrates programmingtechniques on various operating platforms. You may copy, modify, and distribute these sample programs inany form without payment to IBM, for the purposes of developing, using, marketing or distributing application

programs conforming to the application programming interface for the operating platform for which thesample programs are written. These examples have not been thoroughly tested under all conditions. IBM,therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy,modify, and distribute these sample programs in any form without payment to IBM for the purposes ofdeveloping, using, marketing, or distributing application programs conforming to IBM's applicationprogramming interfaces.


12/483

x WebSphere MQ Integrator Deployment and Migration

Trademarks

The following terms are trademarks of the International Business Machines Corporation in theUnited States, other countries, or both:

Redbooks(logo)e (logo)IBMAIXAS/400CICSDatabase 2

DB2DB2 UniversalDatabaseEveryplaceIMSiSeriesMQSeries

RAARedbooksRETAINRS/6000OS/390ServicePacSP

SP2SupportPacTivoliVisualAgeWebSpherexSeriesz/OS

The following terms are trademarks of other companies:

ActionMedia, LANDesk, MMX, Pentium and ProShare are trademarks of Intel Corporation in theUnited States, other countries, or both.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporationin the United States, other countries, or both.

Java and all Java-based trademarks and logos are trademarks or registered trademarks of SunMicrosystems, Inc. in the United States, other countries, or both.

C-bus is a trademark of Corollary, Inc. in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

SET, SET Secure Electronic Transaction, and the SET Logo are trademarks owned by SETSecure Electronic Transaction LLC.

Other company, product, and service names may be trademarks or service marks of others.


13/483

Copyright IBM Corp. 2002 xi

Preface

The release of WebSphere MQ Integrator Version 2.1 has introduced quite a fewchanges; these changes make it more appealing for message flow developersand message set designers to migrate to the new version so that they can exploitthis functionality.

After a brief overview of the product, we look at several migration issues thatmight arise; we also describe the migration process for MQSeries IntegratorVersion 2.0.1 and Version 2.0.2 on Windows NT and AIX platforms. The migratedenvironment is further extended with a broker running on Solaris using Oracle,

and a broker running on Windows using SQL Server. Besides the migration ofbrokers and the Configuration Manager, this redbook also investigates themigration of message flows that use NEON functionality. We discuss severalscenarios explaining how to take advantage of the tighter integration betweenNEON-based functionality and base WebSphere MQ Integrator functionality.

Finally, we introduce the major new features of this release. The extendedfunctionality of the Message Repository Manager and the extended support forXML messages is explored using several examples. The support for Java-based

plug-in nodes and input nodes is demonstrated by developing a simple inputnode which reads files from a directory to initiate a message flow. The newaggregate nodes also demonstrate how to combine several messages of severalformats into a single XML message.

The team that wrote this redbook

This redbook was produced by a team of specialists from around the worldworking at the International Technical Support Organization, Raleigh Center.

Geert Van de Putteis an IT Specialist at the International Technical SupportOrganization, Raleigh Center. He is a subject matter expert in messaging andbusiness integration and has six years of experience in the design andimplementation of MQSeries-based solutions. Before joining the ITSO, Geertworked in IBM Global Services, Belgium.

Chris Griegois an IT Architect for IBM Global Services working in the Enterprise

Application Integration West US practice. He has 25 years of IT experience,working within a wide range of fields including mainframe applicationprogramming, mainframe systems programming and support, and various


14/483

xii WebSphere MQ Integrator Deployment and Migration

Windows platforms and UNIX operating systems. He is an IBM certifiedSolutions Expert for MQSeries Integrator and holds all three MQSeriescertifications. His areas of expertise include MQSeries, MQSeries Integrator,CICS, and DB2.

Brian McCartyis an EAI specialist from the United States. He has worked withIBM WebSphere and MQSeries technology for four years. His areas of expertiseinclude EAI and legacy to e-Business integration. Previously, he was anapplication developer within the finance industry. In his free time, he contributesto several WebSphere and Java users groups in Texas.

Peter Toogoodis a Technical Specialist with Royal & Sun Alliance Insurance inSussex, England. He has a wide range of IT experience, from mainframe systemprogramming to distributed middleware architecture and design. He is an IBM

certified Solutions Expert for MQSeries and MQSI V2. In his spare time, he editsthe Xephon monthly technical journal MQ Update.

Ralf Ziegleris an IT Specialist in the Integration Technology Services of IBM inMainz, Germany. He is an IBM MQSeries Integrator certified specialist and hascomprehensive support and service experience with MQSeries and all versionsof MQSeries Integrator on distributed platforms.

Comments welcomeYour comments are important to us!

We want our Redbooks to be as helpful as possible. Send us your commentsabout this or other Redbooks in one of the following ways:

Use the online Contact usreview redbook form found at:

ibm.com/redbooks

Send your comments in an Internet note to:

[email protected]

Mail your comments to the address on page ii.
http://www.redbooks.ibm.com/http://www.ibm.com/redbooks/http://www.redbooks.ibm.com/contacts.htmlhttp://www.redbooks.ibm.com/contacts.htmlhttp://www.ibm.com/redbooks/http://www.ibm.com/redbooks/http://www.redbooks.ibm.com/


15/483

Copyright IBM Corp. 2002 1

Chapter 1. WebSphere MQ Integrator

features overview

While this redbook is intended for more experienced users of MQSeriesIntegrator, we will start this first chapter with a quick summary of the basicstructure of the WebSphere MQ Integrator product and the roles of eachcomponent. We will then introduce and briefly discuss the major new featurestherein.

1


16/483

2 WebSphere MQ Integrator Deployment and Migration

1.1 WebSphere MQ Integrator basics

WebSphere MQ Integrator, formerly called MQSeries Integrator, is IBMsmessage broker product. It extends the messaging capabilities of MQSeries by

adding message routing and transformation features. It is used for EnterpriseApplication Integration and is part of the WebSphere MQ family of products.

Business rules can be defined by graphically developing message flowswhichconsist of a sequence of nodes. Message flows are processes that canencapsulate simple to extremely complex logic. They can be designed to performa wide variety of functions including (but not limited to):

Routing of messages to zero or more destinations based on the contents ofthe message or message header. Both one-to-many and many-to-one

messaging topologies are supported. Transformation of messages into different formats so that diverse applications

can exchange and understand messages.

Enrichment of the message content en-route, for example by using adatabase lookup performed by the message broker.

Storing information extracted from messages en-route into a database, usingthe message brokers built-in Database nodes.

Publishing messages, using topic/content based criteria for subscribers toselect which messages to receive.

Interfacing with other transport mechanisms, such as MQSeries Everyplace.

Extending the base function of WebSphere MQ Integrator withplug-innodes,which can be developed by customers as well as by IBM and ISVs.

Message content can be handled in a number of ways, including:

XML: for self-defining, or generic, XML messages.

MRM, orMessage Setsdefined in the Message Repository Manager, as:

XML defined to the MRM.

Tagged data.

Custom Wire Format, similar to a COBOL copybook layout.

Delimited data.

PDF, not Adobes PDF, but a specialIzed financial data format.

NEON: for messages defined using the New Era Of Networks formatter. BLOB: for messages with no definition.


17/483

Chapter 1. WebSphere MQ Integrator features overview 3

WebSphere MQ Integrator backgroundWebSphere MQ Integrator builds on the assured messaging capability ofMQSeries, and a typical message flow uses MQSeries messages for input andoutput. There are MQSeries adapters and connectors available for varioussystems and applications, and version 2.1 of WebSphere MQ Integrator alsoallows other interfaces to be linked directly to broker message flows.

WebSphere MQ (formerly known as MQSeries) provides support for applicationswith a number of application programming interfaces (MQI, AMI, JMS) in severalprogramming languages and for a number of communication models (includingpoint-to-point and publish/subscribe). WebSphere MQ also provides a number ofconnectors and gateways to other products such as SAP R/3, CICS, and IMS.

WebSphere MQ Integrator extends the messaging capabilities of MQSeries by

adding message broker functionality driven by business rules. It provides theintelligence to route and transform messages, the possibility to filter messages(topic-based or content-based), and database capabilities for the enrichment ofthe messages or for warehousing these messages. It also provides a frameworkfor extending the functionality with plug-ins for user-written or third-partysolutions.

MQSeries Workflow is the third component of IBMs family of products forbusiness integration and adds a further dimension to the integration of

applications. It is about integrating all the resources in a business process andensuring that the right information gets to the right target, be that a person orapplication, at the right moment in the process flow.

WebSphere MQ Integrator applicationsThese can be existing MQSeries-enabled applications and need not be aware ofWebSphere MQ Integrator; in this case, the message flows set the correctconfiguration values to handle the messages. Alternatively, the application canprefix the user data of the message with a WebSphere MQ Integrator header(MQRFH or MQRFH2) to control the way the broker handles the message.

WebSphere MQ Integrator supports two models for application communication:

Point to Point, which can include one-to-many and many-to-one (where thenumber of message producers and the number of message receivers aredifferent).

Publish/Subscribe, which includes and extends the existing MQSeriespublish/subscribe model.

In addition to MQSeries inputs and outputs, WebSphere MQ Integrator nowsupports MQSeries Everyplace and SCADA protocols. By developing plug-innodes, additional communication protocols can be supported.


18/483


Figure 1-1 WebSphere MQ Family

1.1.1 Architecture of WebSphere MQ Integrator

The components of a WebSphere MQ Integrator environment include:

Control Center(s), a Windows NT GUI for the message broker administrationand is also used for developing the message flows/sets.

Configuration Manager, with its configuration and message repository.

One or more brokers, which can be on different platforms or systems.

User Name Server, used to control publish/subscribe topic security.

MQSeries queue manager(s); at least one is needed per broker. Database software to support the Configuration Manager and broker(s).

Client applications (source and target), which produce and consumemessages (or use the other supported types of protocols).

Optionally, system management software such as Tivoli to monitor the brokerand queue managers operational status.

MQSeries

MQSeriesIntegrator

MQSeriesWorkflow

Modular Set of Offerings

MQSeries Foundation

Common Look and Feel

Management/Monitoring

Messaging Tools

Family TraitsWorkflow, Process FlowApplication ServicesTools

Xform, Rules, RoutingAPI FrameworkTemplates, Utilities

Messaging ServicesStandard FormatsTools


19/483


Figure 1-2 WebSphere MQ Integrator Architecture

1.1.2 The role of the Configuration ManagerA WebSphere MQ Integrator system is controlled by the Configuration Manager,which manages the components and resources within the broker domain. Thedomain configuration information is maintained in the configuration repositoryand administered using the Control Center.

The Configuration Manager has the following main functions:

Maintaining the configuration repository.

This set of database tables records the configuration of the components in thebroker domain, including the message flows.

Managing the initialization and deployment of message brokers and messageflows.

ClientAppl

ClientAppl

Source

ClientAppl

Target

ClientAppl

ClientAppl

ClientAppl

MQ Messages

MQ Messages

MessageDictionary

RDBMS

Filter Node

InputNode

OutputNode

FilterNode

Warehousing Node

Message Flow

Output Node

Broker

Message Flow Engine(s)

Controller

Compute Node

InputNode

OutputNode

FilterNode

Warehousing Node

Output Node

AdministrativeAgent

Admin Appl

ControlCenter

GUI

Configuration

ManagerMessage Flow

Message Formats

ManagementMessage

Repository

UserNameServer


20/483


Maintaining message set information in the message repository database.These are the definitions of message contents and layout.

Checking the authority of defined user IDs to initiate control actions.

.

Figure 1-3 Configuration Manager relationships

The Configuration Manager provides services to the other broker domaincomponents, performing the configuration updates in response to the ControlCenter actions.

The Configuration Manager uses MQSeries messages to communicate with theother components in the broker domain. Windows NT (or Windows 2000) withDB2 must be used to host the Configuration Manager and its databases. Onlyone Configuration Manager is used for a given broker domain.

A set of tables in a database, known as theconfiguration repository, is defined.These tables must be managed using DB2 Universal Database for Windows NT.The Configuration Manager uses a Java Database Connectivity (JDBC)connection to this database.

A set of tables in a database, known as the message repository,is also defined.These tables must also be managed using DB2 Universal Database for WindowsNT. The Configuration Manager uses an Open Database Connectivity (ODBC)connection to this database.

Domain

Configuration Manager

Queue Manager

Shared/Deployed

Configuration

Repository

JDBC

Message

Repository

ODBC

UserNameServer

Queue Manager

Applications

Local

Configuration

Repository

MQSeries

Java

Client ODBC

Broker

Queue Manager

Broker

Database

Database

MQSeries

Control Center


21/483


A set of fixed named queues is defined on the MQSeries queue manager thathosts the Configuration Manager. This MQSeries queue manager must exist onthe same physical system as the Configuration Manager, and is identified at thetime the Configuration Manager is created.

These queues are created when the mqsicreateconfigmgrcommand(including associated variables) is executed through either the graphicalcommand assistant or through the command prompt. No actions are requiredby the administrator to add the required definitions.

A server connection channel is defined to the MQSeries queue manager thathosts the Configuration Manager. This connection is used by all instances of theControl Center that communicate with the Configuration Manager.

Sender and receiver channels need to be defined to each broker in the brokerdomain, except for a broker that shares its queue manager with the ConfigurationManager.

1.1.3 The functionality of the broker

A broker is the named resource that executes the business logic defined in themessage flows. Applications send and receive messages to and from a broker(which is the deployed run-time, or execution, part of the product) using

MQSeries queues or the other suppor ted methods of communication.

More than one broker can be defined per WebSphere MQ Integrator domain,either on the same or on a different physical system. If you plan to use thepublish/subscribe service, you can connect a number of brokers together into acollective, which allows optimization of the subscriber connections.

Message flows run within execution groupson the broker. An execution group isan operating system process, and flows within it are threads. Individual flows and

execution groups can be stopped and started using the Control Center. Bydefault, only one execution group is defined, but more can be easily added.

It is good practice to run message flows belonging to different applications indifferent execution groups, to provide some isolation between them in the eventof a problem. For even more isolation between applications, you can usedifferent brokers, but remember that each broker will need its own queuemanager.


22/483


Figure 1-4 Broker overview

Brokers can run on a number of different platforms: Windows NT (and Windows 2000)

AIX

HP-UX

Solaris

z/OS (using UNIX System Services)

iSeries (using the integrated xSeries server)

Linux (currently available as a technology preview)

Each broker has a related local MQSeries queue manager, which cannot beshared with another broker, and a set of database tables to hold the brokerdefinitions; this set of tables is accessed through an ODBC connection.

These tables can be created using a number of database products:

IBM DB2

Microsoft SQL server (Windows only)

Oracle

Sybase

Broker

Queue ManagerBroker database

ODBC

connectionExecution group

Message dictionary

Message flowMessage flow


23/483


WebSphere MQ Integrator for z/OS brokers only support DB2.

Each broker requires:

A set of tables in a database to hold the brokers local data. This is accessedusing the ODBC connection.

A set of named queues on the queue manager associated with the brokerdomain. These are created automatically when the mqsicreatebrokercommandis issued.

Its own queue manager. It can share the queue manager hosting either theConfiguration Manager or the optional User Name Server, or both.

Since the broker uses a set of predetermined queue names, this creates thedependency of an MQSeries Queue Manager per broker.

For operation, a broker requires both configuration and initialization data.Configuration and initialization data are logically separate and stored indifferent physical repositories.

Each broker instance needs an assigned, permanent and fixed name. This is theBroker Instance name. This name, which is similar to the static identifierassigned to a database before it is created, is used to distinguish tablespertaining to one broker from tables where multiple brokers have been set up touse the same database.

Creating a broker on the target execution platform does not itself update theconfiguration repository; you need to create a reference to it using the ControlCenter. Once this is done, and the broker is deployed (to initialize it), thenmessage flows and sets can be assigned to the broker. Deploying the flowsstarts their execution.

Several different message flows can be assigned to an execution group,although using different execution groups provides better application isolation. A

particular message flow can also be configured to run more than one instance ofitself inside the execution group.

The execution group environment is also known as a data flow(or message flow)engine. The engine is responsible for loading any plug-in nodeswhich can bedeveloped, or provided, to extend the function beyond the supplied IBM nodes.These are known as loadable implementation libraries(or .lil files).

Brokers provide information, in the form of published event messages, inresponse to changes; these can be used by system management tools to updatetheir management agents as to the status of the broker components.


24/483


1.1.4 Publish/subscribe and the User Name Server

Message flows that include a publication node provide a publish/subscribeservice. Publishing applications generate input to such flows, and subscribingapplications register a subscription with the broker based on the topic and/or

content of interest to them. Within a broker collective, subscribers can registerwith their local broker and messages are automatically routed to them.

With publish/subscribe, the broker provides indirection between publishers andsubscribers. Subscribers can use a mixture of topic and content criteria to selectwhich messages to receive.

Topic-based security can be implemented to provide administrative control overwho can publish and who can subscribe. The WebSphere MQ Integratorcomponent User Name Server, which is optional and needed only forpublish/subscribe security, is used to interface with the brokers platform securitysystem. It provides information about users and groups to the ConfigurationManager and brokers.

When the User Name Server is created, a set of fixed named queues are definedto its hosting queue manager, which may be a queue manager shared with theConfiguration Manager or a broker if they are installed on the same system.Figure 1-5shows some applications of the role of publisher or subscriber and ofthe role of the broker as intermediate message router.


25/483


.

Figure 1-5 Subscribing to topic/content

1.1.5 The user interface: the Control Center

The Control Center is a GUI running on Windows NT (or 2000) which allows youto configure and control your brokers. The Control Center connects to theConfiguration Manager over TCP/IP, using the MQSeries Client for Java. It canbe installed on the same physical system as the Configuration Manager, but it istypically used on another PC connected over a LAN.

Any number of Control Center instances can be connected to the Configuration

Manager; the Conntrol Center is also used by the message flow developers aswell as by broker administrators. Several user roles are predefined:

Message flow and message set developer:

Can create message flows and message sets.

Can access the Message Sets view and the Message Flows view.

Message flow and message set assigner:

Can create execution groups within brokers, then assign and deploy

message flows and message sets to brokers.

Can access the Assignments view only.

Broker

Publish: STOCK/IBM

Subscribe: */FORD

Publish: STOCK/FORD

Broker

Subscribe: STOCK/IBMSubscribe: STOCK/*

STOC

K/IBM

data

Select

ed

STOCK/

IBM

data

STOC

K/FO

RDdata

Sele

cted

STOCK

/FORD

data

WHERE value>10000WHERE number>1000


26/483


Operational domain controller:

Can create brokers and collectives, and define the topology.

Can deploy all types of data and monitor the operational domain.

Can access the Topology view, the Assignments view, the Topics view, the

Operations view and the Subscriptions view.

Topic security administrator:

Can create topics and their access control lists, and deploy them.

Can access the Topics view and the Topology view.

All Roles

Can perform all Control Center tasks.

Can access all views.

There are four Windows NT security groups (mqbrdevt, mqbrasgn, mqbrops,mqbrtpic) which are used to grant privileges. Standard Windows useradministration is used to assign membership of one of more of these groups tousers; the role is then selected in the Control Center Preferences dialog.

Figure 1-6 Control Center view


27/483


When you decide to make changes to a resource managed by the ControlCenter, you must check out the resource. This gives you exclusive control over alocked copy until you check in or unlock the resource. You can save a local copyof the changed resource without checking it back in, if you wish to do so.

Control Center resources can exist in three states: Local: a copy of the configuration data being worked on, usually obtained by

checking out the resource from the shared copy. Changes made to the localcopy are not visible to other users until the copy is checked in.

Shared: the version of the resource stored by the Configuration Manager andvisible to all users. Note that only one level exists and no previous version issaved. SupportPac IC04 offers procedures for version management.

Deployed: the active copy of the configuration data that is in the broker and

also stored in the deployment database of the Configuration Manager.

The Control Center allows the export and import of message flows to and from aflat file in XML format. Message sets can be imported and exported using themrmimpexpmsgset command.

1.2 Anatomy of a message flow

Message flows consist of a number of nodes (orprimitives) which are wiredtogether to form a broker application. Each node has terminalsfor input oroutput, or both (most nodes have several output terminals). The Control Centermessage flow view is used to construct the flows using nodes like:

MQInput node - encapsulates anMQGET operation.

MQOutput node - encapsulates an MQPUT operation.

Extract node - extracts specified fields from a message.

Filter node - routes a message according to an ESQL expression.

Compute node - modifies message or database contents using ESQL.

Database node - performs a database operation.

Reset Content Descriptor - resets the message properties.

There are many other IBM primitive nodes available; these are described in themanual Using the Control Center,SC34-5602-04. Each node will have propertieswhich can be set to certain values for the parameters of the node. For example,

the queue manager name is a optional property of the MQOutput node.


28/483


Some nodes have their function controlled byESQLstatements. ESQL is a freeformat, high-level programming language derived from SQL version 3. By codingESQL, it is possible to develop complex logic for manipulating either message ordatabase contents (or both), without using excessive nodes in a flow.

ESQL, like most programming languages, can be used to develop businesslogic, but is generally coded to select and analyze message and/or databasecontents, and to perform the more complicated conversions of data values.Details about ESQL can be found in the manual ESQL Reference,SC34-5923-01.

Although ESQL does not provide subroutines to reuse common code, it ispossible to include smaller message flows assub-flows; such flows can be usedfor generic parts of message flows such as error handling routines. In addition,

the use of sub-flows improves readability of the logic, regardless of reuse.

Extending the available nodesWebSphere MQ Integrator allows the development and implementation ofcustom nodes and custom parsers. These can be written in C or Java and mayalso be provided by an ISV. A number of useful plug-ins are available asSupportPacs and freely downloadable from the IBM WebSphere MQ IntegratorWeb site. It is important to check that any such custom node is compatible withall the target broker platforms onto which you may wish to deploy the message

flows.

Figure 1-7shows a message flow that uses some of the available messageprocessing nodes.

Figure 1-7 Example of a message flow


29/483


Each node has properties that must be set; for example, with the MQInput node,you would specify the input queue name (the queue managers is always thesame as the brokers), conversion and other MQGET options, the transactionmode, the message domain, etc. Generally, it is necessary to specify more ofthese values when handling messages that do not contain an MQRFH2 header.

Figure 1-8shows the default page of the properties on an MQInput node; thismessage flow is able to handle messages that do not have an MQRFH2 header.

Figure 1-8 Properties of the MQInput node

In most cases, the parsed message is passed unchanged from node to node,except in the case of the Compute node, which can modify it; the outputmessage of the Compute node then becomes the input to the next node in theflow. Some nodes provide a drag-and-drop graphical interface to assist withelement selection; for example, the DataUpdate node does so for its outputdatabase table, as shown in Figure 1-9.


30/483


Figure 1-9 Example of DataUpdate node field selection

ESQL statements are added using the ESQL tab window on the Properties pageof WebSphere MQ Integrator nodes such as the Compute and Filter node. Inaddition to simple assignments, ESQL includes branching and loopingcapabilities.

Example 1-1 ESQL statements

SET OutputRoot = InputRoot;DECLARE TotalItemQuantity INTEGER;

SET TotalItemQuantity = (SELECT SUM(CAST(T.itemquantity AS INT))FROM InputBody.Message.receiptmsg.transactionlog.purchaseselement[] AS TWHERE CAST(T.itemname AS CHAR) = 'Shampoo');SETOutputRoot.XML.Message.receiptmsg.transactionlog.totalselement.totalitemquantity = TotalItemQuantity;

The visual message flow builder in the Control Center can be used to assemblesequences of nodes (or primitives) that provide logic similar to that of a

programming language. Loops and branches can be performed using nodes.


31/483


In Figure 1-10, the Filter node tests a counter stored in the message tree; if thecomparison proves false, the flow passes to a DataInsert node. This is followedby a Compute node, which increments the counter. When the filter comparisontest eventually proves true, control passes to the MQOutput node.

Figure 1-10 Example of loop control in a flow

In traditional programming languages, subroutines are used to group commonlyused statements. With WebSphere MQ Integrator message flows, this isachieved using sub-flows. These flows are comprised of the same nodes asmain-line flows, except that they have Input and Output Terminal nodes toconnect to an upper-level message flow.

Main message flows (or higher level sub-flows) simply include the desiredsub-flow as if it were another node, and wire up its terminals accordingly. Thistechnique is valuable for very complex message flows, as it structures the flowinto a series of smaller, more readable sub-flows.

Compute nodes are similar to groups of executable statements in otherlanguages and can be inserted into flows when there is a need to performcalculations or assign values to fields, especially when converting data formats.ESQL has branching and looping features that work inside the Compute node.

Compute nodes can modify the message tree contents and update databases.They are, therefore, the most powerful nodes available. ESQL is also usedextensively in Filter and Database nodes. WebSphere MQ Integrator 2.1 offersan ESQL code palette to allow drag-and-drop development.

E h d M R i M


32/483


1.3 Enhanced Message Repository Manager

WebSphere MQ Integrator supports self-defining XML and JMS messages.These types of messages do not have to be defined to the Message RepositoryManager, although you can do this to exploit the MRM features and the

integration of MRM with other components of WebSphere MQ Integrator.Defining your message formats to the MRM provides greater control over theformats and enables increased functionality within the Control Center tomanipulate messages in the flows.

Predefined message formats are grouped in message sets. These are created inthe Control Center, or imported from previously exported copies. Message setsallow message flows to understand the structure of incoming message data andtherefore need to be deployed to the run-time (or broker) environment.

You can change the format of any message in the MRM domain to any otherformat defined to the MRM; most of the changes are made automatically by thebroker. Drag-and-drop element selection is available in the node propertiesbecause the message structure is known at build time.

The MRM components are:

At build time:

GUI: the Control Center Message Sets tab

Repository: managed by the Configuration Manager

Importers and extractors: used to move message sets from oneConfiguration Manager to another

Deployment: message sets need to be assigned to a broker and deployed

At run-time:

Message parsers

Run-time dictionaries

Physical format descriptors

The MRM message model consists of meta-data representing logical messagedefinitions that are platform- and language-independent. It has layers ofadditional data used for mapping to physical formats. Definitions are made up ofreusable components or objects and organized into message sets. Figure 1-11shows the definition model for a message.

L i l t t i t f d t d f i l


33/483


Logical structures consist of one or more compound types made up of simpletypes, elements or other compound types. Often, it is easier to import thedefinitions than it is to create them in the Control Center; for example, a COBOLor C structure can be imported to create a new compound type. XML DTDs cannow also be imported to create message set definitions.

Figure 1-11 Basic message components

A message set is a group of messages that are related in some way. A messagedefines the format of a single unit of information to be exchanged between

applications. A message can contain, or embed, other messages (so-calledmulti-part messages). An element defines the format of a single unit ofinformation within a message. A type defines the format of an element and canbe a simple or compound type. A category groups messages and transactionswithin a message set. A transaction (in this sense) is a logical grouping ofmessages within a message set.

Figure 1-12shows what simple types are available in the MRM. These simpletypes are the building blocks for constructing compound types.

Message

CompoundType

Element

ElementValue

Simple type

Type

A message has a structure based on..

Made up of data units called

Format defined by ..Element may optionallyhave Element Valueconstraints, e.g., forSTRING elements;minimum and/or maximum

length

User defined

Pre-defined

Or made upof groups ..


34/483


Figure 1-12 Simple types in the MRM

The MRM type composition options are:

Empty

Choice

Unordered set

Ordered set (the default) Sequence

Simple unordered set

Message

Value constraints can be defined for elements for such things as minimumlength, scale, default value, etc. This is primarily for documentation purposes, ascurrently the broker does not perform run-time validation.

Binary The data content does not conform to any numeric orcharacter representation (for example, BLOB)

Do not confuse with COBOL BINARY datatype.

Boolean Used when the element can have only twovalues: true or false.

FloatCan be used for any real number, for fractions andnumbers which are outside of the Integer or Decimaldatatype.

IntegerUsed for whole numbers in the range -2147483648 to+2147483647

String Used for character data

DatetimeUsed for dates and/or

times

DecimalCan be used for any number up to 31 decimal digits,for fractions and numbers which are outside of theInteger datatype.

A message in the MRM domain can have one of the following wire formats:


35/483


A message in the MRM domain can have one of the following wire formats:

CWF (Custom Wire Format or Legacy messages)

TDWF (Tagged / Delimited Wire Format, such as SWIFT)

XML (defined, as opposed to self-defining or generic)

PDF (not Adobe, but a specialized financial format)

CWF messages are often defined by importing a C or COBOL structure. XMLmessages can also be defined by importing a DTD. Some tagged formats arepredefined to the MRM and others can be configured in the Control Center.

Message sets can be imported and exported to XML format flat files using thesupplied mqsiimpexpmsgsetcommand. This can be used both for version controland to copy message sets between Configuration Managers.

Tagged data formatsThe following messaging standards can be selected in the Control Center:

Unknown (this is the default)

ACORD AL3

SWIFT

EDIFACT

X12

Tagged/Delimited messages:

Consist of text strings

Are optionally preceded by fixed text

Are followed by child elements, which:

May be of fixed length or separated by a delimiter.

May be preceded by a tag that uniquely identifies it (the tag may be offixed length or separated from the data by a delimiter)

The data for child elements may represent simple values or sub-structures.

Are followed optionally by fixed text

Example 1-2shows a tagged/delimited message as used in the SWIFT network.

Example 1-2 A SWIFT message


36/483


Example 1 2 A SWIFT message

{1:F01BANKBEBBAXXX2222123456}{2:I100IBMADEF0AXXXN}{4::20:X:32A:940930USD1000000,:50:LINE1

:59:LINE1-}

The example SWIFT message includes two headers and a message body. Eachis wrapped within curly braces ({...})

{ is the Group Indicator.

:is the Tag Data Separator.

}{is the Delimiter. }is the Group Terminator.

1, 2, 4are Tags.

In Chapter 6, New MRM features exploredon page 163, some more examplesof tagged data formats are discussed.

1.4 XML supportXML messages are supported by WebSphere MQ Integrator as either generic (orundefined) XML or as MRM defined XML. This determines whether the genericparser or the MRM parser is used to interpret them. Defining XML messages inthe MRM enables more comprehensive facilities to process them, such as thebroker performing message transformation to Custom Wire Format.

When defining XML messages to the MRM, an XML format layer can be

specified. This allows the output XML to be customized; for example, the dataelements can be given attributes.

Output XML can be created:

With tags

With or without attributes

With or without attribute identifiers

With DOCTYPE, Root Tag Name, XML Name, Render, Format, Encoding

You can now import a DTD into the Control Center. This can be a DTD generatedfrom a WebSphere MQ Integrator message repository, or a DTD from anothersource. You can then use the MRM to model these XML messages.

Several examples of this extended support for XML messages are given in


37/483


p pp g gChapter 7, Exploring the new XML featureson page 205.

1.5 Database support

WebSphere MQ Integrator uses DB2 databases for the Configuration Managerwhich must run on Windows NT (or 2000). The brokers can use DB2, SQLServer, Oracle or Sybase for either the broker database and/or any userdatabase (SQL server only runs on Windows NT and z/OS only has DB2 as asupported database manager).

User databases are where the message flows can optionally access or storedatabase records. Several nodes are provided to assist with developing

message flows that use database information. It is not mandatory to use adatabase in a flow, although the broker uses its own database to retrieveconfiguration data and store persistent run-time data.

When a message flow updates one or more user databases, an importantconsideration is the transactionality of the flow. Many business transactionsrequire that all updates be committed, or rolled back, in a single unit of work. Thisnormally includes updates to message queues as well as databases.

MQSeries is able to coordinate transactions between queue managers anddatabases by acting as an XA transaction manager. Supported releases of DB2,Oracle and Sybase can participate as Resource Managers in a distributed XAtransaction coordinated by MQSeries. On z/OS, all transactions are coordinatedby RRS and all the flows are always globally coordinated by default.

Message flows can be made transactional for message queue and databaseinteraction by selecting the appropriate option in the node properties. XA (or

global) coordination needs some configuration to operate correctly: switch files

must be defined to the resource manager (in the qm.ini file on UNIX).In Chapter 4, Deploying a broker on Sun Solaris using Oracle 8ion page 131,we discuss the use of Oracle 8i as the database for the broker and how toconfigure all products for XA resource coordination.

Supported releases are DB2 version 6.1 and 7.1, Oracle 8.1.6 and 8.1.7, andSybase 12. Fixpaks or service may need to be applied.

1.6 New ESQL features


38/483


1.6 New ESQL features

SQL is the industry standard language for accessing and updating databasedata. ESQL is a language derived from SQL V3 and is particularly suited tomanipulating both database and message data. An ESQL program consists of a

number of statements that are executed in the order they are written.

ESQL programs form an essential part of WebSphere MQ Integrator nodes, suchas the Compute, Filter and Database nodes. In common with other high-levelprogramming or scripting languages, ESQL has statements, operators,expressions, functions, data types, variables, reserved words and so forth.

Example 1-3 ESQL from a Compute node

DECLARE C INTEGER;SET C = CARDINALITY(InputRoot.*[]);DECLARE I INTEGER;SET I = 1;WHILE I < C DO SET OutputRoot.*[I] = InputRoot.*[I]; SET I=I+1;END WHILE;DECLARE elementnum INTEGER;SET elementnum="InputBody"."totalselement"."f_reserve";SET

"OutputRoot"."MRM"."purchaseselement"."q_reserve"="InputBody"."purchaseselement"[elementnum]."itemquantity";

Some of the new ESQL features in WebSphere MQ Integrator version 2.1 are:

Environment: passes user defined variable information between nodes.

ESQL UUID function: creates a universally unique identifier.

Field references variables: these act as variable pointers.

Binary and Date/Time casting. SQLCODE and SQLSTATE database indicators: these are introduced.

ESQL node rationalization for Compute, Filter and Database nodes:

These each can modify both DBMS data and environment variables.Performance is improved by avoiding the need to create new messagetree elements for passing data between nodes in a flow.

They have drag and drop facility for ESQL code, selectable from a palette.

Improved support of NULLs. PROPAGATE keyword to send messages to the output terminal.

Figure 1-13shows the new ESQL palette on the left. From there, you can drag


39/483


and drop ESQL constructs to easily create valid ESQL statements.

Figure 1-13 ESQL code palette

1.7 Summary of new features

Following is a summary of the new features introduced with MQSeries IntegratorV 2.0.2 or are introduced in WebSphere MQ Integrator V2.1. Some of thesefeatures have already been discussed briefly in this introduction chapter. Most ofthem will be looked at in greater detail in the upcoming chapters.

MQSeries Integrator V2.0.2: MQSeries Everyplace and SCADA input and output nodes

New Era Of Networks V5.2 libraries and improved integration

Visual debugger for message flow development

XA transaction support for Oracle on Solaris

Performance improvements


40/483


Support of MQSeries 5.2 and Windows 2000

Supported HP-UX 11 platform

AS/400 supported via the integrated Windows NT environment

Usability improvements and plug-in node development wizard

A new manual: ESQL Reference

WebSphere MQ Integrator V2.1: Addition of the z/OS (OS/390) platform (using Unix System Services)

XML improvements (mixed content, attribute support, DTD import)

Tagged Message Support (user specified delimiters or tags)

Industry Standard messages (SWIFT, EDIFACT, X12, AL3)

Oracle and Sybase XA support on NT, AIX, Solaris and HP-UX

Plug-in API extensions (input nodes can now be developed)

Java environment available for plug-in nodes (but not parsers)

ESQL enhancements, including environment variable storage

New SupportPacs, including command line deploy

Available Control Center security exit

Message aggregation capability (three new nodes)

Improvements to the MRM (logical message model extensions)

MRM performance improvements


41/483

Copyright IBM Corp. 2002 27

Chapter 2. Migration considerations

This chapter discusses a variety of items that need to be considered whenplanning an upgrade to WebSphere MQ Integrator V2.1. We do not mean tocover every consideration possible. Many elements must be considered; many of

these are environment-specific and will depend on your individual installation.Hopefully, the major points we cover will give you plenty of insight to plan yourupgrade successfully.

Some of the things that need to be factored into an upgrade plan are based onthe current environment. Some are based on schedules. Others, and perhapsthe most often overlooked, are the requirements imposed by the software. Manyusers fail to fully read about and understand the differences between the releasethey are implementing and the one being replaced. We cannot stress enough the

need to read all documentation and any readme files shipped with the software.Readme files contain the most current information available on what you need toknow about the release being installed. For WebSphere MQ Integrator V2.1, thisinformation is also available on the Web:

http://www-3.ibm.com/software/ts/mqseries/support/readme/

This is especially helpful if you received the software some time beforeinstallation.

2

2.1 Considerations overview
http://www-3.ibm.com/software/ts/mqseries/support/readme/http://www-3.ibm.com/software/ts/mqseries/support/readme/


42/483


WebSphere MQ Integrator V2.1 contains several changes that require specialconsiderations when upgrading. There are also considerations to be taken intoaccount just because it is WebSphere MQ Integrator that is being upgraded.

WebSphere MQ Integrator requires that the following topics be considered duringan upgrade:

What platforms (operating systems) are involved? Are any components moving from one machine to another? Are there brokers that will move from one Configuration Manager to another? What impact will the upgrade schedule have on your promotion schedules for

your applications? Will there be co-existing multiple software versions? Are there any MRM domain messages with XML format in the environment? What database(s) is/are involved? If the database is DB2, will it be upgraded? If MQSeries Integrator V2.0.1 is installed, will it be upgraded to MQSeries

Integrator V2.0.2 or WebSphere MQ Integrator V2.1? Are all Maximum Message Lengths, Queues, and Sender and Receiver

Channels set to sufficiently large values? Are MQSeries logs large and numerous? Are the brokers or Configuration Managers upgraded first? What user IDs are used? Are the user IDs defined using proper authorities? Is the New Era Of Networks rules and/or formatter used? Will any custom nodes or plug-ins be created? How many message sets are there? Are all message sets and message flows archived outside of MQSeries

Integrator? Will the old version be uninstalled before the new version is installed?

2.2 Uninstalling the older version

Not all platforms require an older release to be uninstalled before installing thenewer one. Unfortunately, some do. Unix is one of those. There are also reasonswhy it would be desirable to uninstall the old version before installing the newone.

One of the reasons to uninstall has to do with the Windows platform. When anolder release was installed, it may have installed in directories that are version-or release-specific. It also may have created folders that are release- orversion-specific, including the one in the Start menu.

The install on a Windows platform will automatically install the upgrade in thedirectories and folders where the current version is installed It gives you no


43/483

Chapter 2. Migration considerations 29

directories and folders where the current version is installed. It gives you nochoice.

This can be quite confusing for some. Imagine that you have WebSphere MQIntegrator V2.1 installed, but all folders and directories have 2.0 in their name. Isthis confusing to you or your clients? You may want to correct this during theupgrade. Either create all folders and directories without any release or versioninformation in the name, or use the current release and version in the names.

The mostimportant thing to remember when performing an uninstall is that theuninstall does a delete of the broker. Therefore, if you plan to uninstall, be sure toundeploy everything from the brokers on the machine you are upgrading, beforeyou begin the uninstall.

2.3 Message repository changes

WebSphere MQ Integrator V2.1 has implemented an enhanced message setenvironment. Because of this, the tables in the message repository database(typically called MQSIMRDB) must be deleted and redefined.

This is significant to the person performing the upgrade. All message sets mustbe exported prior to beginning the upgrade process. Care must be taken toinsure that no one has anything checked out and that no one will be connected tothe product during the upgrade. Each message set must be exported individually.This can be a time consuming effort if there are numerous message sets. A scriptto execute the export for all message sets might be the best solution.

If it is part of the standard procedures to export all message sets and archivethem in another tool, this export step may be bypassed, but only if all messagessets are archived at their most current definition. This probably works best in anon-development environment, such as production or quality assuranceenvironments. Environments where developers are making changes would morethan likely have message sets that have not been archived. You must take thetime to either export all message sets or discover those that are not archived intheir present state and export them.

The upgrade requires that the Configuration Manager be deleted and recreatedto include all changes to the message repository. The delete is executed withoutaffecting the configuration database or the message repository database.

After installing WebSphere MQ Integrator V2.1, the tables in the messagerepository database will be dropped and the tables in the configuration database


44/483


p y pp gwill be migrated. This migration of tables will alter the tables to incorporate thechanges for this release. The changes to the message repository database aretoo complex to perform a migration. Therefore, it is required that you export

message sets, drop the tables, recreate the tables, and import the message sets.This is documented in the WebSphere MQ Integrator Installation Guide.

When the Configuration Manager is recreated, the message repository databasetables will also be recreated at that time. The configuration database is used in itsmigrated state. The newly recreated Configuration Manager will have all theinformation as to what was previously deployed to what brokers still available inthe configuration database. Care must be taken to insure that the ConfigurationManager is recreated with the proper service user ID and password.

2.4 Moving components

If a broker is to be moved from one Configuration Manager to another, it firstshould be totally removed from the current Configuration Manager. What thisentails is the removal of all message flows and message sets currently deployedto that broker. This broker must be deployed empty. Then the broker can beremoved from the current Configuration Managers topology. A full topology

deployment must be performed when deleting a broker. This is necessary toinsure that all database entries are updated properly. The broker can now bedefined to the topology of the new Configuration Manager.

If a broker is to be moved from one machine to another, it first must be deletedfrom the current Configuration Manager. What this entails is the removal of allmessage flows and message sets currently deployed to that broker. This brokermust be deployed empty. Then the broker can be removed from the ConfigurationManagers topology. A full topology deployment must be performed when

deleting a broker. This is necessary to insure that all database entries areupdated properly. Now the broker can be created on the new machine. Thebroker can now be redefined to the topology of the Configuration Manager.

If a Configuration Manager is to be moved from one machine to another, allbrokers must be removed from its topology, all message sets must be exported,and all message flows must be exported. The procedures outlined above formoving a broker are used to delete the brokers from the Configuration Managerstopology. After the message sets and flows have been exported, the

Configuration Manager can be deleted. It can now be defined on the newmachine. All message sets and flows can be imported, and brokers defined tothe topology and message sets and flows deployed.

It is not necessary to delete a Configuration Manager before creating itsreplacement on another machine. However, the brokers must be removed from


45/483


pthe original topology before they can be assigned to the topology on the newmachine.

Again, if the message sets and message flows are archived in an external tool,they may not need to be exported at this point. They can be recovered from thearchive. But if there is any question as to the validity of the archived components,or just as an extra precaution, it may be desirable to go ahead with the exportanyway. This is a risk versus effort decision.

2.5 Enhancement and schedule considerations

When upgrading, it is unlikely that all environments will be upgraded at the sametime. At the very least, a production environment would not be upgraded until theupgrade has been verified in another environment. Because of this, thedifferences between releases and the production schedule must be considered.These factors have an impact on how much time there is to accomplish theupgrade, the order in which the components are to be upgraded, how thorough atest must be performed, and what functions in the new release need to beavoided until they can be promoted to production.

At the most basic level, care must be taken to avoid using new functionality inapplications until that functionality is available in production. This is why theschedule becomes an important factor. The depth and length of testing has adirect impact on how long you have to be careful not to use the new features. It isimportant to avoid a situation in which a change has been thoroughly tested, yetdoes not work as planned when promoted to production, due to incompatiblefunctions between releases.

Although you may be careful to avoid using new functions, you may come acrossthe case where something that was allowed in the current release cannot besupported in the new release or across releases. Messages that are in the MRMdomain and in the XML format are not supported across releases. Therefore,applications that use this scenario cannot be modified in an environment wherethe broker and Configuration Manager are not at the same release level.

Most of the time, when something that was supported in the old release is notsupported in the new release, it is because someone has found a hole in thelogic. When something is documented as not working but someone finds that i twill work, or something works that does not seem like it should, do not use it. If itwas documented as not available or as working one way and not the way it is

actually working, it will probably not work in a near future patch or upgrade or in anew release. This is good advice for any product at any time. Nothing is as


46/483


frustrating as having to rewrite an application because holes have been closed inthe software you have been using.

The changes to the message repository in V2.1 make it necessary to carefullychoose when the upgrade is to be performed. Once the upgrade has begun onany environment that will have components promoted to another environment, allpromotions from the upgraded environment that involve MRM messages mustbepostponed until the environment promoted to is also upgraded to V2.1.

This can have a significant impact on the schedule, especially if there are manyenvironments involved in an applications promotion path. A great deal of timecan pass between the production environment upgrade and the first environmentupgrade. This is where the application problems or enhancements must be taken

into consideration. It is very annoying to begin an upgrade knowing it is going totake a fixed amount of time and then find that an application change is scheduledto begin after you have begun, but is scheduled for production before you are.This affects both the personnel performing the upgrade and the cl ient who mustnow wait for changes until the upgrade is complete.

When performing an upgrade, it is common for the upgrade to begin at the lowestcommon denominator. In the case of WebSphere MQ Integrator, that would bethe broker. The WebSphere MQ Integrator V2.1 Administration Guidedocuments

that you can have a V2.1 broker with a V2.0.1 or V2.0.2 Configuration Manager ifyou install the compatibility modules on the broker. It also strongly recommendsthat the Configuration Manager be upgraded first. The brokers should beupgraded immediately following the Configuration Manager upgrade and beforeanything is deployed.

The best scenario is to upgrade both the Configuration Manager and all of itsbrokers at the same time. If there is a broker on a UNIX machine, the softwaremust be uninstalled before the new release can be installed. Because of this and

the changes in the message repository, it is best if all flows and message setsare undeployed, the software upgraded and flows and message sets redeployed.This provides the surest method of insuring that all components are upgradedand that all deployed flows and message sets match exactly what is in theConfiguration Manager.

If there is no UNIX machine attached to the Configuration Manager, thenundeploying the message flows is not required. However, because of thechanges in the message repository, the currently deployed message sets should

be undeployed and redeployed after the upgrade. Though this is not absolutelyrequired at this point, it will be required before the message sets can be deployedagain after the upgrade. As documented in 2.3, Message repository changeson page 29, the UUID of the message set will change when it is reimported after

the upgrade. When the message set is later deployed, a duplicate message setname error will occur. This is because the message set is already deployed withh UUID d i l d h d d h UUID b


47/483


the UUID created previously and the upgraded message set has a new UUID butthe same message set name. The previously deployed message set must beundeployed before the new one can be deployed.

In an ideal world, there would be only one application environment per machine.That is, if t

Date post:	07-Aug-2018
Category:	Documents
Upload:	atish-jadhav
View:	227 times
Download:	0 times

Websphere MQ Integrator

Documents