2
Agenda
• The DB2 provided MQ Listener and Functions
• QREP – aka InfoSphere Replication Server• What it is and how it works• Issues we have seen• Options and Requirements
3
DB2 provided MQ Listener and Functions
• MQListener• DB2 MQ Services• DB2 MQ Policies• MQSEND• MQREAD and MQREADCLOB• MQRECEIVE and MQRECEIVECLOB• Using WebSphere MQ pub/sub with DB2
4
The DB2 MQListener Process
MQ Listener
MQ Queue
5
The DB2 MQListener
• The DB2 MQListener is a combination of WMQ and DB2 functionality• DB2 Process will listen for messages on a queue• Invoke a stored procedure to act on the message• Possibly send a reply
• Often used to send updates asynchronously, when the data is ‘preformatted’• ‘Near realtime’ updates with all the advantages of queuing! • Growth in usage for event and audit trail collections
6
DB2 MQListener - continued
• The DB2 MQListener on z/OS:• Provides both 1 and 2 phase commit for resources
• db2mqln1 - single phase • Db2mqln2 – two phase
• Installs and runs in USS• Configured via database updates
• SYSMQL.LISTENERS table
• From an MQ perspective:• Standard WMQ administration• Good recommendations for DLQ, Backout queue and
threshold
7
The DB2 MQListener - continued
• The Listeners table entries include:• Configuration name• Queue manager• Input queue• Node – unused• Schema – the schema name for the stored procedure to be invoked• Name – the name of the stored procedure• Type – unused• Instances – number of instances of the procedure that will be
invoked • Wait – the number of milliseconds that the listener should wait
before checking for a new message
8
DB2 WMQ Functions
• MQListener• DB2 MQ Services• DB2 MQ Policies• MQSEND• MQREAD, including
• MQREADALL• MQREADALLCLOB • MQREADCLOB
• MQRECEIVE, including • MQRECEIVEALL • MQRECEIVEALLCLOB• MQRECEIVECLOB
• Using MQ pub/sub with DB2 WMQ functions
9
The DB2 Service Definitions for MQ Functions
• Defines the MQ resources that the DB2 MQ functions use• The connection between DB2 and WMQ• Typically maintained by the DB2 admin
• SYSIBM.MQSERVICE_TABLE - Holds the following information:• Service Name - DB2.DEFAULT.SERVICE• Queue manager - !QUEUEMANAGER!• Queue name - DB2MQ_DEFAULT_Q
• This field is called ‘Input queue’• CCSID• Encoding• Description
10
DB2 Policy Definitions for MQ Functions
• Typically maintained by the DB2 admin• Defines the MQ quality of service for the DB2 MQ functions
• The send and receive values correspond to MQ message settings
• SYSIBM.MQPOLICY_TABLE - Holds the following information:• Policy Name - DB2.DEFAULT.POLICY• Send Values:
• Priority – 0-9, and -1 which is ‘as queue defined’• Persistence
• Q – As queue defined• Y - Persistent - messages are logged and recoverable• N - Nonpersistent – messages are not logged
11
DB2 Policy Definitions for MQ Functions - Continued
• Send Values Continued• Expiry
• Messages can be set to expire if not processed • Retry Parameters
• Retry Count• If the put of the message fails, the put can be retried up
to this limit • Retry Interval
• The wait time in milliseconds between retries of the message put
12
DB2 Policy Definitions for MQ Functions - Continued
• Send Values Continued• Correlid Settings
• New Correlid• The correlation ID can be set to new or not
• Response Message ID• Sets whether the report message will have the same message
ID as the original • Response Correlation ID
• Whether the processing application should set the correlid on the reply message from the request message ID or request correlid
13
DB2 Policy Definitions for MQ Functions - Continued
• Send Values Continued• Exception Actions
• A problem message can be discarded or put to the queue manager’s dead letter queue
• Report Options• Exception reports
• Send or do not send an exception message • Confirm on Arrival
• Send or do not send an confirmation report when the message is put to the target queue
• Confirm on Delivery• Send of do not send a confirmation report when the message
is retrieved from the target queue• Message expiration
• Send or do not send a report when the message expires.
14
DB2 Policy Definitions for MQ Functions - Continued
• Send Values Continued• Message Type• Reply to Queue• Reply to Queue Manager
• Receive Values:• Wait interval• Accept Truncated Message• Open Shared
• Syncpoint
15
DB2 Policy Definitions for MQ
•The foils that follow•Give the DB2/MQ cross reference
•Most are hidden for the presentation
•Highlighted (and not hidden) are where MQ recommendations differ from the DB2 default
Who lives in the details???
16
DB2 Policy Definitions for MQ Functions – Send columns
Confirmation of delivery reply is sent when the message is retrieved from the queue, and the retreivalremoves the message (aka ‘destructive MQGET’).
MQRO-COD BINARY 2048
MQMD-REPORTCSEND_REPORT_COD
Confirmation of arrival reply is sent when the message arrives on the specified queue.
MQRO-COA BINARY 256
MQMD-REPORTCSEND_REPORT_COA
COD reply is sent with the first 100 bytes of the message body
MQRO-COD-WITH-DATA BINARY 6144
MQMD-REPORTDSEND_REPORT_COD
COD reply is sent with the entire message body
MQRO-COD-WITH-FULL-DATA BINARY 14336
MQMD-REPORTFSEND_REPORT_COD
COA reply is sent with the first 100 bytes of the message body
MQRO-COA-WITH-DATA BINARY 768
MQMD-REPORTDSEND_REPORT_COA
COA reply is sent with the entire message body
MQRO-COA-WITH-FULL-DATA BINARY 1792
MQMD-REPORTFSEND_REPORT_COA
MQMD-REPORT
MQMD-REPORT
MQMD Field
N -Default
N -Default
Value
No COA is sent. MQRO-NONE BINARY 0
SEND_REPORT_COA
No COD is sentMQRO-NONE BINARY 0
SEND_REPORT_COD
MeaningMQ ValueDB2 Policy Name
17
DB2 Policy Definitions for MQ Functions – Send columns
• Each report option has four values:• ‘N’ – do not report • ‘C’ – send a report message, but do not include any of the
message data• ‘D’ – send a report message, include the first 100 bytes of
message data• ‘F’ – send a report message and include the entire message
body
18
DB2 Policy Definitions for MQ Functions – Send columns
The reply to queue manager for request and report messages.
SpacesMQMD-REPLYTOQMGR
SAME AS INPUT_QMGRDefault
REPLY_TO_QMGR
The reply to queue for request and report messages. This SHOULD NEVER BE ALLOWED TO DEFAULT LIKE THIS! Infinite loops will occur if left like this.
SpacesMQMD-REPLYTOQSAME AS INPUT_QDefaultREPLY_TO_Q
MQMD FieldValue MeaningMQ ValueDB2 Policy Name
19
DB2 Policy Definitions for MQ Functions – Receive columns
The message body is not converted by WebSphere MQ.
zeroMQGMO-OPTIONS NRCV_CONVERT
The message body is to be converted from the CCSID and encoding from the sender to the CCSID and encoding used by the receiving queue queue manager. Typically the data is in string format.
MQGMO-CONVERT BINARY 16384
MQGMO-OPTIONS YDefault
RCV_CONVERT
The time in milliseconds that the MQGET request will wait on an inbound message. Note that in MQ, waiting is implemented using a value on the MQGET options and an interval.
ZeroMQGMO-WAITINTERVAL
10Default
RCV_WAIT_INTERVAL
MQMD Field(s)Value MeaningMQ ValueDB2 Policy Name
20
DB2 Policy Definitions for MQ Functions – Receive columns
Queue is open in shared mode, so other applications can retrieve messages from the queue concurrently.
MQOO-INPUT-SHARED BINARY 2
MQ Open optionsSDefault
RCV_OPEN_SHARED
Queue is open for input to this application only. Used when data serialization is required.
MQOO-INPUT-EXCLUSIVE BINARY 4
MQ Open optionsERCV_OPEN_SHARED
If the message is larger than the buffer provided, the message is not retrieved. The function will get a reason code of MQRC_TRUNCATED_MSG_FAILED.
Zero.MQGMO-OPTIONS NRCV_ACCEPT_TRUNC_MSG
As much of the message as can be is retrieved into the message buffer available. The remainder is lost when the transaction completes. Note that DB2 recommends this behavior, but those of us in WMQ do not. Some of the message body is lost.
MQGMO-ACCEPT-TRUNCATED-MSG BINARY 64.
MQGMO-OPTIONS YDefault
RCV_ACCEPT_TRUNC_MSG
Queue is opened with the values as defined on the queue.
MQOO-INPUT-AS-Q-DEF BINARY 1
MQ Open optionsDRCV_OPEN_SHARED
MQMD Field(s)Value MeaningMQ ValueDB2 Policy Name
21
The Functions: MQSEND
• MQSEND is a SQL-centric way of putting messages on a queue
• The parameters include:• The Send Service • The Service Policy • Message Data• Correlid
Test Send Service Definition
INSERT INTO SYSIBM.MQSERVICE_TABLE (SERVICENAME, QUEUEMANAGER,
INPUTQUEUE, DESCRIPTION) VALUES('LYNMQSEND','QML1','LYNE.TESTDB2.MQSEND',
'LYN ELKINS MQSEND TEST');
22
Test MQSEND Policy DefinitionINSERT INTO SYSIBM.MQPOLICY_TABLE
(POLICYNAME, SEND_PRIORITY, SEND_PERSISTENCE, SEND_RETRY_COUNT,
SEND_RETRY_INTERVAL,SEND_NEW_CORRELID, SEND_RESPONSE_MSGID,
SEND_RESPONSE_CORRELID, SEND_EXCEPTION_ACTION, SEND_REPORT_EXCEPTION,
SEND_REPORT_COA, SEND_REPORT_COD, SEND_REPORT_EXPIRY,
SEND_REPORT_ACTION, SEND_MSG_TYPE, REPLY_TO_Q, REPLY_TO_QMGR,
RCV_WAIT_INTERVAL, RCV_CONVERT, RCV_ACCEPT_TRUNC_MSG,
RCV_OPEN_SHARED, SYNCPOINT, DESC)
VALUES('LYNE_MQSEND_POLICY','-1','Q', '4',
'1000', 'N', 'N',
'C', 'Q', 'N',
'N','N', 'N',
'N', 'DTG', 'LYNE.TESTDB2.BADQ', 'QML1',
'10', 'Y', 'N',
'S', 'Y', 'LYNS TEST MQSEND POLICY');
23
Test MQSEND results
24
MQREAD and MQRECEIVE
• MQREAD and MQRECEIVE are SQL-centric way of retrieving messages from a queue
• The parameters include:• The Send Service • The Service Policy• CORRELID for MQRECEIVE
• MQREAD and MQREADCLOB are non-destructive MQGET operations
• MQRECEIVE and MQRECEIVECLOB are destructive MQGET operations
25
MQREAD & MQREADALL Tests
Service Definition:
INSERT INTO SYSIBM.MQSERVICE_TABLE (SERVICENAME, QUEUEMANAGER, INPUTQUEUE, DESCRIPTION) VALUES('LYNMQREAD','QML1','LYNE.TESTDB2.MQREAD', 'LYN ELKINS MQREAD TEST');
26
MQREAD Test
27
MQREADALL Test
28
MQRECEIVE & MQRECEIVEALL Tests
Service Definition:
INSERT INTO SYSIBM.MQSERVICE_TABLE (SERVICENAME, QUEUEMANAGER, INPUTQUEUE, DESCRIPTION)
VALUES('LYNMQRECEIVE','QML1','LYNE.TESTDB2.MQRECEIVE','LYN ELKINS MQRECEIVE TEST');
29
MQRECEIVE Test
30
MQRECEIVEALL Test
31
Using MQ Pub/Sub Features from DB2
• When you require more control over the messages that a particular service should receive, you can use the MQPUBLISH function, in conjunction with the WebSphere MQSeries Integrator facility. This facility provides a publish-and-subscribe system, which provides a scalable, secure environment in which many subscribers can register to receive messages from multiple publishers. Subscribers are defined by queues, which are represented by service names.
• MQPUBLISH enables you to specify a list of topics that are associated with a message. Topics enable subscribers to more clearly specify the messages they receive. The following sequence illustrates how the publish-and-subscribe capabilities are used:
• An MQSeries administrator configures the publish-and-subscribe capability of the WebSphere MQSeries Integrator facility.
• Interested applications subscribe to subscriber services that are defined in the WebSphere MQSeriesIntegrator configuration. Each subscriber selects relevant topics and can also use the content-based subscription techniques that are provided by Version 2 of the WebSphere MQSeries Integrator facility.
• A DB2® application publishes a message to a specified publisher service. The message indicates the topic it concerns.
• The MQSeries functions provided by DB2 for z/OS® handle the mechanics of publishing the message. The message is sent to the WebSphere MQSeries Integrator facility by using the specified service policy.
• The WebSphere MQSeries Integrator facility accepts the message from the specified service, performs any processing defined by the WebSphere MQSeries Integrator configuration, and determines which subscriptions the message satisfies. It then forwards the message to the subscriber queues that match the subscriber service and topic of the message.
• Applications that subscribe to the specific service, and register an interest in the specific topic, will receive the message in their receiving service.
32
Publishing a message from DB2
• In WebSphere MQ set up the following definitions:• A WMQ topic
33
Publishing a message from DB2
• A WMQ alias queue
34
• In DB2 set up the following definition:• A new service using the ALIAS queue name
Publishing a message from DB2
35
Publishing a message from DB2 -Testing
• In WMQ, define the following:• Two local queues to
serve as the subscription destination
• LYNE.TESTDB2.Q1• LYNE.TESTDB2.Q2
• Two subscriptions using the topic defined earlier and the local queue as the destination for the publications
36
Publishing a message from DB2 -Testing
• Use the MQSEND function to publish
37
Publishing a message from DB2 -Testing
• Both queues that are targets of the subscriptions have a copy of the message
38
Using native MQ Subscriptions and DB2
• This is basically an administrative task• Define the subscriptions to MQ pointing• Define MQ policies in DB2 that point to those queues• Use the MQREAD/MQRECEIVE functions to retrieve the
publications (messages)
39
Now for something completely different
• QREP• Aka InfoSphere Replication Server
40
Q Replication Technology• Part of the InfoSphere Data Replication product• A software-based asynchronous replication solution
• For Relational Databases• For selected tables/columns/transactions/operations• Changes are captured from the database recovery log; transmitted as (compact) binary data;
and then applied to the remote database(s) using SQL statements. Technology is log-capture/transaction-replay
• Each DB2 is ACTIVE and can be used for READ/WRITE operations
DB2
Control Tables
Site ADB2
Control tables
Q CaptureQ Apply
agentagentagentUser
tablesdatabase recoverylog
User tables
Unlimited Distance
Site B
Configuration & Monitoring
logrdr publishData CenterWebSphere MQ
DB2 Transaction Parallel Replay
AsynchronousLOG change data capture
Active DB2Active DB2 Persistent Staging Capability
SQL statements
41
Q Replication Process
WebSphere MQ
queue manager
DB2
Site 1
DB2
Q Capture
Q Applyagentagentagent
User tables
database recoverylog
User tables
WebSphere MQ
queue manager
Network
Site 2
Applications
Remote Admin QAdmin queue
Send queue and XMITQ
Restart queue
Receive queue(s)
Captured DB2 Transactions from the LOG
logrdr
publish
Q Apply agents execute DB2 SQL transactions
DB2 transactions
SQL statements
SQL statements
• Capture and Apply are DB2 applications• Parallel Capture program publishes captured database transactions as compact MQSeries
messages, only the data is sent• One MQ message = One captured DB2 transaction
• Apply program replays transactions by rebuilding SQL statements• Each component runs independently and can be individually stopped/restarted at any time, in any
order
42
WebSphere MQ Queue Manager (or client on LUW)
DB2
Active/Active Configurations
Control Tables
Site A
DB2
Control tables
Q Capture
Q Applyagentagentagent
Q Applyagentagentagent
User tables
database recoverylog
User tables
Q Capture
WebSphere MQ Queue Manager (or client on LUW)
Network1000s of
kilometers
Site B
Applications
Remote Admin QAdmin queue
Receive queue(s)
Send queue(s)
Restart queue Restart queue
Remote Admin Q
Send queue(s)
Receive queue(s)
Admin queue
Configuration & Monitoring
• Both sites are active• Sites can be asymmetric; with different capacities, and at different HW/OS/SW levels• A workload can be immediately redirected for recovery or planned maintenance (RTO of sub-second)
• Replication is defined as bi-directional• Bi-directional, even if one side is used for read-only or standby for all or a subset of the workloads• Replication is ready for fallback after a failover, from any direction
• Routing decision determines the role. e.g., West-Coast customers to site A and East-Coast customers to site B, until failover. Objective is to avoid conflicts.
West-Coast OLTP workload East-Coast OLTP
workload
sub-secondlatency
Query Reporting workload
database recoverylog
43
Conflict Detection and Resolution with Q Replication -Bidirectional configurations
‘Bidirectional’ mode – detection by value - resolution Force/Ignore; Ignore/Ignore
• 2 or up to tens of sites connected though a central site; a hub and spokes model (or tree topology)
• Conflict detection: Compare current values at the target with the before values carried over from the source. Check key, changed cols only, or all columns
• Possibility of merging updates to different columns of the same row made from different sites
• Conflict action: Force or Ignore (merging updates is not a conflict) - Designated site wins in case of conflict, master/master is OK
• Recapture avoidance: Apply inserts a row into the SIGNAL table with each transaction it applies. If there are only 2 sites, can use Capture IGNORE TRANSACTION for less overhead.
Resolving conflicts at the row level minimize the impact of conflicts by “merging” transactions for non conflicting changes. One conflict in a transaction doesn’t invalidate the whole transaction, which could result in more conflicts later on. Resolved conflicts are reported into the EXCEPTIONS table. Each conflict is reported only once.
A: Master
C: Master or Slave
B
44
Q Apply includes the before values in the where clause for conflict detection (bi-di protocol)In ‘bidi’, Q Apply augments the SQL with a where clause to check old values, e.g.:
• table T1(ID int primary key, C1 int, C2 varchar);• Row1: (100, 5, ‘Jim’)
• SQL by source application:• update T1 set C1=6 where C2=‘Jim’;
• DB2 log record:• Old row: ID = 100, C1 = 5; C2 = ‘Jim’• New row: ID = 100, C1 = 6; C2 = ‘Jim’
• SQL by Q Apply:• update T1 set C1=6 where ID=100 and C1=5 and C2=‘Jim’;
•The SQL where clause was augmented to verify that the before value was not modified and that the correct row is updated.
• If row not found (sqlcode -803) --> CONFLICT!• The same technique is used for uni-directional replication, but only the replication key before value is included in the where clause.
45
The EXCEPTIONS table• All conflicts and errors are reported in EXCEPTIONS table• Complete SQL statement is logged for each conflict• Sample entry:
EXCEPTION_TIME RECVQ SUBNAME OPERATION REASON-------------------------- ----- ---- --------- ---------2009-06-17-12.35.41.387373 Q1 S1 INSERT DUPLICATE
SRC_COMMIT_LSN SRC_TRANS_TIME IS_APLLIED----------------------- -------------------------- -----------x'00000000000002F1F0A5' 2009-06-17-19.35.39.000001 Y
SRC_TRANS_ID CONFLICT_RULE----------------------------------- --------------------x'000000000000000002E3' K
SQLCODE SQLSTATE SQLERRMC------- -------- ---------------------------------803 23505 x’31FF0000...0000000000000000000'
TEXT-------------------------------------------------------------------INSERT INTO "SOMILK"."TARG_T1“( "KEYP“, “PAY1”, “PAY2”) VALUES (1, ‘’, NULL)
46
Q Replication Potential Bottlenecks
WebSphere MQ
queue manager
DB2
Site A
DB2
Q CaptureQ Apply
agent
agent
agentUser
tables
database recoverylog
User tables
WebSphere MQ
queue manager
Channel (TCP/IP)
Site BApplications
Send queue and XMITQ Receive queue(s)
Captured DB2 Transactions from the LOG
(1 per MQ msg)
logrdrpublish
Q Apply agents execute SQL transactions
SQL statements
SQL statements
1. MQ channel (single TCP/IP connection) for OLTP transactions (small messages) limited at 7MB/second for 2K messages2. MQ bufferpool filling up. Reading or writing to the pageset can reduce replication throughput by 4 times.3. Capture publish thread might become CPU bound, this thread includes MQ PUT and COMMIT elapsed times. 4. Workload contention at the target (locking and or dependency of transactions updating the same rows) can be caused by
applications running at the target or by multiple apply agents applying transactions in parallel.5. Apply program limited to one data sharing member.6. CPU and I/O Speed
DB2 IFI 306(read log I/F) MQ pageset
MQGET (browse)browse
prune
MQGET (delete)
MQ Bufferpool
MQPUT, MQCMT
MQ pageset
MQ Bufferpool12
3
4
5
Applications
47
Q Replication Parameter SettingsQ Capture:• MAXTRANS(new parm)=500. At one customer, an effective MQ batch size of 42 was the best
that cold be acheived. Tests have established that an MQ batch size of 200 is optimal, with good values ranging from 50 to 200. COMMIT_INTERVAL=500 (milliseconds). Q Capture commits MQ after a maximum of 500ms. However, at high throughput, the maxtrans limit is reached much sooner, and Capture commits much more frequently, about every 50 ms.
• SLEEP_INTERVAL= 50 (milliseconds) - This was established as being optimal during the customer and internal tests.
• TRANS_BATCH_SZ=3 and TRANS_BATCH_SZ=1. This value may vary, at one high volume customer a value of 3 was established as yielding the best performance results. With value 3, Q Replication replicate 3 DB2 transactions as 1.
• MEMORY_LIMIT=500 (MB) – This setting had no significant impact in our testing, equally good results were achieved with 200MB.
Q Apply:• NUM_APPLY_AGENTS=48 - The optimal number of agents depends on cpu resources, the
characteristics of the replicated workload, and possibly contention for locks. Q Replication provides monitor counters to help tune this parameter (apply_sleep_time).
• MEMORY_LIMIT= 500MB
48
Queue Replication – Ongoing work and suggestions
• The QREP development team are currently evaluating• Using log compression
• Very data dependant
• Some suggestions that have been made: • More flexibility in Message Persistence requirements
• Right now at a ‘universal’ setting• Perhaps using a technique like FTE, where most messages are
nonpersistent, only status (‘blocking’) messages are persistent• Multiple capture and apply queues
• Message groups processed in a round-robin fashion
49
CICS and WMQ - The Resurrection of Useful
Getting the best availability from MQ on z/OS by using Shared Queues
Clustering - The Easier Way to Connect Your Queue Managers
06:00
WebSphere MQ Channel Authentication Records
MQ & DB2 – MQ Verbs in DB2 & Q-Replication
What’s New in the WebSphere MQ Product Family
WebSphere MQ application design, the good, the bad and the ugly
04:30
Using IBM WebSphere Application Server and IBM WebSphere MQ Together
WebSphere MQ CHINIT Internals
BIG Connectivity with WebSphere MQ and WebSphere Message Broker
First Steps With WebSphere Message Broker: Application Integration for the Messy
03:00
The Dark Side of Monitoring MQ - SMF 115 and 116 Record Reading and Interpretation
Migration and maintenance, the necessary evil
MQ on z/OS – VivisectionIntroduction to MQ01:30
12:15
BIG Data Sharing with the cloud - WebSphere eXtreme Scale and WebSphere Message Broker integration
WMQ - Introduction to Dump Reading and SMF Analysis -Hands-on Lab
Extending IBM WebSphere MQ and WebSphere Message Broker to the Cloud
11:00
Diagnosing Problems for MQ
What's New in WebSphere Message Broker
09:30
Are you running too many queue managers or brokers?
08:00
FridayThursdayWednesdayTuesdayMonday
This was session ???? - The rest of the week ……
50
MQ & DB2 – MQ Verbs in DB2 & Q-Replication [z/OS]Wednesday, February 6, 2013: 4:30 PM-5:30 PM