Problems and Solutions withAdabas Replication and Mass Data

Dieter W. Storr – www.storrconsulting.com

9 May 2011

04/22/23 | Software AG User Group 2011 | 2

Contents

• Mirroring and Replication

• Different Ways to Replicate

• Replication Scenario and Used Versions

• Replication Components and their Commands per Second

• Subscription DB: CPU Time With and Without Replication

• Batch Jobs: Duration With and Without Replication

• Buffers and Communication between Source DB and Reptor

• Challenges and Solutions

04/22/23 | Software AG User Group 2011 | 3

Source: Bruno Bebert/DPA

Mirroring and Replication

04/22/23 | Software AG User Group 2011 | 4

Functions Mirroring Replication

Copy tracks / blocks Yes No

Repeat transactions No Yes

Can target database be read during the copy/repeat process

No Yes

Will errors in tracks be copied? Yes No

Can data / fields be selected? No Yes

04/22/23 | Software AG User Group 2011 | 5

Type of Replication – Mainframe to Adabas on Mainframe

04/22/23 | Software AG User Group 2011 | 6

Type of Replication – Mainframe to Adabas on LUW

04/22/23 | Software AG User Group 2011 | 7

Type of Replication – Mainframe to non-Adabas

Replication Scenario – Enterprise Server to Windows

04/22/23 | Software AG User Group 2011 | 8

RPL V3.3.1 ADA V6.2.1

04/22/23 | Software AG User Group 2011 | 9

Commands per Second During Replication

SOURCE DB8.1.3

----------------RPL Buffer

REPTOR8.1.3 / 3.2.1

--------------RPL Buffer

TARGET DB

6.1.8.03CSTZ007

SLOG

WCP6.2.1

WCP7.3.3

KZ007

1,388 cps 79.8 cps ? cps 76.5 cps

100 mbps line circuit

2 mbps

1,307 cps 519.5 cps ? cps 297- 417 cps6 mbps

BOE

DMH

DMV

1

2

73.4 cps

no log/trace

C-drive for log/trace only

16:42:04-16:44:37

15:09:20-15:11:44 15:11:43-15:53:29

16:44:37-16:51:02

15:09:21-15:53:30BFIO=0

16:42:04-16:51:03BFIO=0

1,212 cps 511.5 cps ? cps no log/trace7 mbps3

no log/trace

10:32:08-10:34:53 10:34:53-10:41:24

1,333 cps 520.8 cps ? cps no log/trace7 mbps4

no log/trace

11:11:27-11:13:57 11:13:56-11:20:20BFIO=25

BFIO=0

1,587 cps 190.1 cps ? cps no log/trace7 mbps5

no log/trace

11:39:26-11:41:3211:41:32-11:59:04SLOG used BFIO=50

Was interrupted, and restarted

04/22/23 | Software AG User Group 2011 | 10

Commands per Second During Replication

1,212 cps 350.26 cps ? cps 369.69 cps6 mbps6

no log/trace

14:56:04-14:58:49 14:56:05-15:05:3614:56:36-15:05:37BFIO=50

673 cps 294.55 cps ? cps 294.12 cps5.5 mbps7

no log/trace

17:14:40-17:19:37 17:18:30-17:26:00 17:14:41- 17:26:01

1,105 cps 359 cps ? cps 358 cps7.0 mbps8 no log/trace

15:16:22-15:19:23 15:16:23-15:25:40 15:16:23-15:25:41

1,914 cps 503 cps ? cps 272 cps8.5 mbps9 no log/trace

11:28:14-11:31:43 11:29:00-11:40:25 11:27:49-11:52:17

1,592 cps 361 cps ? cps 358 cps6.5 mbps10

no log/trace

14:56:47-15:03:04 14:56:56-15:24:4014:56:48-15:24:40

1,148 cps 295 cps ? cps 294 cps5.9 mbps11

no log/trace

15:48:48-16:09:08 15:48:56-17:08:08 15:48:49-17:18:10

7 files

3 files

2 files

1 file

1 file

1 file

CPU Time and Duration

04/22/23 | Software AG User Group 2011 | 11

Prod DB

ASSO DATA WORK

9 Batch Jobs Duration HH:MM

Without RPL 02:20

Prod DB CPU Time Per Week With 9 Jobs Running

Without RPL

08:39:17

CPU Time and Duration

04/22/23 | Software AG User Group 2011 | 12

Prod DB

ASSO DATA WORK

Reptor

SLOG

SYS

9 Batch Jobs Duration

HH:MM

Without RPL 02:20

With RPL – unchanged parms

13:22

Prod DB CPU Time Per Week With 9 Jobs Running

w/o RPL 08:39:17

w/ RPL 17:19:59

04/22/23 | Software AG User Group 2011 | 13

Duration of 9 Jobs

24 December 2010

Without Replication

12 January 2011

With Replication

Job From - To Duration From - To Duration5031 19:01 – 21:00 1:59 19:02 – 07:27 12:255032 19:01 – 20:58 1:57 19:02 – 21:38 2:365033 19:01 – 20:57 1:56 19:01 – 07:11 12:105034 19:01 – 21:06 2:05 19:02 – 07:58 12:565035 19:01 – 21:21 2:20 19:01 – 08:22 13:215036 19:01 – 21:18 2:17 19:01 – 08:23 13:225037 19:01 – 20:59 1:58 19:01 – 07:29 12:285038 19:01 – 20:59 1:58 19:01 – 07:30 12:295039 19:01 – 21:00 1:59 19:01 – 07:33 12:32

TOT 19:01 – 21:21 2:20 19:01 – 08:23 13:22

Buffers and Communication Between Source and Reptor

04/22/23 | Software AG User Group 2011 | 14

04/22/23 | Software AG User Group 2011 | 15

RPL Buffer RPL=524,288,000

88 % used

LBP = 70,000,000

I/O BufferLFIOP = 25% of LBP

Files / Tables

Files / Tables

Files / Tables

RPL Buffer RPL=524,288,000

13 % used

LBP = 3,600,000

I/O BufferLFIOP = 900,000 = 25% of LBP

SLOGFile 30

DB 134 (PROD)Source /

Subscription DB

DB 312 (Reptor)Replicator Engine

DB

7,400 buffer flushes

38,000 buffer flushes

803,970 records loaded15,000 Cylinders defined12,375 Cylinders unused

Buffer Usage During Replication- Data Flow -

Dieter Storr – 14 Jan 2011

BufferUsageDuringReplication

Several Tests With 20% of Production Data

Test #

Date RPL LBP in MG

LFIOP in MB

LRPL in MB

ASYTVS

FMXIO

9 Jobs Duration

1 01/27/2011 YES 70 13 500 YES 1 79 min

2 01/28/2011 NO 70 13 500 YES 1 40 min

3 01/31/2011 YES 150 13 500 YES 1 74 min

4 02/24/2011 YES 150 13 700 YES 8 61 min

04/22/23 | Software AG User Group 2011 | 16

STROBE: Buffer Pool Manager – High CPU TimeSource Database

04/22/23 | Software AG User Group 2011 | 17

MODULE SECTION LINE PROCEDURE/FUNCTION STARTING PROCEDURE CPU TIME NAME NAME NUMBER NAME LOCATION LENGTH SOLO   .ADABAS ADANC6 ADANC6 ADABAS BUFFER POOL MNGER 51.49 .ADABAS ADARPN ADARPN ADABAS SYSTEM SERVICES 12.61 .SVC SVC 114 EXCPVR .05 .ADABAS ADANC1 ADANC1 ADABAS THREAD SELECTION .17 .ADABAS ADANC8 ADANC8 ADABAS CMPRESS DVT HNDLR .11 .ADABAS ADANC0 ADABAS USR/HYP EXIT CALL .32 .ADABAS ADAIOS ADABAS I/O PROCESSING .21 .ADABAS ADANC5 ADABAS QUEUE PROCESSING .10 .ADABAS ADANC7 ADABAS FORMAT TRANSLATOR .03 .SUPERVS IGC0024A SUPERVISOR SERVICES .05

STROBE: Buffer Pool Manager – Normal CPU TimeReptor

04/22/23 | Software AG User Group 2011 | 18

MODULE SECTION LINE PROCEDURE/FUNCTION STARTING PROCEDURE CPU TIME NAME NAME NUMBER NAME LOCATION LENGTH SOLO  .ADABAS ADARPS ADARPS ADABAS SYSTEM SERVICES 27.27.SUPERVS IGC0024A SUPERVISOR SERVICES 13.64.ADABAS ADANC0 ADANC0 ADABAS USR/HYP EXIT CALL 9.09 .ADABAS ADANC6 ADANC6 ADABAS BUFFER POOL MNGER 8.18 .ADABAS ADANC1 ADANC1 ADABAS THREAD SELECTION 7.27 .SVC SVC 114 EXCPVR .00 .ADABAS ADANC7 ADANC7 ADABAS FORMAT TRANSLATOR 3.64 .ADABAS ADANC5 ADANC5 ADABAS QUEUE PROCESSING 2.73 .NUCLEUS IEAVESLK SUSPEND LOCK SERVICE 2.73 .ADABAS ADAIOS PSHSTK ADABAS I/O PROCESSING 2.73

 

Reduce CPU Time and Duration - Solutions

04/22/23 | Software AG User Group 2011 | 19

Prod DB

ASSO DATA WORK

Reptor

SLOG

LBP= ++LFIOP= --LRPL= ASYTVS=YESFMXIO=8

Reptor

SLOG

Reptor

SLOG

Broker

Important Parameters

- LBP and LFIOP

- LRPL

- ADADBS DSBI=OFF

- ASYTVS=YES and FMXIO= (6-8, max 16) => SLOG

- LOGINPUTTRANS= (50-70)

04/22/23 | Software AG User Group 2011 | 20

Challenge with Initial-State

04/22/23 | Software AG User Group 2011 | 21

Source DB

ASSO DATA WORK

Reptor

SLOG

SYS

A file with 160 million ISNs takes too long using RPL’s initial-state function

Target DB

ASSO DATA WORK

Broker Network

sFirewallsRouters

160 million READs

+ 160 million STOREs + 160 million ETs

# Files

Adabas Commands/Sec Source / Reptor

160 Millions ISNs

1 990 cps / 438.60 cps 101.24 hours = 4.2 days

7 1,253 cps / 308.57 cps

144.03 hours = 6.0 days

Challenge with Initial-State

04/22/23 | Software AG User Group 2011 | 22

Source DB

ASSO DATA WORK

Reptor

SLOG

SYS

Solution 1:

Target DB

ASSO DATA WORK

Broker Network

sFirewallsRouters

Decompresss

FTP Compresss

Load

Challenge with Initial-State

04/22/23 | Software AG User Group 2011 | 23

Solution 2:

ADAMAGIC

Restore ADASAV backup from the mainframe to Windows or UNIX

Why not from SAG?

Efficient Usage of Adabas

Replication

- A Practical Solution Finder –

By Dieter W. Storr

ISBN 978-3-8348-1730-3

04/22/23 | Software AG User Group 2011 | 24

• 11 Chapters• 262 Pages• 221 Figures

• Natural Monitor Programs• Assembler Programs

Thank You!