ij r r 1< " !1 ,- } ,- , ;' i t: ;. f k. if l" F l' ), I; ,', j- L r. k " ],) t li 0. f . Ii .f- 1, &; t , i1: r " fi:.' i . :' " • .t. l: Ie \ f \, f :F t {T ".F t,- B :g t" '" ;: --- ABSTRACT USER EXPERIENCE IN DESIGNING AND IMPLEMENTING A CICS"I'M PERFORMANCE MONITORING APPLICATION USING THE SASH SYSTEM Erciiment fiLGfiNER AKNET INC. This paper describes an application developed for measuring and monitoring the CICS subsystem performance using Base SAS system along with SAS/GRAPH R software and covers how the data access, management, analysis and presentation techniques in the SAS system are applied. The main purpose of this paper is to acquaint you with practical user experiences in designing and implementing a multi-purpose application by using a single software system, that is to say the SAS system. So, this paper mainly deals with the techniques used for accessing, managing and analysing data, rather than programming details. Also, the interpretation 'and the usage of outputs will be discussed. This paper addresses the following topics - introduction - data access - data storage and management data analysis - reports and their usage (sample graphs) conclusion First, the data processing environment and the purpose in developing such an application will be introduced. In the following sections, data access, data storage and management and data analysis functions are going to be discussed. Before completing, the interpretation and usage of outputs will be illustrated with sample graphics. All the numbers and the graphics given in this paper are only samples and do not represent the actual cases . 683 - ," ;-., -" -.'.' " -'". . ."., -.; - - - "-" -- I- i ;: .... , ..
Transcript
~,
ij r ~.
r 1<
,~ "
!1 ,-
~ ~"
} ~"
~ ,-~'. , ~'
;' ~ ~.
i
t: i.~
;.
f
k. if l" F l' ), I; ,', j-
L r.
t~· k
~: " ~ ],) ~, ~,
t li 0.
f j~ . Ii ~' .f-1,
&; t , i1: r " fi:.'
i . :' " • .t.
l: ~, t'~
Ie ~ ~~ \ f \,
f ~, :F
t '{T ".F t,-
'B :g t" '" ;: ---
ABSTRACT
USER EXPERIENCE IN DESIGNING AND IMPLEMENTING A CICS"I'M PERFORMANCE MONITORING APPLICATION
USING THE SASH SYSTEM
Erciiment fiLGfiNER AKNET INC.
This paper describes an application developed for measuring and monitoring the CICS subsystem performance using Base SAS system along with SAS/GRAPHR software and covers how the data access, management, analysis and presentation techniques in the SAS system are applied.
The main purpose of this paper is to acquaint you with practical user experiences in designing and implementing a multi-purpose application by using a single software system, that is to say the SAS system. So, this paper mainly deals with the techniques used for accessing, managing and analysing data, rather than programming details. Also, the interpretation 'and the usage of outputs will be discussed.
This paper addresses the following topics
- introduction - data access - data storage and management
data analysis - reports and their usage
(sample graphs) conclusion
First, the data processing environment and the purpose in developing such an application will be introduced. In the following sections, data access, data storage and management and data analysis functions are going to be discussed. Before completing, the interpretation and usage of outputs will be illustrated with sample graphics. All the numbers and the graphics given in this paper are only samples and do not represent the actual cases .
CICS (Customer Information Control System) application that we are going to focus on Capacity Planning Department of AKNET.
performance monitoring is developed by the
AKNET provides data processing services for almost all modern banking applications to AKBANK which is the largest private bank in Turkiye. These services include business and technical consultancy, application development, facilities management, operations, technical support and maintenance for a variety of hardware and software platforms. AKNET performs its activities in two main offices in istanbul and seven regional field engineering centers distributed throughout the country with 224 personnel.
AKBANK has nearly 580 on-line branches with nearly 8000 teller terminals, 2000 point of sale terminals, 300 automated teller machines (ATM) and several departmental systems distributed throughout the country. These are interconnected over a Wide Area Network operating under IBM's SNA™ (Systems Network Architecture) architecture. There are also connections to external banking networks such as SWIFT and EFT (Electronic Funds Transfer), and beside these, inter-bank clearing network and inter-bank ATM sharing network connections are in development. Data Processing Center includes two IBM 9021 Model 580 host computers, one of them used for production and the other is used for backup and test purposes. These two machines operate under IBM's MVS/XA™ operating system.
Transactions initiated from the on-line branches of the bank are processed in several CICS regions. So, obtaining the performance related information about the CICS regions have great importance in our current installation. On-line workload of the bank is approximately 50 transactions per second.
From another point of view, it would be useful to think of the CICS performance monitoring application as a contributary application of a company-wide methodology, that is to say the Capacity Planning and Performance Measurement Methodology. This methodology is composed of workload characterization and quantification activities, performance measurement and monitoring applications, forecasting, long term planning and cost justification activities. So, CICS performance monitoring application plays an important role in this complete picture.
In order to explain the reasons behind developing this application, firstly it should be noted that, knowing the current situation in a CICS subsystem gives us a chance to hold the business requirements on target. By the use of the CICS performance monitoring application, it is possible to keep track of the on-line workload level. Also, by analysing the response times of transactions, it is possible to have an idea about customer satisfaction.
Second, it seems to be important to establish a global database which could be used to develop business strategy and long term plans. CICS performance monitoring application gives us the possibility to hold the historical data in an integrated way so that efficient and relatively accurate planning functions can be realized.
Lastly, one of the most important leading trend in today's data processing industry is to reduce the costs. It is obvious that, knowing the current situation and ability to predict the future with the previously established targets on hand, it is possible to create timely and cost effective plans.
So, the main purpose of the CICS performance monitoring application is to create an integrated system to satisfy all those requirements.
DATA ACCESS
Main data sources are Candle Corporation's OMEGAMON/CICS™ and IBM's SMF'I'M (System Measurement Facility) products.
OMEGAMON/CICS is a complete product which is used for the aim of real-time performance monitoring activities and it stores the detailed transaction records in its own compressed format. IBM's SMF product collects and records host system related information. This information also includes the performance related variables about the CICS regions.
As software tools, Base SAS System and MXGR (Merrill's Expanded Guide) software are used.
There is an INFILE EXIT which is supplied by CANDLE corporation and MXG software makes use of this INFILE EXIT in order to process the compressed OMEGAMON/CICS records. Access to SMF records is also accomplished by MXG software. MXG decodes the SMF records and stores them in the SAS system in the form of Performance Data Base (PDB).
DATA STORAGE AND MANAGEMENT
Different banking applications run in different CICS regions. OMEGAMON/CICS product gathers transaction-by-transaction records for each one of these CICS regions. As it was stated earlier, vast amount of transactions are initiated from the on-line branches of the bank and processed in the CICS subsystem. So, in order to handle this amount of data, a summarization process has to be applied.
685
J _ ~ _ _ ~ _ __ _ __
,", ~''" --~ .. '~'- --. ,:. ---'. '-
f ..... f. I ..
First, a daily process has been applied to OMEGAMON/CICS records and MXG software store the data into a temporary SAS file with 135 variables. Then, by using this temporary SAS file, transaction records are summarized and stored into a permanent hourly based SAS file along with 43 variables. A list of those variables are provided in APPENDIX A, with the formats and the labels.
Before discussing what is meant by hourly based SAS file, be useful to look at the information contained in variables. It is possible to divide the variables categories as follows:
- variables providing environmental information
it might those 43
into three
(such as, system identification code, terminal identification number where the transactions are initiated from, CICS region name, transaction identification code, etc. )
- variables providing response time information
- other performance related variables providing information about the resource consumption of a transaction. (such as, CPU time interval counts, I/O request and file control request counts for each transaction, etc.)
In the data analysis section, those variables are going to be investigated in detail, so at that point, it is time to discuss what is meant by summarization process and how MXG software produces hourly based SAS file from transaction-by-transaction records.
Transactions which start times fall into a specified time interval -let's sayan hour- are summarized for a given combination of transactions. Terminal ID, transaction ID and CICS region name variables are coded in SUM BY statement so that summarization process is applied to the transactions for which SUM BY variables are identical.
So, for a given combination of transactions and for a given time interval
total number of transactions are calculated; percentage of transaction counts for each response time class is calculated;
- maximum value of response time is determined; - resource consumption variables are summed.
By this way, detailed transaction-by-transaction records are summarized into a permanent hourly based SAS file. The main steps in data storage and management process are shown in Figure 1.
686
,.., ...
WEEKLY __ SMF RECORDS
WEEKLY PDS
MONTHLY PDB
TREND PROCESSING
DAILY TRANSACTION RECORDS
r--. __
HOURLY TAANSAC110N RECORDS
....... /
~ ::: r-..;; ~
WEEKLY TRANSACTION RECORDS
" ......
"'" ...... ~ 1'0. ::::: -----I.... MONTHLY
'-----, ___ ,-J TAANSACTION
PROCESSING
...!ECORDS ......
REGULAR REPORTS
FIGURE 1 DATA STORAGE AND MANAGEMENT PROCESS
Transaction information accumulates into the hourly based file during one week time period and by applying weekly summarization process, data is transfered to the weekly based SAS file. Meanwhile, hourly based SAS file is deleted and then redefined.
Data is accumulated into weekly file during one month time period and then summarization process takes place to produce monthly based file. During monthly processing, information about CICS regions start-up and end times are moved into monthly summarized SAS data library. This information is obtained from SMF records. In data analysis section, the usage of CICS regions start-up and end times information is going to be discussed.
Finally, yearly trending process is applied to the accumulated data. This yearly trending SAS Data Base historical information and is generally used for regular activities.
687
monthly provides
reporting
} ,1 DATA ANALYSIS :~§
Data analysis functions in CICS performance monitoring application can be investigated in three separate headlines.
Transaction Volume Analysis Response Time Analysis
- Analysis of Other Performance Related Variables
TRANSACTION VOLUME ANALYSIS
The aim of transaction volume analysis is to have an idea about the workload level for a given period of time. However, in practice, we realized that it would be possible to lose OMEGAMON data while the CICS regions are active. In such a case, obviously we should make an estimation in order to approach the actual workload level.
For this purpose, daily CICS start-up and end times are compared with the time interval for which data is available in OMEGAMON. If there is any time mismatch between these two sources, percentage of time is calculated during which data is not available in OMEGAMON. Then transaction counts are adjusted according to the percent time that data is lost. A schematic representation of the transaction volume estimation process is shown in Figure 2.
CICS REGIONS TIME INTERVAL START-UP AND END FOR WHICH DATA
TIMES FROM SMF IS AVAILABLE IN OMEGAMON
I I
~ ..
IS THERE ANY COMPUTE YES .. PERCENT OF
TIME MISMATCH ... TIME THAT DATA
? DOES NOT EXIST
NO
lor .. :::.::::::". : ....... :::: .. TRANSACTION ADJUST VOLUME TRANSACTION CALCULATED DIRECTLY FROM COUNTS AVAILABLE DATA
FIGURE 2 ESTIMATION OF TRANSACTION VOLUMES
688
In the following table, an example for estimating the transaction volumes is provided with sample data.
Total Prime Hours 200 Hours
CICS Region Total 199:55 Hours Active Hours
Total hours In OMEGAMON During Which Data 190 Hours
Is Available
Percent Of Time 4.9 % That Data Is Lost
Total Transaction Count 6,680,000 In OMEGAMON
Estimated Number Of 7,007,000 Transactions
TABLE 1 An Example of Transaction Volume Estimation
The values given in Table 1 is for a specificCICS region and for one month time period. For a specific month, total prime hours that the bank gives services to the customers is assumed to be 200 hours for 20 working days. Total active hours for this CICS region during the month is computed as 199:55 minutes. Total hours that the data is available in OMEGAMON is 190 hours.
So, as it can be seen, some amount of data has been lost in OMEGAMON for 9 hours and 5 minutes during which the data is not available. This time duration means that 4.9 % of the data is lost. Total number of transactions in OMEGAMON for this month is nearly 6 and a half million and the estimated number of transactions is computed as nearly 7 million transactions. This value is obtained by increasing the total number of transactions by 4.9 percent.
Estimated number of transactions is used for approaching the real workload level. Otherwise it is possible to see misleading fluctuations in the data which cause a bias in statistics. Estimating the transaction volumes is an important concept, because those transaction volume figures are used to predict the future workload level.
1 Responlsle timfe values afrehinvCeIcstigabted in 0drder hto int7dr pret b the ~ overa per ormance 0 t e S su system an to ave an 1 ea a out ! ~ the end-user satisfaction. tI. l"'
~
~.' ~ :;,
" u t' r
~ r
Response time values measured by OMEGAMON transaction execution times in the system and network time. In other words, they are not times.
includes only the does not cover the end-to-end response
In MXG modules, a statement has been coded to calculate the response time of a transaction, by subtracting transaction execution start time from transaction execution end time. And then, transactions are classified according to their response times into 8 different variables. MXG provided response time classes are given below:
Response time < 1 second 1 < Response time < 2 second 2 < Response time < 3 second 3 < Response time < 4 second 4 < Response time < 5 second 5 < Response time < 8 second 8 < Response time < 10 second
- 10 < Response time
Number of transactions are then computed for each one of the response time class along with percentages.
In analysing response times, MXG also provides, sum of transaction response times and maximum response time variables for a given combination of transactions and time interval.
ANALYSIS OF OTHER PERFORMANCE RELATED VARIABLES
As it was stated earlier, performance related variables provide information about the resource comsumption of a transaction.
Among those variables, CPU time interval count measures the CPU consumption of a transaction. I/O requests that are issued during the life time of a transaction is also recorded. Besides these, application programs called by transactions issues several kinds of file control requests. Those requests include, add, browse, delete, get or put commands which require access to the master customer files.
For a given combination of transactions, all those performance related variables are summed for a given time interval. And then, while analyzing data, just simple averages are taken for each kind of transaction so that it is possible to judge the efficiency of the application programs.
Although it is possible to produce different kinds of reports for different purposes in any phase of the data management process, in this paper, only the regular reports and their usage are going to be described.
In our installation, as a reporting period, generally three months is chosen and while preparing the reports, the SAS/GRAPH software is used.
First, 'the report~ng activities related with the transaction volume analysis will be investigated. Transaction volume figures give us an idea about the workload level that we have to process every day. Fortunately, with the use of data management functions of the SAS system, it is possible to hold the historical data in an integrated way so that we can see the trends in the data.
In Figure 3, a graphic representation of daily average transaction rates are given for a specific CICS region.
T R A N
600000
S 500000 A C T I o N
C 8 400000 N T S
DAILY TRANSACTION RATES FOR A SPECIFIC CICS REGION
MONTHS
FIGURE 3 : DAILY AVERAGE TRANSACTION RATES
The graphs similar to the one given in figure 3 shows us the current workload level for a specific CICS region or for a specific workload element and one can interpret the reasons that may cause fluctuations in the workload level. For example, looking at figure 3 it can be seen that there is a sharp decrease in workload level in this CICS region. Shifting some workload elements to another CICS region has caused this reduction.
691
T R A N s A c T I 0 N
C 0 U N T S
2000
1500
1000
500
DAILY AVERAGE TRANSACTION RATES FOR ALL CICS REGIONS
However, seeing the overall workload growth rate is also important. In Figure 4, an illustration of the total daily on-line workload of the bank is provided that covers all the CICS regions. This historical workload growth figure constructs a base to make predictions about the future. Although the forecasting techniques used for predicting the future is beyond the scope of this paper, a graphic representation in Figure 5 is added to show how we make use of the transaction volume analysis in facilitating long term planning activities. The sample graph in Figure 5 is prepared using the SAS/ETS R software.
2000
T R A N S 1500 A c T I 0 N
C 1000 0 U N T S
WORKLOAD FORECAST (IN THOUSANDS)
• . . ......•...•.•.......•............
MONTHS
• • • ACTUAL -- FORECAST ------- LSIS ------- UBIS
FIGURE 5 : PREDICTING THE FUTURE WORKLOAD
692
\
\ ..
As a result, we use the transaction rate statistics performance monitoring application as a starting point in the future.
of CICS planning
One of the most important output of transaction volume analysis is the abend rates. For each CICS region or for each workload element, number of transactions which abended are counted and reported.
In Figure 6, for a specific CICS region, percentages of abended transactions are plotted against time. Since the abend rates are generally very small, logarithmic scale is used while preparing this graph.
It is obvious that if an increase has taken rates, systems programmer and application required to overcome the problem.
place in programmer
the abend action is
I I
ABEND RATES FOR A SPE:-=C:-=IF::-=IC~C~IC-::-S=-=-RE=-G~IO-::-N------------------- -- -----1/100000 -LOGARITHMIC SCALE
I 100000
10000
1000"
10 I I I I ITT I I I I T
MONTHS
693
As a result of the response time analysis , it is possible to create valuable reports providing information about the CICS subsystem performance.
As it was mentioned before, transactions are classified according to their response times into 8 different variables. In Figure 7 a bar chart presentation of percentages for each class of response time is shown. Values are based on a monthly time period for a specific CICS region. Figure 7 illustrates that, 87.7 percent of the transactions that executed during one month are completed in less than 1 second. 3.87 percent of the transactions are completed between 1 to 2 seconds, and so on.
This sample graphic representation is one of the most important graphs that is produced for investigating the overall CICS subsystem performance.
Also, response time analysis gives us important warnings about sucessful we are in satisfying the business requirements. At point, Service Level Agreement concept should be mentioned.
Service Level Agreement is a document which is used for regulating the relationship between the end-users and the provider of the data processing services. From the provider point of view, service level agreement is one of the most important step in preparing timely and cost effective capacity plans.
There should also be negotiations and agreements between the two parties, for the response time requirements. As an example, it is assumed that an agreement has been reached for the ATM application with the end-users, stating that at least 90 % of transactions is to be completed within 1 second.
Figure 8 illustrates this agreement. In this graph, percent of ATM transactions which are completed in less than 1 second are plotted versus time. As far as the response time line is over the 90 % agreement level, we state out that business requirements are met. But what happens when response time line falls under the 90 % level. In such a situation, some tuning activities or the option of revizing the capacity plans are taken into consideration.
This regular response time reporting activities shows us how succesful we are in meeting the business requirements and also helps us to decide whether to tune the system or revise the capacity plans.
PERCENT OF TRANSACTIONS WHICH ARE COMPLETED IN LESS THAN 1 SECOND
100 -90 ~-------------80
------------------------
70
P 60 E R C 50 E
~ 40
30
20
10
0T--r-,r-.--.--.--.-,r-,-~--r_~~--~~ I I I I I
MONTHS
FIGURE B : RESPONSE TIME AGREEMENT
695
;,: ~ l<:_'
~ r i f~ ," f<
~
j:: to:
~ f' t ~ ~ r ~" .. ~.
~ r I I ~ r
!. f: K
Finally, in Figure 9 and 10, sample graphics are provided to illustrate, how the performance related variables are reported. Those kind of reports are useful in controlling the efficiency of the application programs. Especially, when there is a change in an application program, it is useful to control the resource consumption values of the transactions along with their response times, before and after the change is put into production.
CPU CONSUMPTION PER TRANSACTION
21
C 20
P 19 U
T 18 I M E 17 J 16 N T E 15 R V
14-A L
C 13 0 U 12 ~
11
10 ,,---r---r---r----T--~--~--_,--~----~I--~--~
MONTHS
FIGURE" : AVERAGE CPU CONSUMPTION OF A TRANSACTION
AVERAGE I/o REQUEST COUNTS PER TRANSACTION ----I
i
10
I
6 8
R -E 6 Q U E S T
4-C 0 U N T 2
l 0'r----,--...---,---.----T---r---r----.-----.---,r-._---r
MONTHS
----j _______ -'FI~G:.:U:::..R!!:E" 1 0 : AVERAGE I/O REQUESTS ISSUED BY A TRf ,:",:~CnON
696
CONCLUSION
Consequently, as an integrated application, CICS performance monitoring application gives us the complete control over the business, from monitoring the daily operations up to the long term planning. In this paper, we generally- focused on how the operational level decision making process and long term planning functions are facilitated by this application. Besides these functions, it is also possible to create detailed reports for a specific transaction or to design exceptional reports in any phase of the data management process.
So, with these properties, the CICS performance monitoring application requires a single software tool that includes all the data driven functions. The SAS System properly satisfies this requirement by providing data access, management, analysis,and presentation fuctions in an integrated way. Also, in the development and implementation phases of this application, we realized that higher productivity and higher flexibility are obtained with the use of the SAS System.
By using the CICS performance monitoring application, we observed that significant improv~ments have been achieved in ;
- reducing the transaction response times and abend rates,
- increasing the efficiency of the application programs and CICS subsystem internal performance.
If it is required to quantify those l.mprovements, it can be said that 75 % of reduction in transaction abend rates is achieved during one year time period. Also, CICS performance monitoring application produces useful outputs which contribute to the modelling and simulation activities and chargeback system.
SAS, SAS/GRAPH and SAS/ETS are registered trademarks Institute Inc, CARY NCUSA. CICS, SNA, MVS/XA and SMF are trademarks of International Machines Inc. OMEGAMON/CICS is a trademark of Candle Corporation. MXG is a registered trademark of Merrill Consultants.
Data Set Page Size: Number of Data Set Pages: Physical Name: Release Created: Release Last Modified: Created by: Last Modified by: Subextents: Total Blocks Used:
