ASGTMON PRODUCTS DATA EXTERNALIZATION...•Enables third party access to TMON monitoring products on...

© 2019 ASGAll rights reserved

ASG TMON PRODUCTS DATA EXTERNALIZATION

TECHNICAL OVERVIEW:

ASG-TMON Z/OS PLATFORM MONITOR API’S, EXPORT FEATURES AND DATA EXTRACT

BATCH UTILITY ASG-TMON PERFORMANCE ANALYZER API AND EXTRACT LOAD UTILITY, EXPORT FEATURE

JULY 2019

PERFORMANCEMANAGEMENT

ASG-TMON Z/OS PLATFORM MONITORING SUITEASG-TMON FOR Z/OS, CICS, DB2, IMS, MQ, TCP/IP


TMON MF DATA EXTERNALIZATIONOVERVIEW

• Enables third party access to TMON monitoring products on z/OS real-time performance data metrics

• For the purpose of…– Improving z/OS systems and business application availability– Viewing End to End z/OS Performance metrics: z/OS, CICS, DB2, IMS, MQ, TCP/IP– Feeding performance metrics to applications supporting business critical value streams

§ Dashboards and Reporting§ Artificial Intelligence/Machine Learning products§ Adhoc analysis§ Actionable performance issues requiring event notification and automated actions

• TMON provides…– RESTful API on MS Windows– API on z/OS– Batch data extract utility– Export features in ASG-NaviPlex


• TMON RESTful API supports extraction of real-time data from z/OS TMON products• ASG-TMON products produce 170+ “aggregates” of data available to be externalized• Capture TMON performance data and exceptions in real-time• The API is an application utilizing MS Windows .net• Documented via SWAGGER ODATA API documentation and doc on ASG.COM support site• Data output in JSON format, making it easily digestible by other applications• Secured via Windows Active Directory, HTTPS and z/OS RACF/ACF2/TOP SECRET credentials• FREE to TMON MF monitor customers

• Usage…– Collect data and write it to a file or database for use by your or other 3rd party applications – Take automated actions using bots– Display data in Dashboards– Queries and Performance Analysis– Custom reporting– Integration with 3rd party products… AI/ML products: SPUNK, ELK, SPARK…

TMON MF RESTFUL API ON MS WINDOWS– REAL-TIME

Manual: ASG-NaviPlex™ Aggregates Data Guide


SWAGGER – HOW DOES IT WORK?

List of data sources, select one


Example: CICS REGION Metrics

SWAGGER


Enter parms and click “try it out”

SWAGGER


Results

SWAGGER


Roadmap

The next release of this API…

• Will provide the capability to log the data aggregates to an SQL database (PostgreSQL) non MF

• We will provide controls to filter the TMON address space data sources so that customers can gather only the data from the sources they desire vs. all TMON address spaces

• We will provide an API to read TMON Log Files (LFS) data aggregates

• Future: We will research ways to create/access summarized TMON report data – TMON Report Writer produces valuable reports that customers use vs. the raw LFS data– We will determine how to best leverage the code and data to externalize it

• Future: We will create OOTB samples to simplify integration with popular third party products



• Documented as the NAVIDATA function supports extraction of real-time data from z/OS TMON products using REXX programming or other languages

• NAVIDATA is a z/OS based API, designed to run on z/OS

• Capture TMON data in real-time on z/OS

• Usage…– Export data to a file or database for use by your or other 3rd party applications – Collect data in spreadsheets for analysis– Display data in Dashboards– Create custom 3270 screen using ISPF Dialogue Manager– Queries and Performance Analysis– Custom reporting– Integration with 3rd party products… AI/ML products: SPUNK, ELK, SPARK…

TMON MF API ON Z/OS – REAL-TIME



How does it work?– ASG-TMON products produce 170+ “aggregates” of data available to be

externalized– Select data using SQL like language (TQL – TMON Query Language)– After retrieving a response from a TQL_string request, the NAVIDATA function

populates REXX variables. – Example…

[ SET SOURCE source-identifier [, .....] ; ]SELECT select-expression [AS alias] [, .....]FROM product.aggregate[ WHERE search-condition][ GROUP BY group-expression [, .....]][ ORDER BY order-expression [, .....]]

NAVIDATA REXX FUNCTION – REAL-TIME



SAMPLE 3270 ISPF DIALOGUE PROVIDED

CPU Busy /Paging/ CICS Response Time panel



• Data Extract allows you to select data elements from the TMON log records and output them to a file in a generalized format (flat file or .CSV) via a batch job on z/OS.

• Data can be imported to another software package (e.g. 3rd party reporting tools, databases, spreadsheets, AI/ML applications…)

• This batch service permits the selection, sorting, and filtering of TMON LOG data (LFS) to create data in normalized form, without the repeating segments of LFS data. The normalized data structure is referred to as an aggregate.

DATA EXTRACT UTILITY ON Z/OS – HISTORICAL

Manual: ASG-TMON® Products for z/OS TMON Log File Services User’s Guide


ASG-NAVIPLEX DATA GRAPHING/EXPORT/ACTION FEATURESASG-NaviPlex provides an Enterprise View of all TMON product alerts and real-time data metrics for graphical and tabular reporting

Manual: ASG-NaviPlex™ User’s Guide


ASG-NAVIPLEX DATA GRAPHING/EXPORT/ACTION FEATURES


For example…1. You can create

an exception expression

2. Capture the error and associated data

3. Export it in the available formats


– Export data to a file or database for use by other 3rd party applications§ Reporting – batch and web reporting tools§ Queries for Performance Analysis§ Spreadsheets§ Dashboards§ AI/ML products

– Capture Exceptions and send an email, WTO, SNMP trap§ Include graphics in the email§ Notify consoles: Tivoli, HP OpenView, NetView…§ Open Service Desk Tickets automatically§ Post charts to a web page so you have a historical representation of exceptions

viewable via a web browser– So when your boss calls, you can say …

“I knew that, I’m working on it”

ASG-NAVIPLEX EXPORT USAGE


ASG-TMON PERFORMANCE ANALYZER

ASG-TMON PA FOR

Z/OS, CICS, DB2,

WINDOWS, LINUX, VMWARE, JAVA APPLICATION SERVERS,

ORACLE DB, SAP SYBASE ASE DB, MICROSOFT SQL SERVER DB


TMON PA DATA EXTERNALIZATIONOVERVIEW

• Enables third party access to TMON Performance Analyzer (PA) database performance management

• For the purpose of…– Improving IT systems and business application availability– Viewing End to End Performance metrics: z/OS, CICS, DB2, Windows, Linux, Vmware, Java application

servers, Oracle db, SAP Sybase ASE db, Microsoft SQL Server db– Feeding performance metrics to applications supporting business critical value streams

§ Dashboards and Reporting§ Artificial Intelligence/Machine Learning products§ Adhoc analysis§ Actionable performance issues requiring event notification and automated actions

• TMON PA provides…– RESTful API on MS Windows– Windows Command line data extract and load utility– TMON PA Server Administrator EXPORT feature


• TMON RESTful API supports extraction of data from the TMON PA products database• Get TMON PA performance data • The API is an application utilizing MS Windows .net• Documented via SWAGGER ODATA API documentation and doc on ASG.COM support site• Data output in JSON format, making it easily digestible by other applications• Secured via TMON PA Portal, HTTPS credentials• FREE to TMON PA customers

• Usage…– Collect data and write it to a file or database for use by your or other 3rd party applications – Take automated actions using bots– Display data in Dashboards– Queries and Performance Analysis– Custom reporting– Integration with 3rd party products… AI/ML products: SPUNK, ELK, SPARK…

TMON PA RESTFUL API ON MS WINDOWS– REAL-TIME



API Categories:

• Convenience APIs: Provide information that aid the exploration and use of the APIs

• Supporting APIs: These provide information about the configuration and collection of performance information being performed by ASG-TMON PA. Supporting APIs can be used to list platforms, systems, groups, periods, intervals, etc.

• Data Query APIs: Which provide actual data values that have been collected and reside within the ASG-TMON PA repository

HOW DOES IT WORK?


Supporting APIs:

• There are a number of APIs that provide information about the configuration and collection of performance data being performed by ASG-TMON PA.– groups - returns a list of service groups as defined to TMON PA– groupplatforms - returns a list of platforms for each defined service group in TMON PA– platforms - returns all valid Platforms with data in TMON PA– systems - returns a list of systems and the platforms that they are associated with– (plat)systems - return a list of all systems for the specified platform.– dayintvls - Returns intervals for Daily pmd identified for Portal (PvDaily)– weekintvls - Returns intervals for Weekly pmd identified for Portal (PvWeekly)– monthintvls - Returns intervals for Monthly pmd identified for Portal (PvMonthly)– dayperiods - Returns periods for Daily pmd identified for Portal (PvDaily)– weekperiods - Returns periods for Weekly pmd identified for Portal (PvWeekly)– monthperiods - Returns periods for Monthly pmd identified for Portal (PvMonthly)– timetypes - Returns a complete list of valid timetypes.

HOW DOES IT WORK?


Data Query API

• A native capability of OData is the ability to filter rows based on values in columns.

• All OData APIs that return data values (ex: wincpu, windisk) will have the same columns as highlighted here:

HOW DOES IT WORK?


JSON OUTPUT FORMAT EXAMPLE


Roadmap

• The next release of the Data Query API will provide the capability to retrieve multiple metrics in a single call from a TMON PA DB record

• Secured via Windows Active Directory

• In the future we will create OOTB samples to simplify integration with popular third party products



• The TMON PA ETL tool extracts the Performance Data from TMON PA’s PMD-based database to a relational database, which is designed to target applications that analyze and report on data warehouses.

• TMON PA ETL is a command line interface based tool for extracting performance data collected by TMON PA from its proprietary database and loading it to a relational database.

• ETL Data Load Configuration file is an XML file that specifies the various Metrics (Facts) that you need to extract into the target database.

• You can use the Command Line interface of the ETL tool to perform these tasks: – Scheduled Execution using Windows Scheduler—This method is suitable for unattended extraction and loading of

performance data at scheduled time(s) of the day on a regular basis. – Adhoc Execution using Command Line Interface—This method is suitable for a one-time adhoc extraction and

loading of performance data. – Can process multiple input files– Select and load specific time periods– Limit the data extraction to specific systems – Update and Purge data

TMON PA EXTRACT AND LOAD AKA ETL

ASG-TMON PA User’s Guide


This diagram illustrates the schema for the tables representing the dimensions from the performance data extracted into the database

TMON PA EXTRACT AND LOAD AKA ETL



• TMON PA gives you the ability to export data for use in other applications.

• The TMON PA Server Administrator offers a convenient, GUI-driven method of extracting data from the detail database. You can perform this operation from the Export Data tab.

• Files can be saved in these formats: – CSV—Comma Separated Values – TSV—Tab Separated Values – TXT—Text Files

• A command-line interface is also available for exporting data from the detail and summary databases.

• The TMON PA Export Utility (ISMNTEX.exe) allows you to export detail or summary data through a command-line interface. It provides an excellent means of automating exports because it can be run through a batch job.

TMON PA EXPORT



TMON PA EXPORT VIA SERVER ADMINISTRATOR


The Export Data Tab:• The most convenient

way to export data from the detail database is to use the Export Data tab in TMON PA Server Administrator

THANK YOU

Date post:	20-May-2020
Category:	Documents
Upload:	others
View:	4 times
Download:	0 times

ASGTMON PRODUCTS DATA EXTERNALIZATION...•Enables third party access to TMON monitoring products on...

Documents