Basic Workload Automation

7/28/2019 Basic Workload Automation

http://slidepdf.com/reader/full/basic-workload-automation 1/65

BasicCA-ESP Workload Automation

Course – Agent Overview



Copyright ©2006 CA. All rights reserved. All trademarks, trade names, services marks and logos referenced herein belong to their respective companies.2

What You Will Learn

- What is an ESP System Agent- Basic Agent Architecture

- How ESP System Agents Interact With CA ESP

Workload Automation Engine

- Basic Configuration Options

- Defining non-mainframe workload

- Value that ESP Agents can provide

- ESP Agent for Databases




Product Components




ESP Agents

- Lean, non-invasive conduits (Approx 50MB disk space)

- Extends CA ESP Workload Automation‟s solutions across a

variety of operating systems and ERP environments

- Unlimited scalability, throughput

- Each platform has unique ESP Agent

- Manage through a single point of control, will be integrated

with CA Job Management products (CA-7, CA-Autosys)




ESP System Agents - Scalability

- Volumes and Concurrency

- Large number of concurrent processes

- Lab tested at 1,000 concurrent jobs, largest known field

implementation 500 concurrent jobs

- Highly scalable

- Can keep up with volumes manager can handle




What is an ESP Agent?

- Processes work

- Notifies Manager of job status

- Started

- Running

- Complete/Failed

- Stores output from jobs in a spool file

- Allows users to control workload

- Waits for work

- Persistence


http://slidepdf.com/reader/full/basic-workload-automation 7/65Copyright ©2006 CA. All rights reserved. All trademarks, trade names, services marks and logos referenced herein belong to their respective companies.7



Workload Objects

Distributed OS Integration

Agent Monitoring

OS Resource Integration

FileManagement

Web App Integration

ERP Integration

Database Integration

Mainframe Integration



Who Controls the ESP Agent?

- Schedulers- Schedule jobs regardless of platform & Have

control over entire enterprise

- Operations

- Monitor entire enterprise & Control every

production job

- System Administrators

- Install ESP Agent & Maintain ESP Agent



ESP System Agent Architecture

- ESP System Agents for UNIX Based Platforms are started

processes.

- ESP System Agents for Microsoft Windows are installed as

Windows Services

- All ESP System Agents Utilize SUN Microsystems JAVA

Runtime Environment as the Base Architectural Component

- All ESP System Agent Functions are Created as JAVA Plug-

In‟s



ESP System Agent Architecture

Job Execution

JAVA Plug-In

File

Monitoring

JAVA Plug-In

Object

Monitoring

JAVA Plug-In

J2EE

Execution

JAVA Plug-IN

Agent

Management

JAVA Plug-In

SUN Java Runtime Environment Base Code

Spawns &

Tracks

Submitted

Commands

& Scripts

Processes &

Monitors File

Trigger &

FTP

Requests

Processes

Machine

Resource

Monitoring

Requests

Processes

EJB & JMS

Publish

Subscribe

Requests

Processes

TCP/IP

Communications

& SNMP

Requests



Communication

- ESP Managers and ESP System Agentscommunicate asynchronously using message

queues.

- ESP Managers and Agents communicate by

sending Automated Framework Messages or AFMs.- Most Agent commands deal with the control of

these AFMs.



Communication

- ESP Managers and ESP System Agents haveSender and Receiver Ports.

- The receiver listens on a predefined TCP/IP port.

- When the sender has messages to transmit, it connects

to the receiver‟s port, sends the messages, and thencloses the connection.



Communication

- ESP Managers and Agents have sender andreceiver ports.

- Each Agent has one dynamic sender port and

one receiver port.

- The ESP Host can have multiple receiver ports (forexample, to separate encrypted and non-encrypted

message traffic) and has one dynamic sender port for

each connected Agent.



ESP Agent Basic Communications



Communication

- Automated Framework Messages include:- ESP System Agent Name

- Command or Script to Execute

- UserId to Execute the Workload Under

- Arguments for the Script or Command

- Environment Variables

- Job Execution Status




Communications

- Sample AFM Messages:20050908 12060405+0500 CM_DE54 DLEIGH_WIN_AGENT

WINNT1/WLMDEMO5.20/MAIN State EXEC SetStart Status(Executing at

DLEIGH_WIN_AGENT) Jobno(1860) User(DLEIGH_WIN_AGENT) Host(DLeigh)

20050908 12063416+0500 CM_DE54 DLEIGH_WIN_AGENT

WINNT1/WLMDEMO5.20/MAIN State COMPLETE Cmpc(0) SetEnd

User(DLEIGH_WIN_AGENT) Host(DLeigh)

20050908 12290376+0500 CM_DE54 DLEIGH_WIN_AGENT

CYBDL01K/WLMDEMO3.3/MAIN State SUBERROR Failed SetEndStatus(Command file not found) Cmpc(20007) User(DLEIGH_WIN_AGENT)

Host(DLeigh)




Basic Configuration

- All ESP Agent Configuration Settings are stored ina single file called agentparm.txt

- This file is created by the installation process andcan be manipulated using any text editor

- This Configuration File Defines the Name of the

Agent, Ports Used, and Other Configurations thatPertain to Other Functions (FTP, J2EE, SNMP, etc.)




Basic Configuration

- Sample agentparm.txt File:# ESP System Agent for Microsoft Windows parameters

#

# Log

#

log.level=5

log.maxsize=1024000

## Agent name

#

agentname=DLEIGH_WIN_AGENT

#

# Communications

#

communication.managerid_1=CM_DE54

communication.manageraddress_1=lparc

communication.managerport_1=6666

communication.monitorobject_1=DLEIGH_WIN_AGENT/AGENTMON1.0/MAIN

communication.managerid_2=ESPRESSO_CALYPSO_47500

communication.manageraddress_2=calypso






Basic Configuration

- Sample agentparm.txt File:

communication.managerid_3=ESPRESSO_ELVIS_7500

communication.manageraddress_3=elvis



communication.inputport=9900

communication.prefixlevel=2#

# Security

#

security.filename=D:/Cybermation/ESP System Agent R6.1/security.txt

security.keystorage=D:/Cybermation/ESP System Agent R6.1/keys.txt

security.cryptkey=0x3132333435363738

security.level=off #

# Initiators

# There will be separate line for each pair of <class, number of initiators>

# initiators.class_N, where N is number running from 1 to ...

# Class is user definable, (should be the same here and in AFM)

# Soft shutdown mode waits for all jobs to be completed

#

initiators.class_1=Default,1000




Basic Configuration

- Sample agentparm.txt File:# Persistence

#

persistence.gcinterval=10000

persistence.level=2

#

# Core parameters (for plugins)

#

core.address=localhost

core.port=35800

#

# General characteristics

#

spooldir=./spool

COLD_START=false

## Runner plugin parameters

#

runnerplugin.managerport=35801

runnerplugin.requestport=35802

#

# Start JVMs

#

oscomponent.attachjvm=true




Basic Configuration

-Sample agentparm.txt File:

#Path to the JRE

oscomponent.javapath=D:/Cybermation/ESP System Agent R6.1/jre/bin

#type of jvm (used only if attachjvm=true)

oscomponent.jvm=client

#What plugins we want to start by the core Java agent

plugins.start_internal_1=runner

plugins.start_internal_2=fileMonplugins.start_internal_3=objmon

plugins.start_internal_4=management

plugins.start_internal_5=ftp

plugins.start_internal_6=j2ee

plugins.start_internal_7=router

#

# SNMP specific

#

management.snmp.mibfile=D:/Cybermation/ESP System Agent R6.1/cybermib.txt

management.snmp.host=DLeigh

management.snmp.port=162

management.snmp.community=public




Basic Configuration

- Sample agentparm.txt File:

#

# FTP specific

#

# ftp.noserver=false

ftp.serverport=21

#ftp.client.ssl=true

#ftp.client.ssl.truststore=

#ftp.client.ssl.truststore.password=055A55EB863D2A5D

#ftp.server.ssl=true

#ftp.server.ssl.keystore=

#ftp.server.ssl.keystore.password=C8B98BFA6652520BC0#

# Windows Service name

#

oscomponent.servicename=ESP System Agent for Microsoft Windows R6.1

oscomponent.servicedisplayname=ESP System Agent for Microsoft Windows R6.1




Successful Communication dependson…

Agent Information that must match…

Agentparm.txt Topology in ESP dSeries

agentname Name

communication.inputport Agent Port number communication.managerid Manager I nstance Host name

commun ication.manageraddress DNS name or IP Address of dSeries Server

coomunication.managerport ESP Server Manager Port

security.cryptkey Encryption key used from server to agent




Test Your Memory

- What Type of Workload Can an ESP System Agent

Execute?

- How Does the ESP System Agent Communicate

with an ESP Server?

Windows Scripts, Windows Commands, FTP Processes,

Machine Resource Monitors, J2EE Based Processes,

etc…

Utilizing Automated Framework Messages over TCP/IP




Test Your Memory

- What is the Receiver Port Used for?

- What Job States Can the ESP System Agent for

Windows Send to an ESP Server?

The receiver listens on a predefined TCP/IP port. When the

sender has messages to transmit, it connects to the

receiver’s port, sends the messages, and then closes the

connection.

READY, EXEC, COMPLETE, FAIL, SUBERROR, &

SUBDELAY



Additional Automationcapabilities

h f




Approaches for use in yourenvironment

- Event-Driven Workload Automation

- “Batch” Environment Monitoring

- Run Book Automation




Event-Driven Automation

- Trigger off new applications based on monitoring capabilities

of the R7 Agent, using ESP Alerts:

- Error message being written to a log file

- A service coming online

- Server‟s CPU stuck at 100%

- ESP:dSeries currently implemented as Jobs, also several

monitors implemented as Event Triggers




“Batch” Environment Monitoring

- The ability to monitor critical environmental dependencies of

an ESP Application to ensure successful completion

- Example: Find out you are going to run out of disk space

hours before the server dies

- Example: If you know you‟re going to start running a jobon your SAP server at 2am, and the database server for

that job crashes at 10pm, why wait 4 hours for everyone

to be surprised?

- (and getting called in the middle of the night)

- Simply another step towards better managed SLA‟s




Run Book Automation

- Latest fad in IT Operations that many

of you, have been doing for years

- Focus on managing (executing,

controlling, monitoring) IT Operations

tasks in a controlled, workflow-style

manner much like workload

automation

- Example: Shutting down certain

processes on a server, running

maintenance, starting back up the

processes

- You can, naturally, do much of this

with your existing ESP Scheduler and

ESP Agents



FTP Workload Objects




Automating File Transfers

- To schedule FTP workload, use the FTP_JOB job type.

- You can automate file transfer with an FTP job. The job canuse an existing FTP server or the Agent‟s in-built FTP server.

- Note: To use the Agent as an FTP server, you need toconfigure the Agent during installation or set the Agent

parameter ftp.noserver to false. The Agent configured as anFTP server does not support anonymous file transfers




Automating File Transfers

- In this sample, a file called /temp/cyberftp181006.txt is

downloaded from a UNIX machine (rem_unix) then copied to alocal machine, a Windows PC. Note that the two locations includea complete path statement. After the download is complete, the

job completes in ESP dSeries:



Properties for „Monitor‟ Workload Objects




Adding Monitoring Capabilities to ESP

- Starting with Release 6 of the ESP System Agent,

new monitoring capabilities were introduced:- Windows

- Windows Event Log Monitoring

- Windows Services Monitoring

- Windows/UNIX/OS400- Disk Space Monitoring

- Process Monitoring

- TCP/IP Address/Port Monitoring

- CPU Monitoring

- Text File Monitoring

- J2EE- JMS Publish/Subscribe to Queues and Topics




Windows Services

- What is it?

- This job type allows you to monitor Windows

Services on a local machine

- Sample Definition:




Windows Services

- Service Name corresponds to

the name of the Service as

identified in the Services

Application




Windows Services

- Status refers to the state you with the Service to be in for themonitor to complete

- Valid Statuses include: RUNNING, STOPPED,

CONTINUE_PENDING, PAUSE_PENDING, START_PENDING,STOP_PENDING, EXISTS, NOTEXISTS

- Can be coded with either WAIT or NOW. Now will return aCOMPLETE/FAIL immediately, while WAIT will remain until thecondition is met or the JOB is forced complete




Windows Event Log

- What is it?

- A job to monitor the Event Log of a Local

Windows Server. It can monitor any one of three

types of logs

- Application Log (Programs)

- System Log (System Components, e.g. a Driver)

- Security Log (Security Events like an invalidlogin, file access)




Windows Event Log




Windows Event Log

- Sample Job Definition




Windows Event Log

- EVENTLOG values are generally Application, Security, or System

- EVENTTYPE values may be ERROR, WARN, INFO, AUDITS,AUDITF (AUDITS and AUDITF related to EVENTLOG Security only)

- EVENTSOURCE value is typically generated by software vendor.Values with spaces requires quotes

- EVENTCATEGORY represents a classification by theEVENTSOURCE, in this case, „Norton AntiVirus‟ is catagorizing thisas a Disk event.




Disk Space

- What is it?

- Allows you to find out how much disk space has been used

or is free

- Can be expressed in MB, GB, or as a Percentage

- Can be CONTINUOUS monitored using an ESP Alert




IP Monitoring- What is it?

- It allows you to monitor specific IP addresses or IP Address/Portcombinations to validate that network resources are accessible

and listening ports are available

Valid Status include RUNNING and STOPPED and are monitored for

immediate state NOW or monitored until condition is met WAIT




CPU Monitoring

- What is it?

- Allows you to monitor CPU utilization to determine

success/failure criteria of your job monitor

- Why would I want to do this?

- Provides you with a means of ensuring that the machine has

sufficient CPU available before submitting workload to it- If a machine is in a continuously busy state, allows you to fire off

notifications

- CPU Monitoring is also tied into Physical Resources, which can be

used for physical load balancing (required HPO for ESP:mSeries)

- How is CPU utilization measured?

- Windows uses CPU as recorded by the system

- UNIX uses Load Average




CPU Monitoring

- This example completes if the CPU Usage is greater than

80%.

- Can be used to send warnings to System Administrators.




Text File Monitoring

- Allows you to monitor INSIDE a text file for a

specific string of text for a matching value

- Monitoring has a great deal of flexibility with

Regular Expressions





- This example monitors the file by lines for a text

string, but only the first 20 lines





Example using Regular Expressions:



Properties for JMSWorkload Objects

- JMS Monitoring




JMS Subscribe

- What is it?

- Java Messaging Service

- A Java-based standard for connecting to, and

communicating with, message brokers such as

MQ Series, Sonic, WebLogic, or Tibco- Allows you to monitor a JMS Queue or Topic for

a specific message that matches your criteria




JMS Subscribe

ESP AgentTopic or

Queue

JMS Provider Receive

Acknowledge

- JMSS_JOB job type

- Subscribe to Topics

or Queues

- Filter results

b b




JMS Subscribe

S S b b




JMS Subscribe

- A few uncommon terms

- INITIAL_CONTEXT

- CONNECTION_FACTORY

- JNDIUSER



ESP Agent forDatabases

ESP D t b A t




ESP Database Agent

- Provides integration to a variety

of database platforms

- Utilizes the agent plug-in

architecture to adapt to the

System Agent core- Written entirely in JAVA and the

JDBC API

ESP D t b A t F t




ESP Database Agent Features

ESP DatabaseAgent

R e s ul t s

S Q L

ESP DatabaseAgent

SP

SP

ESP DatabaseAgent

- +

ESP DatabaseAgent

TRG

TRG

E v e

n t s

SQL Updates

and Queries

Stored

ProceduresTable Monitor Data Triggers

R

e s ul t s

P a r a m e t e r s

E t D i SQL St t t




- Event-driven execution of SQLcommands i.e. Insert, update,delete, etc.

- Success criteria specified throughregular expressions

- Output stored to file

ESP Agent for

Databases

S QL

R e s u l t s

Event Driven SQL Statements

I ki St d P d




- Event-driven execution of stored

procedures

- Retrieve output values generated by

the stored procedure

ESP Agent for

Databases

SP

SP

Invoking Stored Procedures

Monitor Database to Trigger an




- Monitor a table for a net change insize

- i.e. number records added or deleted

which satisfy a user defined condition

- Complete or selected (columns) records

can be defined

- One-time or continuous alerts

ESP Agent for

Databases

- +

Monitor Database to Trigger anESP Event

D t b T i




- Very granular notification of changes to adatabase table

- Database generates the events as they

occur

- Excellent performance, with very lowoverhead

- Generate an event for every record added,

deleted, or modified which satisfies a user

defined criteria

- One-time or continuous alerts

ESP Agent for

Databases

TRG

TRG

E v e n t s

Database Triggers

Database Agent Technical




- Supported databases

- Oracle 9i & up

- Microsoft SQL Server 2000 & up

- IBM DB2

gDetails

D t b A t E l




Database Agent Examples

- This example uses SQL to Insert a line into a Table

called „stores‟ in the „pubs‟ database



Questions?

Date post:	03-Apr-2018
Category:	Documents
Upload:	mskchaitanya
View:	222 times
Download:	0 times

Basic Workload Automation

Documents