ORSYP: Getting From CA AutoSys onto Dollar...

White Paper

ORSYP: Getting From CA AutoSys onto Dollar Universe White Paper

White Paper

© ORSYP 2013 ▪: ORSYP: Getting from AutoSys to Dollar Universe 2 / 18

Table of Contents

1. Introduction ............................................................................................ 3

2. AutoSys Core Concepts ........................................................................ 4

3. Dollar Universe Core Concepts ............................................................. 5

3.1 Dollar Universe Object Approach........................................................................ 5

3.2 Terminology ........................................................................................................ 6

3.3 JOBs and BOXes in Dollar Universe .................................................................. 7

3.4 Commands ........................................................................................................ 10

3.5 Predecessors (conditions) ................................................................................ 11

3.6 Scheduling ........................................................................................................ 12

3.7 TASK ................................................................................................................. 13

3.8 Workload Balancing and Priorities .................................................................... 14

4. AutoSys Replacement: ORSYP Project Model ................................... 15

4.1 Environmental Analysis ..................................................................................... 15

4.2 Automation Strategy & Architecture .................................................................. 15

4.3 Software Deployment ........................................................................................ 16

4.4 Applications & Infrastructure Integration ........................................................... 16

4.5 Migration of Job Definitions ............................................................................... 16

4.6 Testing & Verification ........................................................................................ 17

4.7 Production Support ........................................................................................... 17

About ORSYP ..................................................................................................... 18

White Paper


1. Introduction

It is natural to be concerned about the potential risks of moving from one automated scheduling platform to another. The objective of this white paper is to provide CA AutoSys users with some details of how ORSYP Dollar Universe works and how AutoSys capabilities can be mapped to equivalent Dollar Universe functions. This knowledge has been acquired through the numerous successful conversions that ORSYP has already completed.

In addition to mapping some of the features and functions from one product to the other, this report will also explain conceptual differences between the two tools. There is also an overview of the conversion process where ORSYP Professional Services teams can be leveraged. Autosys users can take advantage of conversion tools that ORSYP has developed that can automate the creation of up to 95% of all the objects in Dollar Universe

White Paper


2. AutoSys Core Concepts

Before explaining how users configure and schedule workload with Dollar Universe, it is important to be reminded that AutoSys has three main objects:

1. The job includes: Functional action/script to be executed (script with parameters, SAP job, FTP, …)

A listing of all predecessors, which fall into four different types:

i. Completed: starts when the required job is completed ii. Aborted: starts when the required job is aborted iii. Done: starts whether the required job is completed or aborted iv. Condition on a file watcher job (see below)

Scheduling information defining the two ways jobs can be controlled:

i. Simple scheduling where the days of a week on which a job is to run is selected from a list

ii. Complex scheduling with calendars determining when a job will run The execution time is input directly in the job and will be the same for all the days of the schedule.

Technical information related to job submission including execution server, log file, error file and owner details.

Workload and priority information assigning jobs a load value, based on importance

and the amount of system resources required. A maximum workload is attached to the server. When a job is about to start, AutoSys checks if there is enough workload on the system regarding the job load of the job and the maximum workload actually used:

i. If there is enough resource available, the job starts ii. If not, the job is queued (status “QUEWAIT”) to be started when the

maximum workload has decreased enough to accommodate the job load In the job definition, it is possible to define a priority which will be used by AutoSys to decide in which order jobs will run when several are queued.

2. The file watcher job is a specific job that will wait for files. This object contains (amongst

other definitions) the file name, location and its polling frequency.

As this job is not scheduled, it will start within a box (see below) or it is updated in “Active Status” (necessary so this kind of job can start) by another job (which runs a specific AutoSys command).

3. The box is a combination of jobs and a file watcher job. It is also possible to nest boxes (to have boxes of boxes). A box always has a header (box job) which has a schedule but is not really executed, which can be thought of as a technical job to identify the beginning of the box. The jobs inside a box may or may not have dependencies between themselves. If not, they will start simultaneously after the header has been completed.

White Paper


3. Dollar Universe Core Concepts

When upgrading to an enterprise automation platform it is important that an AutoSys user, be they administrator, developer, operator or engineer, understand how they can find the equivalent features to work with in Dollar Universe. It is also advantageous to be familiar with the architectural differences they can utilize. It is strongly recommended that users attend the basic training course when converting to Dollar Universe from any legacy job scheduler.

3.1 Dollar Universe Object Approach

The most noticeable difference when switching to Dollar Universe from AutoSys is very straight forward. The ORSYP scheduler utilizes an object approach.

What does that mean? For optimal job definition and flexibility, Dollar Universe offers different objects for each of the challenges that are faced in workload automation related to:

Jobs & job flows (WHAT to schedule?) Scheduling (WHEN to schedule?) Execution environment (WHERE to schedule?) Submission accounts (WHO executes the jobs on the server/VM?)

For each type of challenge, ORSYP Dollar Universe provides objects that can be created and be re-used in different contexts, thus limiting the number of objects that need to be managed and maintained:

The same job can be inserted in different job flows The same job flow can be executed in different execution contexts – as required to support

workload for different countries, business divisions, subsidiaries, applications, etc. The same calendar or scheduling rule can be used by different jobs or job flows Generic data sets can be created

For systems/applications/countries/regions with replication, this approach simplifies job definition/change management, with attributes defined only once with changes being deployed to agents allocated to process workload on behalf of countries, subsidiaries, business divisions, subsidiaries, applications, etc.

With AutoSys, if the same script/command is submitted with different parameters it needs to be added in the JIL file as a separate job.

White Paper


3.2 Terminology

Before starting a detailed description of Dollar Universe functionality, the table below is provided to summarize the most important features in AutoSys and how they can be accessed/found in Dollar Universe:

AutoSys Dollar Universe

Job UPROC

Owner SUBMISSION ACCOUNT

File watcher job RESOURCE - File

Box SESSION

Box Job Header UPROC of session

Box of box BUSINESS VIEW

Description Label (UPROC/SESSION / RESOURCE)

Max run alarm UPROC – Time Control Management

Job load / Workload DQM (Distributed Queue Manager) - queues

Machine MU (Management Unit) -> Linked to a Node

Condition Done(Job1) UPROC condition

Command COMMAND

Days of week RULE

Run calendar CALENDAR

Start time TASK – field called “start time”

Run window TASK – field called “Launch window”

Std out file JOB LOG

Std err file Retrieved in the JOB LOG

Box success Last UPROC of the SESSION

Box terminator Last UPROC of the Business View

Alarm if fails Embedded Alerting

Watch file RESOURCE - File

Watch file interval RESOURCE – File

Watch minimum size RESOURCE condition - File

White Paper


The following schema provides an overview of the objects that can be involved in the definition of workload to be scheduled by Dollar Universe.

Expanding on this overview, the following sections detail exactly what can be done answering the who, what, where, and when questions that need to be answered when automating workload.

3.3 JOBs and BOXes in Dollar Universe

Five different types of object - UPROC, RESOURCE, SESSION, BUSINESS VIEW and PRODUCTION PLAN - are used by Dollar Universe to determine what workload needs to be scheduled. Find below a brief description (the advanced concepts are all explained during the training at the beginning of any project):

UPROC (job) is defined by a

o Name (up to 64 characters)

o Label (description)

o Program/script/job (SAP, OEBS, FTP/SFTP…)

o Parameters to be passed to the program

o Application/Domain (to group the jobs/UPROCs in different levels)

TASK

RESOURCE

UPROCs SESSIONS

CALENDARS

RULES

SUBMISSION ACCOUNTS

DQM QUEUEs

entity 1 MANAGEMENT UNITS

WHERE?

WHEN? WHAT?

WHO?

White Paper


o Time control management (trigger alert based on historical execution time of that UPROC)

o Status Management rules (customize return codes)

o Set Predecessors (conditions in the launch formula). Different types:

UPROC completed/failed/absent

RESOURCE (file, logical, SAP, JMS, available CPU, available RAM memory…)

INCOMPATIBILITY with other jobs/UPROCs

o Email notification

RESOURCE definitions can be set for the following:

o File (file watcher): object that points to a specific file in a directory

o Logical (simple workload): to control the number of parallel executions of a group of jobs (based on the quotas/tokens granted)

o JMS: JMS message/event that is expected in a JAVA topic/queue

o SAP Event: SAP event that is expected in the SAP instance

o SAP Job: SAP job event that is expected

o Script: path and script name to be executed prior to the UPROC execution

o System:

Space required on a file system

RAM memory required

CPU required on the server

o SP Monitor alert can be triggered

When configuring RESOURCEs different fields will be presented as options for each resource. This will depend on the type of resource being used.

White Paper


SESSION (box): define the grouping and process flow of UPROCs that will be submitted.

There is always a single “Header UPROC” in a session and then the sequence of UPROCs will be submitted as defined by the:

o OK path (blue lines)

o KO path (red lines)

o Predecessors defined at the UPROC level. Conditions displayed in thin green/red/blue arrows)

The UPROCs in a SESSION can be submitted on different servers/VMs. Note that the same UPROC can be used many times in the same or different SESSION. For instance: a UPROC “BACKUP” can be reused to repeatedly execute backups (same command) for different databases.

BUSINESS VIEW (box/es in a box): group of sessions put together. These sessions can be linked (or not) through predecessors/conditions. The red-squared boxes represent multiple SESSIONS (boxes) in a Business View:

PRODUCTION PLAN (box/es in a box of boxes): group of Business Views (with their corresponding sessions) put together. The Business Views can be linked (or not) through predecessors. By double-clicking the boxes, the Business Views will be automatically displayed.

White Paper


3.4 Commands

The commands of standard jobs in AutoSys include:

The name and location of a script with or without parameters

A command of the OS

A specific order to run SAP jobs (thanks to the Autosys SAP agent)

A specific order to run Oracle application jobs…

In Dollar Universe, the same options are available when defining UPROCs (jobs). These are the options available to input “commands” or job types:

White Paper


3.5 Predecessors (conditions)

The predecessor conditions are always defined at the UPROC level (job). Find below a detailed list of the conditions you will be able to define for each UPROC (therefore also available at the session, business view and production plan):

UPROC dependency (job condition): the conditioned UPROC will wait for the expected UPROC to be:

o COMPLETED

o ABORTED (failed)

o ABSENT (not completed or failed)

Multiple UPROC conditions can be defined for the same UPROC.

For each expected UPROC in the condition, different parameters can be modified:

o Submission user

o Execution date (same day, previous day / week / month / quarter / year, working or holiday or weekend, etc.)

o Session where the UPROC is expected (since the same UPROC can be reused in other sessions and servers)

RESOURCE dependency: once the type of resource has been defined, the user will be able to create any type of dependency with any UPROC already defined. Based on the type of RESOURCE, different conditioning parameters will appear:

o File (File Watcher dependency): size of the file expected, file stability (last day/time changed…) scanned frequency, …

o Logical: number of tokens that will be taken from the total quota when the UPROC will be executed. These tokens will be given back once the UPROC has been completed

o JMS: name of the JMS message and contents of the message that will need to be in the JAVA topic/queue

o SAP Event: name of the SAP event expected in the SAP instance

o SAP Job: name of the SAP job expected in the SAP instance

o Script: script (path and name with parameters) that will be executed before the UPROC is triggered. Only if the return code of the script specified is completed successfully, will the UPROC conditioned be submitted

o System:

Storage: amount of available space in the file system specified

CPU: % of CPU available so that the UPROC can be executed

RAM: amount of available RAM memory so that the UPROC can be executed

White Paper


INCOMPATIBILITY dependency: Dollar Universe’s way to define incompatible UPROCs. If one of the incompatible UPROCs is running, then the conditioned

UPROC will not run, until all of them are not being executed.

3.6 Scheduling

In order to answer the question WHEN to schedule objects, Dollar Universe provides two different types of objects:

CALENDAR in Dollar Universe is used differently than the way it is used with AutoSys. In Dollar Universe it is only seen/used as a day reference to determine the type of days:

o There are no limits on the number of calendars that can be created

o Each calendar can be defined with three different type of days – WORKING, HOLIDAY and CLOSING (weekend)

o Each calendar is associated with an application country, a datacenter or other resource by a link through the MANAGEMENT UNITs in the TASK. The same SESSION/UPROC can be linked to multiple calendars and be submitted on different days.

RULE is the object that will allow the periodicity of execution of jobs (UPROCs) and SESSIONS (jobflows) to be defined. The RULEs will allow the user to define the frequency of job (for example: every working day, every Monday, 15

th day, last working

day of a month). Dollar Universe offers, as standard, the most frequently used scheduling rules with custom rules used to define any frequency of execution (n days, n weeks, n months etc.), type of day (working, closed, holiday), authorization (no working Tuesday or holiday Thursday, for example), off-set (before, after, without).

White Paper


3.7 TASK

TASK is the object that puts all the different elements of a workload together within Dollar Universe resolving the WHAT, WHEN, WHERE and WHO.

A TASK is defined by the following fields:

UPROC or SESSION to schedule

SUBMISSION USER that will execute the job on the server/VM

MANAGEMENT UNIT that will select the execution context and the CALENDAR

RULEs that will establish how often this job can run (there is a type of TASK that will allow on-demand execution, so in this scenario, no RULES will be defined in the TASK)

QUEUEs where the UPROCs/SESSION will start running (then they can be load-balanced)

Execution time with multiple options:

o Single launch (once a day)

o Cyclical launch (multiple times a day)

o Cyclical: once the SESSION has been finished, the SESSION will automatically start from the beginning)

Duration of the “Launch Window” (Run Window in AutoSys) how much time will the job have to start if the UPROC has one or multiple conditions

Other options

White Paper


The same SESSION (or UPROC), can be scheduled on different MANAGEMENT UNITS (for instance, countries), that will pre-select their own CALENDAR (for instance, including the country’s bank holidays, weekends and working days), by the same or different SUBMISSION ACCOUNTS. At the same time, this task will have a pool of RULES that will establish how often that SESSION/UPROC will be submitted (eg last working day of the fiscal quarter) and at what time. In summary the possibilities are boundless with Dollar Universe offering the flexibility and agility needed to address current and future workload automation demands.

3.8 Workload Balancing and Priorities

Dollar Universe features a job batch queue system that ensures optimum resource distribution by:

limiting the number of parallel executions configurable for each queue (maximum number of simultaneous jobs between 0 and 9999),

load balancing between different machines/VMs (notion of logical queues pointing to one or more physical queues sitting on one or more machines),

In addition, this system allows the management of job priorities.

White Paper


4. AutoSys Replacement: ORSYP Project Model

With many years of experience and accumulated expertise, ORSYP has a proven track record and implementation methodology migrating AutoSys users to an enterprise automation platform that offers extra flexibility and new agility. Critical to the success of each project is ensuring that all aspects of the conversion are covered.

4.1 Environmental Analysis

Key individuals responsible for automation architecture, infrastructure, applications and deployment will benefit from initial training. This will provide the knowledge to support the many decisions that need to be made in project planning, The following outlines an example of the multiple steps of an upgrade.

Project plan needs to address all aspects of migration

4.2 Automation Strategy & Architecture

Prior to product deployment, there needs to be agreement on architectural design for the automation landscape. Consideration needs to be given to how best practices as well as norms and standards can be applied throughout the implementation. Topics that need covering include:

Security requirements

Naming conventions

Staging environments for lifecycle management

Integration with applications and infrastructure tools

Problem escalation procedures

Automated recovery plans

Disaster recovery and business continuity

At this stage it is also imperative to identify any issues that might cause delay or where extra resources may be required. Working through each of these topics, and anticipating any factors that might add project risk, a detailed plan is built upon which the rest of the implementation will be based.

White Paper


4.3 Software Deployment

Automated installers ensure product deployment is swift – not only advantageous when first implementing the software but also crucial when workload needs immediate deployment across thousands of servers in a production scenario.

4.4 Applications & Infrastructure Integration

Workload executing within key business applications needs to be identified along with system administration and monitoring tools used to manage IT infrastructure. For leading enterprise applications out-of-the-box connectors are available. Centralizing configuration and control of all application workload accelerates implementation and enhance end-to-end visibility of IT processes. Adapters that integrate with IT supervision tools, such as HP Operations Manager, enable drill-down investigation and analysis when resolving problems.

Custom code, scripts and in-house developed tools used to control IT processing will require analysis. In many cases the purpose they serve may become redundant with the upgrade or can be accomplished by a feature of the ORSYP solution. Obviously, this exercise can determine any functionality that does need to be transferred onto the new automation platform.

4.5 Migration of Job Definitions

ORSYP has extensive experience converting CA AutoSys jobs and job flows to Dollar Universe. The conversion of CA AutoSys to Dollar Universe is accomplished with the use of the extracted JIL files including the “Jobs” and “Box” definitions. There is a second file that may optionally be provided, which can allocate a user-defined name for each Dollar Universe job name.

Automated conversion utility extracts JIL data

While this exercise can provide a 1:1 image of the current scheduling rules from AutoSys it can also be used to refine schedules. For instance, multiple jobs performing basically the same function are rationalized down to a single re-usable object – reducing future maintenance and administrative overhead.

White Paper


4.6 Testing & Verification

This critical stage in the process will typically be broken down into manageable repeatable units. A pilot application may be identified as an initial target and once this exercise is successfully completed, the process can then be repeated for other applications, business units, or job flows.

After initial review and fine tuning of converted objects, it is possible to deploy job definitions and process flows with ORSYP for simulation and unit testing. This enables sequencing and scheduling of jobs to be confirmed prior to submitting workload for rigorous acceptance testing in a QA environment. Once validation is complete, the new objects are ready to go live, with existing legacy scheduling procedures being disabled at the same time.

Staging areas manage roll-out from test to production environments

This phase of the process is usually handled by ORSYP and client teams working together. Quite often there will be a knowledge transfer exercise with ORSYP staff sharing best practices on how to structure test plans and creating template examples with client staff becoming increasingly independent as skills develop.

4.7 Production Support

To provide complete assurance of the success of each upgrade, ORSYP consultants can remain on-site during the first few cycles. This can be advantageous supporting IT Operations through the transition period as staff gets familiar with the new tools. Of course, this helps drastically reduce any perceived risk when transitioning to a new automation platform.

© ORSYP 2013 ▪: ORSYP: An Alternate to CA AutoSys

About ORSYP

ORSYP is an IT Operations Management specialist that

develops and markets innovative solutions that automate

IT processes and optimize resource utilization. ORSYP’s

proven software – including enterprise job scheduling,

workload automation, performance monitoring and

capacity management – and its ITSM consulting services

give organizations enterprise-wide visibility and proactive

control of their IT operations. Headquartered in Boston,

Hong Kong, and Paris, ORSYP has been chosen by over

1400 customers as their trusted partner for software,

consulting and education since 1986.

www.orsyp.com

Americas Headquarters

300 TradeCenter 128 Suite 5690 Woburn, MA 01801 USA

+1 781 569 5730

www.orsyp.com

–––

APAC Headquarters

Honest Motors Building 9-11 Leighton Road Causeway Bay Hong Kong, China

+852 2575 5966

www.orsyp.com

–––

EMEA Headquarters

Tour Franklin 92042 Paris La Défense

Cedex France

+33 [0]1 47 73 12 12

www.orsyp.com

http://www.orsyp.com/

Date post:	01-Feb-2018
Category:	Documents
Upload:	lamdat
View:	222 times
Download:	4 times

ORSYP: Getting From CA AutoSys onto Dollar...

Documents