IBM Software GroupIBM Software Group
© 2009 IBM Corporation
Tivoli Workload Scheduler for z/OS 8.5
Alexandra ThurelTWSz Development Manager
Paris, 12 Mai 2009
2© 2009 IBM Corporation
Présentation de la nouvelle version Tivoli Workload Sc heduler for z/OS v8.5
�Principes de base TWS for z/OS
�Nouveautés TWS for z/OS 8.5� Les dépendances
conditionnelles� Enhanced Event Management
�Overview sur les solutions Tivoli Workload Automation et leursroadmaps
http://www.flickr.com/photos/jahovil/542568413/
3© 2009 IBM Corporation
A TWS for z/OS Topology
Sysplex z/OS Domain
TWS for z/OSAgents
TWS for z/OSEngine
TWS for z/OSHot standby Engine
SchedulingDataBase
SchedulingPlan
Job Scheduling
Console
Info Mgmt
TWS for z/OS/ESA
Monitoring ALLSYSTEMS
CITY1 CITY2
CITY3
ISPF
z/OS Domain
eWas
TDWC
Tivoli Dynamic Workload Console
4© 2009 IBM Corporation
IBM TWS for z/OS architecture
� TWS for z/OS Engine, implements all the TWS for z/OS control logic
– Maintain TWS for z/OS database and plans
– Choreograph and triggers the execution of the jobs
– Manage the connections with the agents
– Handles requests from the user interfaces
� TWS for z/OS Agents, receives jobs to be started from engine
– Submit jobs to operating system for execution
– Tracks jobs execution status via JES and SMF exit routines
– Sends job status change events to the engine
� TWS for z/OS Datastore, collects all the Jobs output for later use by the Restart&Cleanup feature
z/OSSYSPLEX
z/OSSYSPLEX
Agent
TWS z/OS Engine
Agent Agent
Standby Engine
Standby Engine
TWS z/OS Agent
VTAM VTAM
TWS z/OS Agent
TWS z/OS Agent
TWS z/OS Agent
TWS z/OS Agent
TWS z/OS Data-store
TCPIP (da 8.3)
5© 2009 IBM Corporation
IBM Tivoli Workload Scheduler for z/OS V8.3
1. Web-based operations console and integration with TEP
2. Ability to define Critical Workloads with analysis to automatically promote Critical Workloads based on Critical Path analysis
3. Enhanced planning and choreography services including “every” option, wait workstation, enhanced loop analysis
4. Event-triggered tracking variables and special resource additional flexibility for more dynamic, real-time workload scheduling and execution
5. End-to-end flexibility, easier setup and fully centralized control
6. Integration with IBM WLM to dynamically route workloads to best available resources, and new integration with SA z/OS to start/stop or otherwise automate resources from within TWS
z/OSExecutor
z/OSExecutorz/OS
Executorz/OS
Executorz/OSExecutor
z/OSExecutor
Triggering ServiceTriggering Service
Planning and Choreography Services
Planning and Choreography Services
Brokering and Execution ServicesBrokering and Execution Services
Modeling UIModeling UI OperationsConsole
OperationsConsole PIFPIF
Time and calendaringrulesEventsEvents
EventsEvents
TWS z/OS Controller
TWS z/OSTrackers TWS Distributed
Agents
1
IBM WLMIBM
WLMSA
z/OSSA
z/OS
2
4
3
5
6
Tivoli Workload Scheduler z/OS
6© 2009 IBM Corporation
�Nouveautés TWS for z/OS 8.5� Les dépendances conditionnelles� Enhanced Event Management
7© 2009 IBM Corporation
Tivoli Workload Automation - last updates timeline2H07 1H08
Tivoli Workload
Scheduler for z/OS
Tivoli Dynamic Workload Broker
Previous versions
� SAP load balancing� SAP events in EDWA� Import SAP calandars
Tivoli Dynamic
Workload Console
V8.4
� Event based scheduling� Reporting feature� LDAP and IPv6 support� SAP job submission (alignment to JSC)� z/OS 8.3 features alignment
Tivoli Workload
Scheduler
V8.4
� Event based scheduling� TEP integration� IPv6 support� LDAP support� Consumability improvements
Tivoli Workload Scheduler for Applications
V8.4
� Reporting feature� Virtual Workstation� IPv6� NOERROR enhancements
V8.3 SPEs
� Workload Service Assurance
� CCMDB Integr.� Events for business logic� Eclipse-based TWS
Workbench
V8.4 FP1
V8.4 FP1
� BDC session management� IDOCs management� Process chains
V8.4 FP1
� Workload Service Assurance
V8.3
V8.3
V8.3
V1.2
V8.3
V8.3 SPEs
� Enhanced JCL Variables
� Variable tables for streams� Installation improvements
(deployment scheme)� Workload Service Assurance
V8.5
� Modeling and Administration on Single GUI
V8.5
3Q08 4Q08 1Q09 2Q09
� Conditional dependencies� Enhanced event-driven
management� OPC removal� TEP improvements
V8.5
� Basic Job throttling � Monitor Alerts� Solution Manager integration
and certification
V8.5
V8.5 FP1
� Conditional dependencies alignment
� Usability enhancements
V8.5
�Conditional dependencies�Enhanced event-driven management
�OPC removal�TEP improvements
V8.3 SPEs
8© 2009 IBM Corporation
Conditional dependencies
Business scenario
� The energy company needs to automate conditional execution logic, and dynamically perform different branches of invoice processing for different typology of clients.
� The organization desires to flexiblyimplement self-recovery system, automating complex branches of recovery processing.
� General need is to support real-timeprocessing decisions, based upon combined conditions on return codes and statuses of specific workloads.
JOB1
JOB2
JOB3
JOB4
JOB6
RC=0
RC=4
JOB5
JOB3 status = C ORJOB4 status = C ORJOB6 status = C
RC<>0 and RC<>4
9© 2009 IBM Corporation
Conditional dependencies - purpose
Solution
� Simple static workflow definitions (job execution conditioned on successfulcompletion of predecessor jobs) are notenough to map dynamic and complexworkflows. It will be possible to definecombined workflows definitions on return codes/statuses.
Customer value
� Easily map business processes with reduced TCO
� Replace IF/THEN/ELSE JCL coding to simulate branches of workflows
� Easy monitoring of workflows
10© 2009 IBM Corporation
OP1 in ACONDC1
OP1 in ACONDB1
RC GE 4
A2A1 A3
B
OR, AND, N out of M
OP1 in ACONDC1
RC LT 4
Conditional dependencies - definition
� Predecessor Id
� Predecessor info to be checked:• Return Code � RC• Status � ST
� Relational Operator • EQ, NE, GT, GE, LT, LE, RG
� Rule• AND, OR
11© 2009 IBM Corporation
Conditional dependencies - rules
A4
At least 2 out of 4 A1, A2,A3, A4 cond deps mustbe true
A1 A2 A3
B
N out M
A1 A2 A3
and
B
andand
All of A1, A2 A3 cond deps must be true
A1 A2 A3
orB
oror
At least one of A1, A2 A3 Cond deps must be true
12© 2009 IBM Corporation
Start operation Z1 when the following Condition is true:
((A1 RC > 0) and (A1 RC <= 12)) or ((C1 RC >0) and (C1 RC <= 12))
A1 C1
Z1 At least one must be true:A1 RC range 0-12C1 RC range 0-12
Condition example
13© 2009 IBM Corporation
D1
All must be true:C1 rc GE 8D1 rc GE 8
A1 B1 C1
Z1
or
Y1
and
Y2
andAll must be true:A1 rc GE 4B1 rc GE 4
At least one must be true:Y1 Status = CY2 status = C
Y1 and Y2 are DUMMY operations
Condition example
Start operation Z1 when the following Condition is true:
((A1 RC >= 4) and (B1 RC >= 4)) or ((C1 RC >= 8) and (D1 RC >= 8))
14© 2009 IBM Corporation
JOBA
JOBA rc LT4
At least one must be true:JOBA rc=0JOBRLT4 rc=0JOBREGT4 rc=0
Autorecovery application
APPLA is designed in a such a way to manage possible operations errors
JOBRLT4
JOBA rc GE4
JOBREGT4
JOBB
APPLA
15© 2009 IBM Corporation
� JOBA performs a critical task
� JOBRL4 is the recovery job that handles error condition JOBA RC<4
� JOBRGE4 is the recovery job that handles error condition JOBA RC>=4
� JOBB must be run after successfull completion of JOBA or success fulrecovery through JOBRL4 or JOBRGE4
Example – Autorecovery Application
16© 2009 IBM Corporation
� JOBRL4 - Define conditions
� JOBRL4 - Define condition dependency (Run if 0 < JOBA <4)
Autorecovery Application- Definition
17© 2009 IBM Corporation
Autorecovery Application - Definition
� JOBB - Define condition dependency (Run if JOBA=0 or JOBRL4=0 or JOBRGE4=0)
18© 2009 IBM Corporation
Autorecovery Application - Definition
� Define condition dependency through Tivoli Dynamic Workload Console
19© 2009 IBM Corporation
Autorecovery Application - Definition
� Define condition dependency through Tivoli Dynamic Workload Console
20© 2009 IBM Corporation
R
How TWSz evaluates conditions
JOBA
JOBRL4 JOBRGE4
0 < RC <=4
JOBB
RC = 0RC = 0
RC = 0
or or
UNDECIDED
UNDECIDED
UNDECIDED
� JOBA ends withRC=8
TRUE
4 < RC
FALSE
� JOBRGE4 endswith RC=0
TRUE
E
W
W
W
R
X
C
C
� APPLA instanceis ran
21© 2009 IBM Corporation
� Conditional Dependency status / evaluation
� T True� F False
� U Undecided
� Evaluation is done each time an involved operation status changes� Until final values: True, False
� Operations status
� An operation with condition dependencies defined changes status among:
� W Waiting - Conditional dependency is not yet valuable � R Ready - Conditional dependency is True
� X X (dummy complete) - Conditional dependency is False
� X status is propagated to any normal successor at any level
� To prevent that, you can define a condition dependency based on the X
status
How TWSz evaluates conditions
22© 2009 IBM Corporation
X status
Propagation of X status
Unless you define a condition dependency on X status
Ex: JOB3 cond dep on JOB2 ST X
JOB1
JOB2
JOB3
JOB4
JOB6
RC=0
RC=4
JOB5
JOB3 status = C ORJOB4 status = C ORJOB6 status = C
RC<>0 and RC<>4X
X
XC
C
23© 2009 IBM Corporation
Meaning of recovery operation
� In the previous example, APPLA instance is in Error status and therefore it is not deleted at Daily Plan
� Nevertheless, you may want to consider the instancesuccessfully completed, since you designed the job JOBREGT4 to recover JOBA error
StatusOperation
CJOBBCJOBREGT
4
XJOBRLT4EJOBA
APPLA
� In this case, when defining JOBRLT4 and JOBREGT4, besides specifyingJOBA as conditional predecessor, youcan also set the COND RECOVERY JOB option to Y
� So, APPLA instance will be removed byDaily Plan
24© 2009 IBM Corporation
Daily plan
� The Daily Plan process removes the occurrence in the foll owing cases:
– The occurrence status is complete
– The occurrence is in error, including only operations in one of the followingstatuses:
• Complete
• Suppress by condition
• Ended-in-error, with the RECOVERED BY CONDITION field set to Y
25© 2009 IBM Corporation
Summary
– Defined in the AD operations
– Tracked in the Current Plan, either created at plan creation time or dynamically added
– Checked by Scheduler to decide if an operation is eligible to be submitted
– Condition dependencies can be modified in the CurrentPlan, if their status is Undecided
26© 2009 IBM Corporation
Tivoli Workload Automation - last updates timeline2H07 1H08
Tivoli Workload
Scheduler for z/OS
Tivoli Dynamic Workload Broker
Current versions
� SAP load balancing� SAP events in EDWA� Import SAP calandars
Tivoli Dynamic
Workload Console
V8.4
� Event based scheduling� Reporting feature� LDAP and IPv6 support� SAP job submission (alignment to JSC)� z/OS 8.3 features alignment
Tivoli Workload
Scheduler
V8.4
� Event based scheduling� TEP integration� IPv6 support� LDAP support� Consumability improvements
Tivoli Workload Scheduler for Applications
V8.4
� Reporting feature� Virtual Workstation� IPv6� NOERROR enhancements
V8.3 SPEs
� Workload Service Assurance
� CCMDB Integr.� Events for business logic� Eclipse-based TWS
Workbench
V8.4 FP1
V8.4 FP1
� BDC session management� IDOCs management� Process chains
V8.4 FP1
� Workload Service Assurance
V8.3
V8.3
V8.3
V1.2
V8.3
V8.3 SPEs
� Enhanced JCL Variables
� Variable tables for streams� Installation improvements
(deployment scheme)� Workload Service Assurance
V8.5
� Modeling and Administration on Single GUI
V8.5
3Q08 4Q08 1Q09 2Q09
� Conditional dependencies� Enhanced event-driven
management� OPC removal� TEP improvements
V8.5
� Basic Job throttling � Monitor Alerts� Solution Manager integration
and certification
V8.5
V8.5 FP1
� Conditional dependencies alignment
V8.5
�Conditional dependencies�Enhanced event-driven management
�OPC removal�TEP improvements
V8.3 SPEs
27© 2009 IBM Corporation
Business scenario
� The pharmaceutical company needs to produce consolidated daily reports, gathering data provided with files FTP-ed on distributed systems
� General need is to raise the automation level one step, intercepting and reacting to a wider pool of events, and integrating to e-business applications - while simplifying existing mechanism
Solution
� Easy and consolidate way to define flow of triggering events
� File dependencies on USS, Windows and Unix files
Event-driven workload automation
28© 2009 IBM Corporation
Dataset triggering
� Besides the acquired dataset triggering method to define dataset triggering, now you have an additional easier and more powerful way to do it
Acquired method
� Edit and compile assembler macros to build the triggering dataset table
� Manually deploy the table to each system where data set monitoring must be active
New method
� Single centralized job to build and automatically deploy configuration files
� Define event rules, triggering files and destinations editing an XML sample
� Deploy configuration files to all the defined tracker destinations
29© 2009 IBM Corporation
Customize triggering criteria, build and deploy config files
30© 2009 IBM Corporation
Filewatcher
� Filewatch utility enables reactions to file system changes of files and directories
�HFS
�Windows and AIX files
� You can implement file dependencies defining a job that runs the filewatch utility
Filewatch
The Filewatch runs until the specified deadline or until the condition is matched
Condition – File created, modification running, modification completed, modificationrunning completed, deleted
Condition is checked through file size and modification time changes, which are checked at user specified interval time.
31© 2009 IBM Corporation
HFS/ZFS Filewatch utility
32© 2009 IBM Corporation
E2E Filewatch utility
Paul creates an application to trigger a job when a HFS file has been updated via FTP on a windows machine
FILEWATC job can bedefined in a static way:
or in a dynamicway:
33© 2009 IBM Corporation
Tivoli Workload Automation - last updates timeline2H07 1H08
Tivoli Workload
Scheduler for z/OS
Tivoli Dynamic Workload Broker
Current versions
� SAP load balancing� SAP events in EDWA� Import SAP calandars
Tivoli Dynamic
Workload Console
V8.4
� Event based scheduling� Reporting feature� LDAP and IPv6 support� SAP job submission (alignment to JSC)� z/OS 8.3 features alignment
Tivoli Workload
Scheduler
V8.4
� Event based scheduling� TEP integration� IPv6 support� LDAP support� Consumability improvements
Tivoli Workload Scheduler for Applications
V8.4
� Reporting feature� Virtual Workstation� IPv6� NOERROR enhancements
V8.3 SPEs
� Workload Service Assurance
� CCMDB Integr.� Events for business logic� Eclipse-based TWS
Workbench
V8.4 FP1
V8.4 FP1
� BDC session management� IDOCs management� Process chains
V8.4 FP1
� Workload Service Assurance
V8.3
V8.3
V8.3
V1.2
V8.3
V8.3 SPEs
� Enhanced JCL Variables
� Variable tables for streams� Installation improvements
(deployment scheme)� Workload Service Assurance
V8.5
� Modeling and Administration on Single GUI
V8.5
3Q08 4Q08 1Q09 2Q09
� Conditional dependencies� Enhanced event-driven
management� OPC removal� TEP improvements
V8.5
� Basic Job throttling � Monitor Alerts� Solution Manager integration
and certification
V8.5
V8.5 FP1
� Conditional dependencies alignment
V8.5
�Conditional dependencies�Enhanced event-driven management
�OPC removal�TEP improvements
V8.3 SPEs
34© 2009 IBM Corporation
�Overview sur les solutions Tivoli Workload Automation et leurs roadmaps
http://www.flickr.com/photos/tjblackwell/2319789174/
35© 2009 IBM Corporation
Tivoli Dynamic Workload Broker
Job Scheduling Console
Tivoli Workload Scheduler Agents
AIX, HP, Solaris, Windows, Linux, OS/400, z/OS…
Tivoli Workload Scheduler for Applications
SAP PeopleSoft Oracle
Web-Based Workload Console
End to End Workload
Automation
Tivoli Workload Scheduler for
z/OS
Tivoli Workload Scheduler
Mainframe Distributed
Tivoli Enterprise Portal
TWS LoadLeveler
BlueGene HPC Grid
� Evolve towards a simple WorkloadAutomation backbone
� Be the most flexible WorkloadAutomation solution in the market
� Expand scheduling space topopular applications
� Summarize all information in asynthetic GUI on Web
� Create a foundation for On Demand Enterprise scheduling
� TWS: a long-term platform
� Certified for numerous standards
� Excellent backward compatibility
Tivoli Workload Automation: themes and portfolio
36© 2009 IBM Corporation
IT Infrastructure & Production Services
IT Infrastructure & Production Services
IBM Service ManagementIBM Service Management
Processes
Align Business and IT Objectives, and Execution of Services with Workload Automation
Business ProcessesBusiness Processes
IBM TivoliWorkload
Automation
IBM TivoliWorkload
Automation
� Financial consolidation and reporting
� Supply chain management
� Customer support
� Online banking, pharmacy benefits, . . .
� File transfers, data backup. . .
� Data processing and analysis
� Service catalog and delivery management
� Change and configuration management
� Release management
� Incident and availability management
� Business continuity management
37© 2009 IBM Corporation
Business and Application ServicesBusiness and Application Services
IT Infrastructure ServicesIT Infrastructure Services
Performance Management ServicesPerformance Management Services
Tivoli Workload Automation Integration Points
SAP Systems
Oracle Systems
PeopleSoft Systems
. . .
Tivoli Storage Manager
Tivoli NetView
Tivoli System Automationz/OS & Multiplatforms
Tivoli Monitoring
Tivoli OMEGAMON
Tivoli Business ServiceManager
Tivoli INFOMAN
Tivoli Enterprise Console
Tivoli ProvisioningManager
Workload Manager Tivoli ConfigurationManager
Tivoli Netcool
Tivoli CCMDB &Other Process Managers
WebSphere ServiceRegistry & Repository
WebSphere ApplicationServer
WebSphere ExtendedDeployment
Open Grid Architecture
Web Services and J2EE
IBM Tivoli WorkloadAutomation
IBM Tivoli WorkloadAutomation
Tivoli Service RequestManager
Process Management Services
Tivoli Enterprise Portal
38© 2009 IBM Corporation
Evolve toward a workload integration backbone
– Scalable - Flexible
– Compatible - Separable
– Virtualised - Optimising
– Single interface
Process Consistency: visualize and extend coverage
– Provide the constructs to map unattended Business Processes
– Handle workflow variability and template reuse
– Interoperate with other flow drivers
– Facilitate criticality mapping and business consistence
– Report on every compliance issue and governability
– Build higher level scenarios for entire LOB.
Be part of the foundations of Dynamic Data Center
– Integrate with business system management for policies and service level management.
– Share data with CMDB in and out
– Streamline relevant information on the UIs
– Dynamically schedule & choreograph enterprise workloads across virtual resources
TWA theme summary
39© 2009 IBM Corporation
2H06
Tivoli Workload Automation 2006-2008: Roadmap2H081H06
Tivoli Workload
Scheduler
V8.4 SPE
Tivoli Dynamic
Workload Broker
� CCMDB Integr.� Events for business logic� Eclipse-based TWS
Workbench
1H082H07
V8.5
Tivoli Dynamic
Workload Console
V8.4 SPE� CCMDB navigation.� TEP Integr.� Workload Service
Assurance� Manage SAP process chain� Launch in context
V8.5
� Modeling and Administration on Single GUI
� Dojo Monitoring tables� Workload Service Assurance
views
Tivoli Workload Scheduler for
z/OS
� Advanced Process Chain mgmt� Manage IDOC� Embed BDC sessions
V8.3 SPE
� Workload ServiceAssurance
V8.3 SPE
� Facilitate migration withkey enhancements toreduce migration costs
Tivoli Workload
Scheduler for Applications
Last family Plan DCP
� VarIable tables for streams� Streamline configuration (simplify
deployment scheme)� Workload Service Assurance
V8.4 SPE
� TCP/IP communication support
� Advanced Reporting� Virtual workstation� IPV6 compliance
SPEs
On V8.3V8.3
� Enhanced planning/choreography
� Critical Path� Deadlock detection & analysis� Dynamic event-based scheduling � End-to-end new configuration� TEP integration
V8.3 V8.4
� Advanced Event Triggering� Alert norification� Advanced Reporting� TEP integration� TDWC Trial and forecast planning� CCMDB data exporting� Standard LDAP , IPV6
� RDBMS integration� WEB based application
infrastructure� Enhanced planning
functionalities� J2EE and WEB services API
V1.1 V1.2� Extended platform coverage
(Solaris, HP)� Auditing� Oracle RDBMS support
1H07
V8.3 V8.4� Monitoring views� Tasks
� Optimized Monitoring tasks� LDAP SSO� Reports� Event management� Task sharing� Dashboard
V8.3 V8.4
� SAP Event monitoring� Load balancing for
SAP jobs
� Enhanced SAP scheduling� PeopleSoft and Oracle
extended support
� Dynamic job routing� Automated resource scanning and
matching with requirements� Dynamic topology� Optimisation and resilience policies
for load assignment.
� Enhanced event driven automation (additional variations on file z and dist)
� Conditional dependencies� Additional key enhancements to
reduce migration costs� Eliminate OPC references
V8.5
V8.5
� Basic Job throttling � Monitor Alerts� Solution Manager integration
and certification
1H09
40© 2009 IBM Corporation
Market analysis
Emerging market needs combine with immediate customer desires and enriches the strategic perspective
Customer connect and other drivers
Customer projects
Success stories
Validation/review Innovative delivery methods provide early feedback:
Customer involvement in whole development lifecycle
Customer Advisory Council
Design validation, early code verification
Report quality feedback
Project success generates feedback from Customer and enhances collaboration
IBM collaboration
Direct interaction of Lab with Customers, Customer advocacy program, events and user groups provide the needed channel to know customer needs and offer technical expertise and involvement
Discussions with Customer over the past 3 years indicate they need careful planning to deploy successful Workload Automation projects
41© 2009 IBM Corporation
Agile is a user-centric development process that focuses on delivering systems that meet dynamic business requirements
Agile is designed to not only cope but welcome changing requirements. It is efficient because it builds quality into it’s processes.
The customer gives up some certainty about scope and cost. In return, they get quality, value for money and visible progress.
All parties benefit from the open collaboration and transparency fostered by Agile processes.
Agile value
IBM Software GroupIBM Software Group
© 2009 IBM Corporation
43© 2009 IBM Corporation
Collaboration Zone as part of Development Process
Collaboration
Zone
Collaboration
Zone
Publishing
product themes
and areas of
investment
Grouping,
filtering,
prioritizing
feedbacks..
comments..
...suggestions
Feature selection
and Milestone 1
content definition
Milestone 2
Dev, Test and Doc Milestone 3
Dev, Test and Doc
bugs..
Product
Release!
enhancementenhancementenhancementenhancementssss........changes..
Product
Themes
Milestones
Update
Input
collection
parsing, and
organising
Time boxed iteration
download..
...use...
..and post!Plan
Bin
Endgame
Test
enhancementsenhancementsenhancementsenhancements.. .. .. .. changes... Bugs...
Integrated
In the current Milestone
next
Milestone 1
Dev, Test and Doc