1 3.1.3..................................................................Monitor Production Batch Jobs With CA-78
3.1.4.1 CA7 Queues.................................................................................................................................83.1.4.2 Determine Job/Cycle Status with CA-7 List Commands...............................................................93.1.4.3 Determine status of Time Jobs in CA-7........................................................................................93.1.4.4 Place Jobs on Hold with CA-7......................................................................................................93.1.4.5 Demand in a Job with CA-7.........................................................................................................93.1.4.6 Delete a Job from CA-7 Production Queue.................................................................................103.1.4.7 Restart a Job through CA-7/CA-11.............................................................................................10
Page 1 of 42
Fon’s Batch Procedures
3.1.4.8 Force a JOB Complete Using CA-7............................................................................................103.1.4.9 Post a Condition to a Job............................................................................................................103.1.4.10 List Documentation for a Job....................................................................................................113.1.4.11 Use the Forecast Commands.....................................................................................................113.1.4.12 Use the List Log and List Prior Run Commands........................................................................113.1.4.13 Use the HELP Function Through CA-7.....................................................................................113.1.4.14 Production Scheduling Interface (PSI).......................................................................................113.1.4.15 Search for Logon Ids for Jobs that were put on Hold.................................................................133.1.4.16 Escalation Procedures when a Cycle is Running Behind............................................................13
3.1.5.1 Initial Response to ABENDS......................................................................................................143.1.5.2 Research / Resolve Common ABENDs.......................................................................................14
3.1.5.2.1 Common Space Problems....................................................................................................153.1.5.2.2 Code Override Statements for SMS SB37 ABENDs Involving Dataclasses..........................153.1.5.2.3 Job Failed - JCL Error..........................................................................................................163.1.5.2.4 Job Not Run - JCL Error......................................................................................................163.1.5.2.5 Migrated Datasets................................................................................................................163.1.5.2.6 Tape Errors..........................................................................................................................163.1.5.3 Abend Tracker Database.........................................................................................................173.1.5.3.1 Sign into the database..........................................................................................................173.1.5.3.2 Monitor abend entries..........................................................................................................173.1.5.3.3 Mark an entry resolved........................................................................................................173.1.5.3.4 Secondary escalation............................................................................................................17
0 3.1.6...............................................................................Use On-Call SE Response Facility18
3.1.20 Cancelling Jobs in IMS on 231A......................................................................................................250 3.2......................................................................................................................... Use $AVRS
251 3.2.1..................................................................Use SAR (Sysout Archive Retrieval System)
282 3.3...............................................................................SUPPORT PRODUCTION RESPOOL REPORTS
293............................................................................3.4 ACCESS VPS AND VTAM MENU PANELS............................................................................................................................................................... 314 3.5..............................................................................................................RESET VPS PRINTERS
325 3.6...................................................ACQUIRE CREDIT UNION PRINTERS / TERMINALS TO A ZONE
3719 3.17....................................................................................DOCUMENTATION CHANGE CONTROL
37
20......................................................................................APPENDIX A - QUALITY RECORDS............................................................................................................................................................... 39
21..............................................APPENDIX B - COMMONLY USED DATASETS / FILENAMES............................................................................................................................................................... 40
22..............................................................................APPENDIX C - COMMON ABEND CODES............................................................................................................................................................... 41
23..........................................APPENDIX D - BATCH EXECUTION TRAINING PROGRESSION............................................................................................................................................................... 43
Page 3 of 42
Fon’s Batch Procedures
INTRODUCTION
3. PROCEDURES
3.1Provide Production Batch Cycle Monitoring SupportProduction batch cycles are monitored on the MVS operating system and UNIX system throughout each shift to ensure cycles complete according to customer requirements. Production batch cycle monitoring requires the execution of the following actions:
3.1.1 Perform Shift TurnoverBatch Execution operators perform shift turnover to communicate cycle status, outstanding issues, and any special instructions between shifts. This includes two processes: Prepare Shift Turnover and Review Shift Turnover. Where applicable, account specific turnover procedures are documented within account specific Batch Execution work procedure documents.
3.1.1.1 Prepare Shift Turnover1. Using TSO MON, update as necessary and complete by the end of the shift.
A) Type TSO MON from the ISPF primary option menu (SAIPC1A).B) Type the appropriate monitor number.C) Type the generation (0 is for the current date).D) Type the action taken (E (edit) or B (browse)).E) Document and detail all account status’s and special instructions.F) Document problems relating to the system.G) List any pending items. This is to ensure that the person taking over the cycle can follow-up
on pending items in an appropriate and timely manner. H) In the comment section of TSO MON, list discrepancies and the Ses that have been
contacted. This is to ensure that the person taking over the cycle will not call on the same item.
2. Update the paper Monitor log sheet to document individual cycle status and all account specific checklist items by the end of the shift. The Monitor log sheets (ICSBEX**) contain a list of the account(s) each operator is responsible for during that shift as well as specific account check-list items that are to be monitored. Day shift: using the appropriate monitor sheet, place an N/C and monitor initials next to the
cycles that are not scheduled for that day. All shifts: using the appropriate monitor sheet, indicate which cycles have completed by
initialing next to the cycle. All shifts, using the appropriate monitor sheet, initial all check-lists items to denote that
checklist items are addressed.3. Update the ABEND Worksheet form (IMSBEX01) with any outstanding ABENDs to be turned over. 4. Give verbal turnover to the oncoming shift
Discuss problems, special changes, or any related information. Answer any questions from the oncoming shift. Document the answers in turnover for reference when necessary for review.
5. Update / close all ABEND records.
3.1.1.2 Review Shift Turnover Read on-line turnover. Status the assigned cycles.
Page 4 of 42
Fon’s Batch Procedures
Check for outstanding ABENDs. Outstanding ABENDs should be documented on the ABEND Worksheet (IMSBEX01).
Check for previous ODATEs that have not started to execute. Receive verbal turnover from previous shift. Check Temp Change Members on SAIPC1x:
PSC1x.OACSC.TEMPCHN1 For changes effective on MondayPSC1x.OACSC.TEMPCHN2 For changes effective on TuesdayPSC1x.OACSC.TEMPCHN3 For changes effective on WednesdayPSC1x.OACSC.TEMPCHN4 For changes effective on ThursdayPSC1x.OACSC.TEMPCHNW For changes effective on Friday thru SundayNote: x = System ID. (A or B)
After receiving turnover, cycle monitors will use appropriate monitor turnover log checklists. Release the out-going cycle monitor. After turnover, the incoming cycle monitor will release the
out-going cycle monitor if all turnover items have been answered and all completed ABEND records have been closed. Written and on-line turnover must also be updated even if there were no problems to report.
File previous day’s Monitor log sheet, if completed.
3.1.2 Document Batch Cycle Phone MessagesTimely and accurate communication is essential to our ability to support our customer’s production batch cycles. Due to the importance of production batch related phone calls, minimum requirements for documenting these phone messages have been established and are listed below.
Who the message is for (cycle monitor) Caller’s name Caller’s account or customer name Caller’s remote site (if relevant) Caller’s phone number Description of the request/problem
Upon documenting the information above, the individual taking the message will promptly convey the information to the intended cycle monitor. Once messages are processed, they may be discarded. Phone messages are not considered quality records.
3.1.3 Monitor Production Batch Jobs With CA-7The CA-7 Commands Guide contains additional information about the commands used in these work instructions. The following examples can be used to assist with CA-7 instructions.
Using the Forecast facility in conjunction with the List commands will be a very effective way to monitor and status cycles in CA-7. The Forecast facility projects the scheduled workload for a given period of time. The List commands are used to show the status and modify the jobs in the different CA-7 queues. When entering CA-7 commands, the cursor needs to be above the function line. To get into CA-7 type E.7.7. from the ISPF menu.
3.1.4.1 CA7 QueuesCA7 uses different queues to monitor batch cycles. A few queues are detailed below.
Request Queue - When a job is scheduled to run. Ready Queue - When all requirements for the job are satisfied. Active Queue - When the job is executing in the system. Prior Run Queue - List the last time that a job executed.
Page 5 of 42
Fon’s Batch Procedures
3.1.4.2 Determine Job/Cycle Status with CA-7 List Commands1. Select option 7 and then option 1 from the Scheduling Products Menu.2. Type XQ,JOB=Y
Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Wild cards can also be used, such as WCBS* to show all WCBS jobs. This command will display jobs in the Request queue and allow the status of those jobs to be updated.
3. Type LQ,JOB=Y or LQ,ST=ABND,JOB=* Where Y= is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Wild
cards can also use used , as in the previous examples. This command will show the status on all production activity in CA-7. The second command will show you all the ABENDed jobs. This command is helpful in determining the queue in which a job currently resides.
4. Type LJOB,JOB=Y,LIST=ALL Where Y is any job cycle being monitored such as WCBSTCAR or MCAJCW01. Wild cards
can also be used as in the previous examples. This command will show all the information on a job such as in-conditions and prose.
3.1.4.3 Determine status of Time Jobs in CA-7This can be done by executing either command "LQ,JOB=TIME*" or "XQ,JOB=TIME*" in CA7. If there is a time job in the queue with a time that has already past for the current day, the cycle monitor needs to manually post the submit time requirement.
If there are system problems, check account cycle jobs(current julian date) that are waiting for a submit time through the PSI function QUEUES in the START TIME column.
3.1.4.4 Place Jobs on Hold with CA-7There are two ways to place a job on hold with CA-7. You can use the Hold command or go through the XQ screen.
1. Type HOLD,JOB=Y Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. CA-7 will
prompt you for text and the job will be held.2. The second way is through the XQ screen. Type XQ,JOB=Y
Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Wild cards can also be used, as in the previous examples. Move the cursor down to the job you want to hold and type an H in front of it. CA-7 will prompt you for text for holding the job. Type in your text and the job will be held.
3.1.4.5 Demand in a Job with CA-7When demanding a job in with CA-7 it is very important to know if there are jobs that are going to be triggered. It is a good idea to demand out a job in Hold status and then release it.
1. Type DEMAND,JOB=Y Where Y is any job you want to demand out.
2. Type DEMANDH,JOB=Y (To force out a job on Hold) Where Y is any job you want to demand out on hold.
3. Type DEMANDH,JOB=Y,SCHID=Z (To force out a job on Hold with a certain Schedule ID). Where Y is any job you want to demand out and Z is the Sched ID the job resides in.
4. Type DEMAND,JOB=Y,SET=NTR Where Y is any job you want to demand out and SET=NTR is to force out without any
triggering.
3.1.4.6 Delete a Job from CA-7 Production QueueUse the XQ screen to delete a job from CA-7.
Page 6 of 42
Fon’s Batch Procedures
1. Type XQ,JOB=Y Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Wild cards
can also be used, as in the previous examples. Move the cursor down to the job you want to cancel and type an C in front of it. CA-7 will prompt you for text for canceling the job. If the job is in a Skeleton status you will have to enter a Top-Line command: CANCEL,JOB=Y,REASON=____,FORCE=YES.
3.1.4.7 Restart a Job through CA-7/CA-11There are two ways to restart a job with CA-7.
1. Type RESTART,JOB=Y Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. This
command will restart a job awaiting restart in the Request queue.2. Type XQ,JOB=Y
Where Y is any job being monitored such as WCBSTCAR or MCAJCW01. Move the cursor down to the job to be restarted. Type an F in front of the job. This will pull up the CA-7/CA-11 restart panel. Place an X in front of the Resubmit For Production function. CA-7 will display a prompt for some documentation on why the job is being restarted. The cycle monitor can also force complete a job from here. Be CAREFUL when force completing a job because any follow on jobs will execute.
3. REQUEUE,Job=Y This command will requeue a job that is hung in the Ready queue.
4. To restart a job with CA-11 Type XQ,JOB=Y Where Y is any job being monitored, such as WCBSTCAR or MCAJCW01. Move the cursor
down to the job to be restarted. Type F in front of the job. This will pull up the CA7/CA11 restart panel. Place an X in front of the CA-11 RESTART/RERUN function. CA-11 will have the Jobstep and Procstep in the CA-11 restart fields. You can overtype these fields also.
3.1.4.8 Force a JOB Complete Using CA-71. Type XQ,JOB=Y Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Move the
cursor down to the job you want to Force Complete and type a F in front of it. This will take you to the Restart screen. Type an X in front of the FORCE COMPLETE function. CA-7 will prompt you for text on why you forced complete the job.. Be CAREFUL when force completing a job because any follow on jobs will execute.
3.1.4.9 Post a Condition to a JobUse the XQ screen to add a condition to a job in CA-7.
1. Type XQ,JOB=Y Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Wild cards
can also be used. As in the previous examples. Move the cursor down to the job you want to add a condition to and place an X in front of it. Cursor down to the condition that is to be remove and type an X in front of it.
2. Type ADDRQ,JOB=Y,DEPJOB=X Where Y is the job in the queue, and X is the job you want to add to Y as an additional
dependency.
3.1.4.10 List Documentation for a JobUse the LPROS command to list documentation for a particular job.
1. Type LPROS,JOB=Y
Page 7 of 42
Fon’s Batch Procedures
Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Wild cards can also be used, as in the previous examples. This command will show you the documentation the scheduler or programmer has provided.
3.1.4.11 Use the Forecast CommandsUse the Forecast command to forecast the scheduled work for a given period of time. Successor jobs triggered by predecessor jobs or any jobs triggered by data sets created in the predecessor jobs may also be included.
1. Type FJOB,FROM=(mmddyy,hhmm),TO=(mmddyy,hhmm),JOB=** This command will show you all the jobs that will run for a specific period of time by jobname,
or you can use wild cards as in the previous examples. You can replace JOB=** with SYS=** to forecast by System ID.
2. Type FRJOB,JOB=Y Where Y is any job cycle that is being monitored, such as WCBSTCAR or MCAJCW01. This
command will show a reverse job flow (reverse trigger flow) based upon information from the database.
3. Type FSTRUC,JOB=Y,SCHID=NN Where Y is the job you want to forecast from and NN is the schedule ID you are forecasting
from. This command will show you the cycle flow from the job you specify.
3.1.4.12 Use the List Log and List Prior Run Commands1. Type LRLOG,SPAN=*,JOB=Y
Where SPAN=* is the amount of days or time you want to list and Y is the Job you want to search for. This command will list information from the CA-7 Run Log. These events include job and network completions and exception events such as restarts, force completes, and cancels.
2. Type LPRRN,JOB=Y Where Y is any job being monitored. This command will list job information from the prior run
queue. The prior run queue contains information about the last successful completion of each job.
3.1.4.13 Use the HELP Function Through CA-71. Type HELP
This will take you to the CA-7 Help screen or the CA-7 Online Tutorial screenType in a Y at the INDEX: prompt. This will give you an Index of all the CA-7 commands you can
get help on. Overtype the word INDEX with a Command you want to get help on. Such as LQ or XQ. The
CA-7 Help will give you Syntax and Examples of commands.
3.1.4.14 Production Scheduling Interface (PSI) Access PSI
1. Enter “E” from the Primary Option Menu to enter the EDS Corporate Application screen.2. Enter “7” - Scheduled Products screen.3. Enter “P” - Production Scheduling Interface screen.4. There are several areas displayed , however you want to enter “1”for active queue information..5. Enter Job name and System name, or asterisk in both fields.
Queue Display
The information displayed on the screen is what is in the queue at that moment. Jobs will not display until the time they have been scheduled into the queue. In order to see a job you may have to go to
Page 8 of 42
Fon’s Batch Procedures
CA-7 Account Forecast utility. Do not rely on PSI to find the information you need use any other option available.
Sixteen different fields are displayed using two queue panels (use PF11 to view the second screen).Three of the fields ,”QUE”,JOB STATUS” and “ENTRY MODE” have variable entries as listed below. All other fields do not display variable information.
1. QUEUE- information displayed will be one of the following: REQ - Job is in the Request Que. RDY - Job is in the Ready Que. ACT - Job is in the Active Que.
3. JOBS WITH READY TO RUN STATUS IN PSI Check in SAM if job took JCL error Check SAM if job is waiting for initiator Check job in CA7 using RM.1 if the job is waiting for resources
4. JOB STATUS - information will be one of the following: ABENDED - Job has abended. DUP JOB -Duplicating job in the request Que. EXECUTING - Job is executing. HOLD - Job is on hold. JCLERROR - Job has JCL error. JOBFAIL - Invalid job card or JES card. READY TO RUN - Job ready to run in the Ready Que. REQUEUED - Job has been requeued. RETRY - Job in Retry status. SKELETON - Job in SKELETON Status. WAIT RQMTS - Job is waiting on submit time. Z-DOWN - ZOOM is inactive. Z-ERROR - ZOOM has encountered a severe error. Z-JCL FAIL - ZOOM could not resolve the JCL. Z-PENDING -CA-7 is notifying ZOOM to process the job. Z-QUEUED - Zoom has queued the job for processing. Z- RUNNING - Zoom is resolving the JCL. Z-SSCAN - JCL has been resolved and is waiting for CA-7’s schedule scan.
4. ENTRY MODE - Job entry mode has three possibilities as follows: DMND - Job was demanded. SSCN - Job was schedule scanned. TRIG - Job was triggered in.
4. LINE COMMANDS
Various line commands can be entered beside a job name to performauthorized functions. Since we have “read only” authorization the following commands are usable:B - Browse the JCL. If the job status is Z-JCL FAIL, the JCL is displayedfom the ZOOM error PDS to let the user view the ZOOM error messages in the JCL.Otherwise the JCL is displayed from the CA-7 trailer queue. Also look at the top rightcorner of the screen to view any message should the JCL not be displayed. For exampleif you place a “B” next to the job name and the screen does not change you may see themessage “SKELETON STATUS”.S - Displays job requirements. There are five different types as follows:
DSN - Data set requirement. JDEP - job dependency requirement.
Page 9 of 42
Fon’s Batch Procedures
HOLD - Job is on hold. TIME - Submit time requirement. User - User requirement. OTHER - Screen displays.
1. On the “Job Requirements Display” screen there is a field labeled “NEG”.If there is a “YES”, this indicates the requirement is a negative requirement,which is the same as a “conflict job”. A “NO” indicates the requirement is not a negative. 2. “REQUIREMENTS” - A description of the requirement. If “USER” and the requirement is added by PSI, ;you will see a short description ,the s.e. name andphone number.
ABENDS DISPLAY
The Abends function reads the CA-7 abend queue to display information about all jobssubmitted to JES that abends and has been requeued for execution. Jobs in the abend queue are there until they are restarted, forced complete or canceled using CA-7 commands.
A. Abend Codes - errors associated with jobs. JCLERROR - job ended with JCL error. JOBFAIL - Job contains an invalid job card or JES card. REQUEUED - -Job requeued for execution. Unnnn - User abend code where nnnn is the user code. Snnnn - System abend code where nnnn is the system code.
SKELETON PROCESSJobs which show in the STATUS field as skeleton have no JCL attached. The cycle monitor should contact the S.E. for that job and advise them to resolve the problem.
3.1.4.15 Search for Logon Ids for Jobs that were put on HoldSometimes a job in CA7 will be on hold. The batch cycle monitor can execute job SC1*RPT2 for a desired time period to locate the logon ID of the person who put a job on hold. The JCL for this job is in TSC1*.OACLB.JCLLIB(SC1*RPT2). Instructions for running the job are in the job’s JCL. Once job SC1ARPT2 has run, search the output and get the logon ID of the person who put the job on hold. CA7 and SC1*RPT2 will not show who placed a job on hold using the DEMANDH or RUNH command. The ‘*’ is to be replaced by the system in which the job will run. For example: for A side jobs you will need to run SC1ARPT2 and the JCL for this can be found in TSC1A.OACL.JCLLIB(SC1ARPT2).
3.1.4.16 Escalation Procedures when a Cycle is Running BehindWhen a cycle is running behind, notify the oncall S.E. The oncall S.E. will put jobs on hold or have you put jobs on hold for the next cycle day to ensure the cycle does not kick off until the previous cycle finishes.
3.1.5 ABEND ResolutionABEND resolution is a Batch Execution operator’s response to a job that has terminated abnormally. While monitoring production batch cycles, production batch job ABENDs are resolved as quickly as possible by providing proper notification, escalation and job restart support as necessary. Production job ABEND resolution requires the execution of the following actions:
3.1.5.1 Initial Response to ABENDS1. Gather the following ABEND information:
Job name Time of ABEND ABEND Code
Page 10 of 42
Fon’s Batch Procedures
ABENDed Job step / PROC step System Message(s) that pertains to the ABENDed job System on which the job was executing Other information the SE may request
2. Review documentation in CA7 and check the appropriate SYS4.SERESP member or TSO ONCALL member and the Batch Execution Account Procedure Manual to determine if the SE or Batch Execution operator has responsibility to correct the ABEND. Also check for any special instructions within in the job before calling the S.E.
3. Provide notification to the SE within 15 minutes or as otherwise documented. Whether the ABEND is resolved or not, proper verbal communication of an ABEND is important. If there are major problems, work with the most critical requirements first. Review the appropriate SYS4.SERESP member to determine ON-CALL Procedures. If appropriate, utilize the ONCALL Database by following the “Using ON-CALL SE Response Facility” section of this manual.
4. If Batch Execution is Responsible for the ABEND, attempt to resolve the ABEND a minimum of one time or a maximum of three times for the same ABEND, before contacting the responsible SE (unless otherwise specified by the Account).
5. If the SE is responsible for the ABEND, communicate the details of the ABEND and inform the SE of the actions taken. Notify the Primary On-call Person as documented within the appropriate Account Procedure
Manual or SYS4.SERESP. Unless indicated otherwise when calling a beeper, wait 15 minutes for a response. If no response is received, start the next step of the notification process. An answering machine is not considered a sufficient contact for ABEND notification.
If Primary SE can not be contacted, notify the Secondary on-call person as documented within the appropriate Account Procedure Manual or SYS4.SERESP
If the secondary can not be contacted, escalate to the next level as defined within the appropriate Account Procedure Manual or SYS4.SERESP.
Provide the ABEND information that was previously gathered to the SE who takes responsibility for the ABEND.
6. Once the SE is contacted, it is that SE’s responsibility to resolve the ABEND or make the additional contacts necessary to resolve the ABEND.
7. Perform the following work instructions as necessary: Code overrides, restarts, and checkpoint restarts as requested Reset PGENs as needed Provide System DASD information
8. Document the ABENDed Job within abend documentation facility.
3.1.5.2 Research / Resolve Common ABENDsBatch Execution Monitors often research and/or resolve common ABENDs. Listed are some research hints for common ABENDs as well as resolution procedures for jobs that have account documentation specifying which ABENDs the Batch Execution cycle monitor may try to resolve without contacting the oncall SE. Only restart jobs if the account documentation states that it is appropriate for cycle monitors to restart jobs without contacting the oncall SE.
For additional Common ABEND Codes, refer to Appendix C.
For more information regarding ABEND codes, refer to the MVS / ESA Messages and Codes Manual, the MVS / ESA System Messages, or utilize the on-line Quick Reference Facility (TSO QW). TSO QW is only available on SA19, SA20, or SA72.
3.1.5.2.1 Common Space ProblemsThere are different types of space problems. Generally, for most space ABENDs, double the space for the specified DDNAME in the JCL and restart the job from the ABENDed step or according to restart instructions. If the job ABENDs on two additional tries for a space problem, contact the oncall SE. Below are some common ABEND codes for space problems.
Page 11 of 42
Fon’s Batch Procedures
SB37 - No space available or specified space for ABENDing DD is not large enough for file. Double the space for the specified DDNAME in the JCL and restart the job from the ABENDed step or according to restart instructions. If the job ABENDs on two additional tries for a space problem, contact the oncall SE.
SB37-04 - During end-of-volume processing, this happened for one of the following reasons:1. All space was used for an output dataset, and no more volumes were specified.2. A volume was dismounted for one of the following reasons, and no new volume could be
mounted.a). No more space was available on the volume.b). The dataset already has 16 extents, and needs more space.c). More space was required, but the volume’s VTOC is full. The solution for this ABEND is to
increase the primary space quantity for the associated output dataset. If the dataset is not already in 16 extents, allocate the dataset’s space on another volume that contains more free space.
SB37-08 - The VTOC for the volume could not be converted to the standard format for one of the following reasons:1. Two datasets were allocated to the same space on the volume.2. A split cylinder dataset was located on the same cylinder as a non-split cylinder dataset. The
solution for this ABEND is to contact the System Programmer for assistance. SD37-04 - This is a space ABEND of a particular dataset that requires the dataset to be compressed
or expanded. Note: Be aware that if a dataset is to be compressed, no job or region can have the use of the dataset.1) Obtain the dataset information and the job’s access code from the JCL.2) Compress a dataset. There are several ways to compress a dataset. A few are listed here.
a). Type TSO OACQ in the ISPF primary screen, choose option 2 of the OACQ panel, and fill out the compress requested information.
b). Type 3.1 from the ISPF screen, choose option C and fill out the required information. SE37 - No space on volume (dataset is larger than number of volumes specified in the JCL). This is
on an output file.1. Check the JCL to see if the Vol parameter has a value in its volume count parameter.2. Increase the volume count parameter if it was coded, or put in an override if it was not coded.
Notify the SE so he can make the necessary change.3. It may be necessary to contact DASD oncall to request more space on a volume.
3.1.5.2.2 Code Override Statements for SMS SB37 ABENDs Involving DataclassesIf a job ABENDs with a SB37 space ABEND and it is coded with a dataclass parameter instead of a space parameter, follow these steps:
1. Obtain the data class name from the dataclass parameter in the step the job ABENDed. For example, DATACLAS=DCPSFB00.
2. Type in E.I from the ISPF Primary Options Menu to access ISMF.3. Select option 4 (Data Class) from the ISMF Primary Options Menu.4. Type in the DATACLAS name in the “Data Class Name” field.5. Select option 1 (Generate a List of Data Classes) for the “Select One of the Following Options”
field.6. Press the Enter key.7. From the Data Class List window, write down the values from the LRECL (column 5), AVGREC
(column 8), SPACE Primary (column 10), and the SPACE Secondary (column 11) fields. An example is displayed below.
LRECL
---(5)---
AVGREC
----(8)----
SPACEPrimary
----(10)----
SPACESecondary
-------(11)-------80 U 24000 12000
Page 12 of 42
Fon’s Batch Procedures
8. In CA7 edit the ABENDed job to code the override doubling the Primary and Secondary space values.
9. The format for the SPACE parameter is as follows: SPACE=(LRECL,(Primary,Secondary),RLSE),AVGREC=Units of allocation
10. Code the override: The override would look like the following:11. //PSTEPNAME.DDNAME DD SPACE=(80,(48000,24000),RLSE),AVGREC=U12. PF3 to save any JCL changes.13. Restart the job as appropriate.14. If the job ABENDs again, double the space again using the above procedure.15. If the job ABENDs for a third time, contact the oncall SE.
For more information on DATACLASS refer to the MVS/ESA-JCL User’s Guide manual.
3.1.5.2.3 Job Failed - JCL ErrorThis is mostly non-JCL syntax errors such as missing datasets or inaccurate dataset names. Research what type of error was received by checking the sysout. Determine the type of error that occurred. For example,. Dataset not found, UNIT field specifies incorrect device name, DISP field incompatible with dataset name, and so forth.
3.1.5.2.4 Job Not Run - JCL ErrorThis is generally a JCL syntax error. (For example, missing comma) some accounts appreciate your fixing these ABENDs and resubmitting the job.
1. Check the sysout to locate error message(s).2. Go to the system messages section and find the statement # where the error occurred.3. Find the statement # in the JCL section of the job.4. Overtype the physical error or correct the error by using an override.5. Resubmit the job from the top or restart according to restart instructions..
3.1.5.2.5 Migrated DatasetsSometimes a job ABENDs because the dataset has been migrated.
1. This is usually easy to fix by doing a 3.4 on the dataset and selecting it, you will see the volume on change from migrated to a DASD volume.
2. You can also recall the dataset by typing in Hrecall beside the dataset in 3.4.3. Then restart the job according to the restart instructions.
3.1.5.2.6 Tape ErrorsIf jobs ABEND due to EQC, contact the RMS group and request that the tape drive be monitored and possibly varied offline. RMS will check to see if the drive is having problems. If necessary, RMS will vary the drive offline and call out for maintenance. If jobs ABEND due to data checks on the same drive multiple times, the drive may be causing the datachecks and RMS will need to be notified.
If there is an identifiable tape error on an output tape, restart the job in the ABENDed step, from the last datacheck, or per restart instructions. If the error occurs on input tape, it could be bad for different reasons.
Wrong DSN - Verify that the correct tape was loaded, by checking the DSN that was read by the tape, see messages in the JESMSGLG, have tape pool find the duplicate if option 3.4 shows that DSN was correct.
Bad Label - SE will have to recreate tape or use backup. Bad Blockcount - Have tape pool run a checktape to verify the blockcount. Bad Data - Have SE recall the backup tape if there is one or the SE will need to recreate the tape. Bad Equipment (Equip Check) - clean drive, run tape on another drive.
Page 13 of 42
Fon’s Batch Procedures
Data Check - Clean tape, clean drive and rerun.
3.1.5.3 Abend Tracker DatabaseThe abend tracker database is used to gather information from the system and through operator intervention, about batch job Abends. The database will record specific information and store it for reporting and Abend resolution.
3.1.5.3.1 Sign into the database
1. Open MS Internet explorer.2. Type the URL http://sam.sac.dsmc.eds.com/abends Into the address box.3. Press the enter key.4. If you are not at the Abend tracker sign in screen, see your team lead. You may need to change
some of the settings on your browser.5. Choose your name from the list.6. Enter bexec1 as the password.7. Press the enter key.
3.1.5.3.2 Monitor abend entriesThe abend tracker is set up for both specific cycle monitoring and for team leads to monitor the escalation times of the entire shift. By selecting the "view all open abends" option, team leaders can monitor all of the abends for all of the cycles.
1. Click View specific open abends.2. Type the cycle numbers of the monitor sheets you are monitoring.3. Click View abends.4. Restart the job or call it out to the appropriate oncall.5. Click on the jobname of the entry you want to update.6. Under the escalation date and time, click Now or fill in the current time.7. Click the SE contacted button.8. Fill in the details and information section.9. Click the Submit changes button.
3.1.5.3.3 Mark an entry resolvedEntries can be marked resolved when the SE restarts the job, When an operator restarts the job, or when a job is forced complete.
1. Click on the jobname of the entry you want to update.2. Under the resolution date and time, click Now or fill in the current time.3. Click the abend resolved button.4. Click the Submit changes button.
3.1.5.3.4 Secondary escalationCritical jobs that abend and are not restarted within two hours are required to be called out to a secondary source. Most of the time these abends are called out to SE team leaders or managers to promote timely correction of the problem.
1. Click on the jobname of the entry you want to update.2. Under the escalated to, select the appropriate selection from the list.3. Under the escalated other, type in the name of the secondary escalation person.4. Click the Submit changes button.
3.1.6 Use On-Call SE Response FacilityThe On-Call System enables the cycle monitor to display the responsible SE for a job or a group of jobs. There are a few ways to access the oncall facility. One way is to type TSO ONCALL from the ISPF command line on the A and the B complexes. Another way is to type E.OC from the ISPF command line. Within the On-Call SE Response Facility there are six functions an operator can do.
1. SEARCH List persons responsible for a job.2. JOB List, add, update, and delete job record.3. PERSON List, add, update, and delete person record.4. GROUP PAGER List, add, update, and delete pager records.5. REPLACE PERSON Replace person code in job records with new code.6. SYSTEM RECORD Maintain System information records for OUTCALL processing.
The SEARCH function allows the operator to list the responsible SE for the job. To display the list, enter the jobname in the field “Job name” and USERID in the “System Name” field from the On-Call System Menu. In the option field place a 1 and press Enter. Call the SE in the order that the numbers are listed. When using the SEARCH function and after entering the jobname, additional information may be obtained by using the primary commands DOC and JOB. DOC will browse for system documentation. JOB will browse for job documentation.
The JOB function will list all job records that are in a System ID . Using this option is helpful when the messages “DUMMY Record” or “ERROR Invalid Job Name” are displayed in the Primary or Secondary SE fields when using the SEARCH function. These messages may appear when an account has failed to update the oncall database past the previous day or the cycle monitor entered the wrong job name or system name. Select the latest jobname entry to find the last oncall SE. Note: The Cancel command may need to be used to exit a job record rather than PF3.
The PERSON function will display all SE’s for a particular System ID. This function can be helpful when trying to locate an SE’s phone number that is not oncall.
The options 4 through 6 are for SE use only. For more information, refer to the On-Call Users Guide.
3.1.6.1 Contacting Batch Scheduling oncallTo contact batch scheduling oncall using TSO ONCALL facility: log on to SA19 and enter TSO ONCALL. Select option 1 (search) at the command line, then enter “all” at the jobname field and “sc1apcg” at the system name field.
3.1.7 Code Basic JCL For ABEND resolution, it is often necessary to edit the job’s JCL. Some basic JCL coding is documented below.
3.1.7.1 Reset PGENs1. When a job ABENDs, always check for PGENs that will need to be reset.2. All PGENs that are defined prior to the ABENDing step and later referenced after the ABENDing
step must be reset.3. If the job is creating more than one generation of the same data set, (for example, +2, +3) extra care
must be taken before restarting the job. Find which generation was created and reset all following
Page 15 of 42
Fon’s Batch Procedures
PGENs accordingly. For example, a data set was created with a +2 in PS020 and the job ABENDed in PS030. The same data set is being created with a +3 in PS040. Therefore, reset the +3 to +1, the +2 to 0, and the +1 to 1.
4. If the number of PGENs that need to be reset continue to a second line, be sure to code // in card column 1 and 2 then add additional PGENs beginning in card column 16.
Example: //JS010 EXEC PROCABC,PGEN010=0,PGEN020=0,PGEN030=0, // PGEN040=0NOTE: Be sure to code the PGEN the way the job has it coded (e.g., PGEN010 or PGEN10)
3.1.7.2 Code Override StatementsAn SE may request that the operator code JCL overrides in an ABENDed job. JCL overrides are used to change particular statements in JCL without changing the JCL member. For more information about overrides refer to the MVS/ESA JCL User’s Guide.
1. Overriding statements must appear in the same order as the statements that they are overriding.2. Added statements must appear after all overridden statements
(//PSTEPNAME.DDNAME DD (Parameters))
Example override JCL://JOBNAME JOB (ACCTCD,BOX),’PROGNAME’,
3.1.7.3 Code CA11 RestartsFor more information about restarts, refer to the MVS/ESA JCL User’s Guide.
1. Edit the job’s JCL.2. Code the Restart Card in the Job Card. RESTART=JSxxx.PSxxx3. Reset PGENs (see “Reset PGENs”) if the job uses them.4. Save the changes. NOTE: Be sure to save the changes at this point because using JEM sometimes
exits the operator from the member resulting in all changes (if not saved) being lost.5. Use JEM on the JCL to verify coding is correct.6. Exit out of the JCL.7. Restart the job.
3.1.7.4 Code Checkpoint RestartsThe SE will sometimes set a job up to take checkpoints when the job runs for a long time. This allows the job to be restarted at the checkpoint instead of from the beginning. For more information about checkpoint restarts, refer to the MVS/ESA JCL User’s Guide.
1. Edit the job’s JCL.2. Code the Restart Card in the Job Card. RESTART=(JSxxx.PSxxx,Cxxxxxxx)3. For multi-system complexes, code the “SYSAFF” card to point to the same system where the job
ABENDed. (/*JOBPARM SYSAFF=xxxx ).
Page 16 of 42
Fon’s Batch Procedures
4. Code the SYSCHK DD statement before the jobstep to be restarted in. It must follow the JOBLIB DD statement if the JOBLIB DD statement is present.
5. Even if the checkpoint DSN is catalogued, the following must be coded: Absolute generation Disposition Unit VolserIf the checkpoint data set is multi-volume, the SYSCHK DD Statement must use the VOLSER that contains the desired check point entry.
6. Reset PGENs if the job uses them, refer to section “Reset PGENs”. If the job has been step-restarted earlier and PGENs were overridden, do not change or add to the PGEN override.
7. Use JEM on the JCL before submitting the job to verify coding.8. Exit out of the JCL.9. Restart the job.
EXAMPLE : Checkpoint-restart of a job that was step-restarted earlier in PS080. //JOBNAME JOB (ACCTCD,BOX),’PROGNAME’,// CLASS=H,// MSGCLASS=6,// RESTART=(JS010.PS080,C0000020)/*JOBPARM P=ABCDEF,S=231A/*OUTPUT STD D=NONPRINT//JOBLIB DD DSN=xxxxx.xxxxx.PRODLIB,// DISP=SHR// DD DSN=xxxxx.yyyyy.PRODLIB,// DISP=SHR//SYSCHK DD DSN=Absolute generation on the Dsn of the last checkpoint,// DISP=(MOD,KEEP,KEEP),// UNIT=TAPE,// VOL=SER=Fxxxxxx//JS010 EXEC PROCABC,
PGEN010=0,PGEN020=0,PGEN030=0
3.1.8 Monitor and Follow-up on Late Cycles1. While monitoring your assigned production or model office batch cycles, the following can help to
determine if jobs are late: Late shout messages Account Procedure Manual CA7 list prior run or LJOB command
2. When a shout message comes up, check the job in CA7 to determine if the job is running or waiting to execute. If the job is waiting to execute, investigate why. Do we need to escalate? If the problem is due to an ABEND, is the SE working on the ABEND aware that it is holding up critical jobs?
3. All shout messages require investigation! Inappropriate shout messages must be reported to the Batch Scheduling group for correction; see “Document Late Shout Message Instructions” section.
4. Implement whatever corrective action is necessary to help the cycle catch up.5. Consult the Account Procedure Manual to determine which late jobs are critical.6. Several Monitor sheets have a list of the critical path jobs listed on them. These jobs should be done by a
specific time or the customer will be impacted. If the job does not complete on time, a Vantive Case record will need to be opened documenting the problem. Note: No need to open a late load ticket if the cycle is requested to run late by the SE or customer as long as there is no customer impact.
7. If a critical job is late, create a Case record in Vantive. Document the reason the job was late and assign the record to the RTI group for follow-up. Describe the reason why a problem ticket is not necessary and document it in turnover.
Page 17 of 42
Fon’s Batch Procedures
8. When a critical job or cycle is late, the operator monitoring that cycle must leave a voice mail for the account’s Business Support Representative and contact the appropriate SE, unless specified differently in the Batch Execution Account Procedure Manuals (IPSBEXB0 and IPSBEXC0).
9. If system utilization is near 100% and there is minimal tape usage, batch may request that RMS bump performance on a critical job.
3.1.9 Investigate Pending Tape Mounts Using AOS, watch for outstanding tape messages throughout the shift. If tape mounts are not
answered efficiently, it could affect critical path jobs such as on-line load and critical reports to the customer.
If a tape mount is pending over 15 minutes, ascertain the impact to the production batch cycle and follow-up with Tape Operations as appropriate.
If the tape cannot be found or is located at the DRA site, contact the appropriate SE. Typically, the job will be canceled - DO NOT CANCEL A PRODUCTION JOB WITHOUT CONTACTING THE SE FIRST.
3.1.10 Update Job Performance GroupsThere are several performance groups that jobs may be assigned. It is important to understand the impact of changing a job’s performance. Only when the system’s CPU utilization is over 85% will the performance of a job have an effect. For jobs using heavy tape usage, changing the job’s performance will not have much of an effect.
Performance Group Description Special Notes013 To swap out the job. The job will not take any I/O or CPU EXPCTS160 Slow Batch Lower priority260 Normal Batch280 HOTTER Batch Recommended that no more than 6 jobs be
changed to this performance group per system.
To change a job’s performance, contact an RMS operator. Batch Execution no longer has authority or permission to change a jobs performance.
3.1.11 Perform HOLD, RELEASE, CANCEL and PURGE Functions in SAMUse the hold, release, cancel, and purge functions through SAM in support of these types of batch processing requests.
Refer to the System Activity Manager Users Guide under section ‘JES2 Line Commands’ for the hold, release, cancel and purge commands.
3.1.12 Use SAM to Monitor Looping Production Batch Jobs A looping job is one that continues processing the same instruction(s) over and over without taking in any new input or generating any output. Generally, the symptoms of a looping job will be that it is not taking any I/O or increasing EXCPs, and it may be taking a high percentage of CPU. Depending on the performance group of the looping job, it can impact the system’s performance severely. Refer to “Update Job Performance Groups” section.
1. Access SAM and use the ‘DR’ (resources) option and check the SIO and the EXCPCNT fields. If the fields are not incrementing, the job may be looping.
2. Check the job to see if it is processing within DB2. Status it within SAM. Then select the job. Make sure you are on the same system that the job is
executing on. Check the jobstep that is executing for the program and datasets that are coded. If they reflect
DB2 data set names, consult with the RMS group.
Page 18 of 42
Fon’s Batch Procedures
If the job is verified to be executing within DB2, do not proceed further.3. If the job is NOT processing within DB2, refer to procedure “Monitor Production Batch Jobs With
OMEGAMON.4. If the job is confirmed to be looping, notify the appropriate on-call SE for necessary action.
3.1.13 Use SAM to Monitor Swapped Out Production Batch JobsA swapped out job is a job that is not performing I/O but will retain resources. The job sits idle in the work queue.
1. Display the job’s resources in SAM.2. Look at the POS (current swap position of the job).
If the POS field is displaying a <> sign, the job is rewinding or forwarding a tape If the POS field shows “N/S,” it is waiting on a tape mount or WTOR. If the POS field shows “IN,” the job is executing.
3. Look at the SWP (swap reason code - see System Activity Manager Users Guide). If any of those conditions exist, display the job to see when JES was last updated.
4. If the job’s JES log was recently updated, continue to monitor the job.5. If the job’s JES log has NOT been updated within five minutes, perform the following steps:6. Check for outstanding WTORs.7. Check for outstanding tape mounts.8. Check for data set contention. Refer to sections “Monitor Production Batch Jobs with
OMEGAMON” and “Utilize the XENQ Display Facility to Research Contention Situations” for researching dataset contentions.
9. Take appropriate action.
3.1.14 Use SAM to Perform Modifications to a Job’s Output CriteriaWhen working within the JES2 queues, an SE or an output processing center operator may request that the output criteria be changed. To perform this function, utilize one of the following two methods:
Through SAM, use the JES2 command to change the output criteria (e.g., /$TOJxxxx,OUTGRPxxx,D=xxx,F=xxx).
Access SAM to display the output information (DQO) and change the output criteria as requested.
3.1.15 Use SAM to Display Initiators, WTORs, and the MVS System Log Use the DI command to display initiators. If any closed initiators should be opened, contact the
Resource Management group for assistance. Initiators cannot be opened without approval from the Resource Management group.
Use the DOR command to display outstanding WTORs. Answer appropriate WTORs through AOS (Routed to the correct system).
Use the DL command to access the MVS system log. SAM also allows the selection of a different MVS system log by specifying the S= selection criteria (or from within the log, enter a “?” from the command line then the desired system).
3.1.16 Use SAM to Display JES2 Queue, JES2 Held/Non-Held Output InformationThere are three different JES2 queues that SAM uses:
Use the DQ command to display the input queue The input queue displays jobs that are waiting execution (waiting for an initiator), jobs that are
executing, and jobs that have already run. The DQ information allows the operator to find the status of a job. Throughout the shift, an operator should view the JES2 queue for information to help with an ABEND or per an SE request.
Use the DQO command to display the output queue. The output queue displays all the job’s generated output reports (e.g., JESLOG information).
Page 19 of 42
Fon’s Batch Procedures
Use the DQH command to display the held queue. An SE or an operator at an output processing center may request a job’s output to be placed on hold. Route the reports to the DQH queue. Jobs that have been placed in the DQH queue will remain until: a) An SE or operator from the output processing center releases it;b) An SE or operator from the output processing center requests the jobs to be released;c) Resource Management Group deletes output to free up spool space after it is determined that the
jobs no longer need to be on hold.
3.1.17 Monitor Production Batch Jobs Using OMEGAMONOMEGAMON is a software subsystem designed to monitor real-time performance (it displays the information as it occurs). It runs as a started task and monitors all activity in an MVS or VM environment. Use OMEGAMON as a tool to help detect jobs that are swapped out or appear to be in a loop. Refer to the OMEGAMON MVS Volume I User’s Guide and / or OMEGAMON for CICS Volume 1 Status Monitor OMEGAMON Users Guide for more information.
1. Access OMEGAMON. There are several ways to logon to OMEGAMON, here are a few ways.a) From the TSO primary menu select option “E.9;O” option #3 ORb) From the Netmenu screen, signon to the OMEGAMON Status Monitor ORc) Log directly into OMEGAMON from the Welcome to EDSNET screen, type in : LOGON
APPLID(ATFCICSM) LOGMODE(D4A32XX3)1. For Options 1b and 1c:
a) Press ENTERb) Type in your NetID and passwordc) Type in $MVS_ALL on the Startup Profile field to start all sessions. Note: Make sure you
use the underscore between MVS and ALL.d) Press ENTER
1. Display the “Job Screen.”a) For 1a above, issue the command “#systemID” (e.g., #19) or enter “DEXAN” (an
OMEGAMON subsystem command). For 1b and 1c above, issue the command “Z and the number listed on the screen for the correct system (e.g., Z1). Typing a “Z” next to the system id will also work. The commands for 1a will also work for 1b and 1c access. The following conditions can be analyzed from these screens:
Looping batch jobs Contention Swapped out jobs Bottlenecks Within the “#systemID” screen, ensure that the Major status command of “BSWP” is issued
frequently. Any job that is currently in a swapped time, the job has been swapped out. If any job has been swapped out for an inappropriate amount of time, take appropriate action.
b) The job screen “QUICK” will display jobs executing, the amount of time each job has been executing in the system, and all the information stated above is listed here.
c) The two (2) minor commands under BSWP are “SWPR” and “WAIT”. “SWPR” will state the type of wait the specific job is in. Reference OMEGAMON II for
MVS Volume 1 User’s Guide for clarification on each type of swap. “WAIT” shows the amount of time a job has been swapped out. Use this screen for further
problem analysis, if need be.
Note: The ENQUEUES Display Facility (in TSO option E.Q) can also be used to research contention situations.
4. If any of the conditions described above exist, contact Resource Management and other necessary personnel.
Page 20 of 42
Fon’s Batch Procedures
3.1.18 Use the XENQ Display Facility to Research Contention SituationsXENQ is an enqueue display that will show jobs that are in contention with each other and is used to research contention situations where appropriate.
1. From the ISPF primary option menu, enter E.Q to bring up the XENQ display.2. To see jobs that are in contention, enter a “Y” in the conflict display field and press enter. This will
show all jobs in contention on that system.3. To see all jobs that are in conflict, leave Jobname and R-Name blank.4. After the information has been keyed, press enter. The display will come back showing jobs in
contention with the data set entered.
The display will look like the following:
Enter requested enqueue information below:
Jobname: Q-name: SYSDSNR-name : Data set nameRequest information from grs N |
Enter Y in appropriate fields to request enqueue conflict data:
Conflict display (Y,N) Y extended (Y,N)
3.1.19 Monitor Transmissions Through Network Data Mover (NDM)Use NDM to view transmits from other sites to see if the file successfully arrived at the appropriate time. Different sites will transmit at different times throughout the day.
3.1.19.1 Sign on to NDM1. Sign on to the SAIPC1A TSO (SA19)2. Exit ISPF Primary Option Menu to initial ISPF ready screen3. Enter ‘NDMCB’, ‘NDMUP’, or ‘NDM’ according to customer requirements4. Enter ‘U’ for User Application5. Enter ‘N” for NDM6. Enter your TSO UserID and Password7. You will now be at the NDM Primary Options Menu
3.1.19.2 View NDM Transmits In Process1. Enter ‘SP’ for Select Process2. At the command level, type ‘D’ for Display3. At process name, type in user’s job name4. Queue shows the current status5. RECS shows total transmitted at that time6. If there is no transmissions in process, the message - “No Process Found...” will be displayed.
3.1.19.3 View NDM Transmits Already Ended1. Enter SS for viewing statistics for completed NDM processes.2. At the command level, type “D” for display.3. At process names, type in user’s jobname.4. At the start date type “TODAY: or the current ODATE in MMDDYY format.5. Look for the words ‘PROC-END’ under the MSGID heading. There will be a end-time listed 2 lines
under it. This is the time that should be entered in the NDM log. Note: If there is no ‘PROC-END’
Page 21 of 42
Fon’s Batch Procedures
message or if there is a “*” next to the PROC-END message then the NDM transmit has not finished! A successful NDM transmit will have the word “PROC-END” with no “*” next to it.
6. If there are any “*” displayed to the left of the MSGID fields then the NDM transmit has received an error!!!! To determine the cause of the error, type the letter “S” next to the “*”. A new screen will be
displayed. There should be a message on the first line stating the error such as “Send of an RU failed”.
The return code can be found about 5 lines down on the very right hand side of the screen in the “RC:” field. This may be helpful.
If you look about 7 lines down from there, the word “NODE*” with the name of the node taking the error should be displayed. If the word “OTHER” appears next to the word “NODE” then NDM is telling you that NDM has taken an error at another site.
Check the NDM transmit sheet and contact the appropriate site (Dayton or Des Moines) if the NDM transmit does not get resent or restarted. The other site may request that you re-run the MVS job. This is OK. A successful resend or restart of a NDM transmit will have the word “PROC-END with no “*” next to it.
3.1.20 Cancelling Jobs in IMS on 231AFollow the steps below when cancelling a job that is running in IMS on 231A:
From TSO Primary Option Menu on 231A, enter L for Local Apps, O for Online IS Tools, and I for the IMS/VS Selection Panel. These options can be run together from the primary option panel as L.O.I.
From the IMS/VS Selection Panel, enter the IMS system the job is running on (IMTH, IMMH, or IMPH) in the blank at the upper right corner, and select option A which takes you to the Primary Option Menu.
Enter I which takes you to the IMS Operator Workstation, then 6 to take you to the Display/Modify IMS Regions screen, and place a ‘C’ next to the job you wish to cancel.
3.2 Use $AVRS$AVRS is the output queue used by the Philips account to store sysout. Use $AVERS to view archived sysout while troubleshooting problems or assisting customers.
To get into $AVRS from TSO: Type A.8 (if your logon concatenation contains Philips account libraries), otherwise, from the SIG panel, type “2”. The following panel will appear:
TYPE MAY BE: SN-STC J# - JOB# JN-JOBNAME AH-ARCH.HIS ST-ARC.STAT
NAME THRU: RLSE 40A-10TYPE DATABASE USAGENUMBER CLUSTER 23.6%START DATE INDEX 6.3%END DATE DATA 63.4%START TIME END TIME RETURN CODE SORT FIELD A or D TUE 12 JUL 94.193Item descriptions:
1. NAME- Philips jobname or started task name.2. THRU- for specifying a range of jobs.3. TYPE:
AH- Archived History JN- specify for jobnames SN- specify for started task J#- specify for job number
4. Number- for specifying an individual job number.
Page 22 of 42
Fon’s Batch Procedures
5. Start date- start date when to retrieve job. If start date specified alone the range will be from date specified to current date. Need to use Julian date format. - i.e.- Start date 92343 or the date in MM/DD/YY format.
6. End date- end date when specifying a range of dates. Need to use Julian date format. I.e. - range of dates: start date 92343 this range will retrieve 5 end date 92347 days of output or the date in MM/DD/YY format.
7. Start time- when specifying a start time. Need to use military time. Code the start time when you want to retrieve output starting at a specified time up to the current time. When start time coded with end time then a range of time is requested.
8. End time- when specifying an end time. Need to use military time.9. Range of times: start time 1700 This range will retrieve out- end time 2300 put from 5 PM
to 11 PM.10. Return code- when retrieving output by return code.
S0C4 - to retrieve all jobs that ABENDed with a S0C4. Other specific ABENDs can also be specified.
s**** - to retrieve all jobs that have system related ABENDs. nzero - to retrieve all jobs that did not complete with a return code zero.
11. Sort fields - sort the output (ascending/descending) by using the following codes. SD- by start date ED- by end date RO- by room CC- by return code. CPU time- by CPU time.
12. Ascending/Descending- D- represents fields that will be sorted in ascending order. D- represents fields that will be sorted in descending order.
SEL# ACT FILE C FORM FCB UCS DESTID STEP PROCSTEP LINES
_ 1 JESLOG 7 STD SIGSAV JES2 27
_ 2 JCL 7 STD R826 JES2 56
_ 3 SYSMSG 7 STD R826 JES2 107
_ 4 RMSRPT 7 STD R826 RMS@2X RMS@2X 83
_ 5 SYSPRINT 7 STD R826 JS020 BTERM 21
******************************** END OF DATA **********************************
Any of these 5 files can now be selected. The advantage of the ‘S’ option is that you can retrieve selective information without having to browse through the entire job listing.
P=PRINT- to print job listing.
Example of print option:
B=Browse,S=File,P=Print,I=Info,D=Delete,OR O=Output #Lines Total=636 SEL # ACT Jobname Job# Start Date Time CPU Time ELAPS CCODE P 1 SIGBUA13 0638 08 DEC 92.343 0400 0.00.29 0 RC=00 I=INFO- Brief Description Of The Run Of The Job. This option is similar to the browse, but
does not get as detailed.Example of info option:
B=Browse,S=File,P=Print,I=Info,D=Delete,OR O=Output #Lines Total=636 SEL # ACT Jobname Job# Start Date Time CPU Time ELAPS CCODE I 1 SIGBUA13 0638 08 DEC 92.343 0400 0.00.29 0 RC=00 D=Delete- to delete job listing
Example of delete option:
B=Browse,S=File,P=Print,I=Info,D=Delete,OR O=Output #Lines Total=636 SEL # ACT Jobname Job# Start Date Time CPU Time ELAPS CCODE D 1 SIGBUA13 0638 08 DEC 92.343 0400 0.00.29 0 RC=00O=ouput-output data set
allocation B=Browse,S=File,P=Print,I=Info,D=Delete,OR O=Output #Lines Total=636 SEL # ACT Jobname Job# Start Date Time CPU Time ELAPS CCODE O 1 SIGBUA13 0638 08 DEC 92.343 0400 0.00.29 0 RC=00
For more detailed information about $AVRS press PF1 for the tutorial, within ISPF.
3.2.1 Use SAR (Sysout Archive Retrieval System)SAR is used for on-line report viewing and reprinting. It performs the following functions:
Page 24 of 42
Fon’s Batch Procedures
Store (archive) job sysout and reports based on user specifications. Retrieve archived output making it available for on-line screen viewing or reprinting.
It is currently used by Del Monte and Hitachi Data Systems.
SAR for Del Monte:1. At the Netmenu screen look for application FFSAR12. Select the number for the application(for example it might #6 or your netmenu)3. Enter: 6,'pdmfc.zsadb.dbase'4. Determine whether you are in Mode: SARO or Mode: EXP5. You want to be in Mode: SARO6. If you are in Mode: EXP, then Enter: mode saro7. At SYSOUT ID you can enter a full jobname or something like DMFC* to get all jobs that begin with DMFC8. You can use Generation and Date with a specific number or with an * to indicate all.9. When looking at a specific job, it is recommended that you enter the jobname and then place an * at Generation. That way you'll see the current day and past days when that job ran.Example SAR screen:
COMMAND =1) SYSOUT ID = JOBNAME/REPORTID
MODE: ALL
SELECTION CRITERIA:
2) GENERATION =ALL
SPECIFY N FOR ABSOLUTE GENERATION NSPECIFY N:M FOR ABSOLUTE GENERATIONS N THRU MSPECIFY -N FOR RELATIVE GENERATION -NSPECIFY -N:M FOR RELATIVE GENERATIONS -N THRU -MSPECIFY * OR ALL FOR ALL GENERATIONS
3) DATE =SPECIFY MM/DD/YY FOR SPECIFIC DATESPECIFY MM/DD/YY:MM/DD/YY FOR DATE RANGESPECIFY -N FOR RELATIVE DATE -N DAYS AGO
SELECTION ONLY SPECIFY TO RESTRICT SELECTIONOPTIONS:EXCEPTIONS = X EXCEPTIONS ONLY, NX NON EXCEPTIONS ONLY, AX/(BLANK)ANYPERMANENT = P PERMANENT ONLY, NP NON PERMANENT ONLY, AP/(BLANK)ANY
ENTER END COMMAND TO TERMINATE THIS SAR 6.1 SESSION.
3.3Support Production Respool ReportsA Respool is a reproduction of a report. It consists of coding JCL to produce a report based on customer backup files. Output Processing Centers may request an operator to respool a report.
1. Obtain the name of the person requesting the respool.2. Obtain the jobname.3. Obtain the form or forms that need to be respooled.4. Obtain the location or destination to which the jobs should be sent.
Page 25 of 42
Fon’s Batch Procedures
5. Obtain the generation to respool.6. Determine if the SE or Batch Execution operator is responsible for doing the respool by checking the
appropriate Batch Execution Account Procedure Manual.7. Check for the original job in SAM for a WAAPSPLT step. Verify the executable job is loaded out in
CA-7 for requested resdate. If it does not, contact the oncall SE since batch execution will be unable to do the respool.
8. Edit TSC1A.OACLB.RESPOOL.9. Use either of the following work instructions below to continue.If a Respool Exists
1. Locate and select the appropriate jobname.2. Check the access code Woodland should be ZLA1MM and Micrographics should be ZLA1MM.3. Verify the OUTPUT cards of the job in the TSC1A.OACLB.RESPOOL library with the OUTPUT
card of the job that was loaded out in CA-7 or last job that executed in SAM, if SYSOUT exists.4. Verify the data set name field of the job in the TSC1A.OACLB.RESPOOL library with the data set
name field of the job that was loaded out in CA-7.5. Go to the top of the job and scan for the word “EXEC” or “WAAPSPLT”. Do a find on the
appropriate forms. Next do a find on the appropriate locations, if specific locations were requested. The only EXEC statement should be the WAAPSPLT statement. If there are any other EXEC statements, do not submit the respool!!! Instead the respool member should be deleted and a new one created. Running a respool with an EXEC statement on any other PROC or PGM besides a WAAPSPLT can cause serious damage.
6. Go to the top of the respool and scan for “+”. The respool should not be creating any new datasets. Note: (+0) is all right to have. If the respool is creating new datasets, the respool member should be deleted and a new one created. Running a respool that creates new datasets can cause serious damage.
7. If everything matches, submit the job.8. Check the output queue to verify the respool has completed successfully. In SAM, move to nonprint,
R99 or delete any unnecessary forms.9. Notify requester of successful respool.If No Respool Exists
1. To create a respool member, one way is to enter S xxxRxxxx on the command line. Where xxxRxxxx is the job name to be respooled with a “R” in the fourth character. The fourth character may also contain “#” , “@” or “L” or any other unique character.
2. Copy the job’s executable JCL, master JCL, or PROC from the RESPOOL library.A. A few ways to locate a job’s JCL are listed below.
If the PROC JCL needs to be pulled in, one way to obtain the PROC JCL is to change the executable JCL dataset. The first node would remain the same, the SC in the second node would be changed to CH, and the third node would be PROCPDS or JCLPDS. An example would be PSC1A.OACCH.PROCPDS(PROCname). The PROC name is obtained in the JCL EXEC statement. Depending on the type of respool, the PROC JCL will not always need to be used.
B. One way to copy in the JCL is listed below. In the newly created respool member:1.) Type COPY on the command line. If step 2 is skipped, hit enter.2.) (Optional) Type “A” in the line command line to designate where the copied information
should begin. Hit enter.3.) Fill out the name of the data set where the JCL is being copied from.
For example: DATA SET NAME = PWCBS.HRBCH.JCLPDS(WCBSTCA9)C. Check for the WAAPSPLT step after the JCL has been copied in by doing a find on
WAAPSPLT. If no WAAPSPLT step is found, copy in the first PROC that is executed the JCL. It would look like the following: //JS010 EXEC WCBAC500. One way to copy in the PROC is listed below.1.) Type COPY on the command line.2.) Type an “A” in the line command line just after the EXEC statement to designate where
the copied information should begin. Hit enter.3.) Fill out the name of the data set where the JCL is being copied from.
Page 26 of 42
Fon’s Batch Procedures
For example: DATA SET NAME = PWCBS.HRBCH.JCLPDS(WCBAC500)D. After the JCL has been copied in, do a find on WAAPSPLT again. If there is still no
WAAPSPLT, continue copying in more PROCs . If no WAAPSPLT step is found after copying in all the JCL possible, contact the SE because the operator can not respool this job. If the WAAPSPLT step is found, continue to step 3.
3. Edit the JOBNAME and place an R in the fourth character. The fourth character may also contain “#” , “@” or “L” or any other unique character.
4. Enter the access code (Woodland = ZLA1MM Micrographics = ZLA1MM).5. Delete //*LOGONID6. Delete unnecessary Output statements.7. Delete everything in JCL from Output statement to WAAPSPLT card.8. Delete all steps after the WAAPSPLT step.9. Change WAAPSPLT PSxx to JS010.10. Delete Input and Control cards when appropriate. If unsure, contact the on-call SE.11. Dummy out MSGSET DD.12. Change BACKUP DDNAME to INPUT01.13. Change relative GDG to appropriate number (usually 0).14. Change DISP = SHR.15. Delete all other DD parameters for newly created INPUT01 (e.g., vol=, unit=).16. Dummy out or delete other DD Names with “+” generations.17. Delete unnecessary OUTPUT cards when appropriate. In some cases the job will not spool properly
if any of the required DD’s for output are missing or changed (Unwanted DD’s could be changed to DD DUMMY). In these cases, change the unwanted DD’s output destination to nonprint or R99. If unsure, contact the on-call SE.
18. Go back to the top of the respool member and scan for “EXEC”. The only EXEC statement should be the WAAPSPLT statement.
19. Scan for “+”. The respool should not be creating any new datasets. Note: (+0) is all right to have.20. Submit the job.21. Check the output queue (DQO in SAM) to verify the respool has completed successfully. In SAM,
move to nonprint, R99, or delete any unnecessary forms.22. If necessary, change the destination of the requested forms to LOCAL for Woodland or to the desired
destination that was requested.23. Compare the total record counts for each DDNAME applicable.24. Notify requester of successful respool.
// DISP=(SHR,KEEP,KEEP)//INPUT01 DD DSN=PWCBS.HRWBU.BDRBSW40(0),// DISP=SHR//*//AUDTW90I DD SYSOUT=(4,,AH8A) *AUDT REPORT*//*//W54290IL DD SYSOUT=(4,PAGE,),OUTPUT=(*.LH8A), LODI IFP//W54291IL DD SYSOUT=(4,PAGE,),OUTPUT=(*.LH8A), LODI IFP//W54292IL DD SYSOUT=(4,PAGE,),OUTPUT=(*.LH54), LODI IFP//W54293IL DD SYSOUT=(4,PAGE,),OUTPUT=(*.LH8A), LODI IFP//W54294IL DD SYSOUT=(4,PAGE,),OUTPUT=(*.LH8A), LODI IFP
3.4 ACCESS VPS AND VTAM MENU PANELSIdentify printer type and which panel to use Frequently callers will not know what type printer they have, but they can provide their printer id. Here are guidelines to follow:Printer id = HFF* VPS printer on the A-side (i.e. tape library) or VTAM printer on the A-side (i.e. Freemont account)Printer id = VPS* VPS printer on the B-side (i.e. Del Monte)Printer id = P0*/P1* VPS printer on AS400(i.e. Del Monte and Freemont)*ask EOG for assistance with AS400 calls. Printer id = PRT* JES2 printer on the A-side (i.e. Woodland) or JES2 printer on the C-side (i.e. banks) These are not VPS or VTAM printers. Use S;DUPRT command from ISP to display printer status.VPS PANEL - ISPF E;V;1The most common method to access the VPS menu panel is from the ISPF menu using the selections E;V;1. There are restrictions to using this method. User must log onto 231A with TACT* or NETID for access to A-side printers. For B-side printers, log onto sa18 with TACT*ID. To display all printers: do an E;V;1;2 and leave criteria fields blank. Hit enter twice.
VPS PANEL - TSO OACQThe VPS menu can also be accessed using clist TSO OACQA-side printers TSO OACQ ; option 1B-side printers TSO OACQ ;option 7; option 1To display all printers and their statuses: Leave input fields blank; do an option 2-display VPS printers; hit enter twice
VTAM PANEL – TSO OACQ ;VThis procedure only works on the A-side. From any A-side system type TSO OACQ. V - VTAM Print. User id = OAC1 – DPRT Printer Display & Control
For more information about VPS, reference the appropriate VPS manual by LRS (Levi, Ray, and Shoup Inc.)
3.5Reset VPS PrintersWhen a call is received from a customer asking to check or acquire a VPS printer, perform the following steps.
1. Ask the customer the printer ID. (If the call is a Del Monte call, the customer will give a VPS ID (ex. VPS11). Utilize AOS to determine the printer ID by entering “BR DMVPS”.)
2. Verify that the printer is active. If the printer is connected, use either AOS or the VPS menu panels to start the printer.
Page 28 of 42
Fon’s Batch Procedures
3. If the printer is not active, use either AOS or the VPS menu panels to acquire the printer. Once the printer is acquired then start the printer.
4. Verify that the printer is working with the customer.
Note: One way to find out the AOS commands for VPS is to go into AOS and type “BR EXPERT”. Once in the “EXPERT” CLIST type “F VPS” for find VPS. This will give a listing of AOS commands for VPS. For additional information refer to VPS 6.0 and VPS 6.2 manuals.
3.6Acquire Credit Union Printers / Terminals to a ZoneWhen a call is received from a Credit Union Account asking to check or acquire a printer / terminal, perform the following steps.
1. Ask the customer if the printer / terminal belongs in a particular zone, if yes, which zone? I, P, W, or Y.
2. Set LOGMODE from EDSNET selection screen … SELECTION = > *LM,LSX327033. From the NETMENU selection screen select FLAGSHIP PREMIER PROD – ZONE X. (x) =
zone letter. This will bring up a Welcome screen: WELCOME TO PRODUCTION CICS 4.1.0 – PLIPC3E OLSJCIP__
4. Type in: DCFI (leave a space after DCFI, then press enter) You will overtype the word WELCOME at the top of the screen. (the cursor will be under the W).
5. Type in: INQUIRY The next screen is blank with PLEASE ENTER DCFI COMMAND at the bottom.
6. Type in: DCFI INQ NET(HFFxxxxx) (for a printer) or DCFI INQ NET(TFFxxxxx) (for a terminal).The following information will be displayed. Note the TER(XXXX) id and status.NET(HFFXXXX) PRI( 000 ) AUT INS ATI TER(XXXX) REL CRE Status of REL CRE indicates the printer is released, and needs to be acquired into the zone.
7. Type : DCFI SET TER (XXXX)INS ACQ . 8. Type: DCFI INQ NET(HFFxxxxx) to ensure printer is now acquired:
NET(HFFXXXXX) PRI( 000 ) AUT INS ATI TER(DBDB) ACQ CREIf the printer does not acquire, check the HFFXXXX id in AOS: D NET,ID=HFFXXXX,ENote: Most CUNA printers are owned by Plano, for an accurate status of the LU, call Plano oac.
9. See the above instructions along with VTAM Commands to display, inactivate, activate, or recycle the printer / terminal in AOS.
10. To exit Flagship : PF3 PAUSE/BREAK LOGOFF
3.7Reset PasswordsA customer may call and request to have their password reset due to too many invalid logon attempts or a suspended USERID. The standard for passwords is to use eight characters and at least one number. No character can be used more than twice. To reset a person’s password perform the following steps.
1. In TSO, type in TSO ACFRESET 2. Type in the USERID the customer is requesting to have reset and press ENTER.3. If the user has password violations:
a) Verify their EIN as described below and press ENTER.b) Use option B to reset the violations, but not their password, OR use option C to reset the
password if the user does not remember their password. This password should expire after the customer’s first use. The user will then need to enter a new password.
4. If the ID is suspended:a) Verify their EIN as described in the “Resetting Password Guidelines” section of this
manual, and use the CLIST TSO ACF command and issue the command List (user ID) (press enter) this will display the person’s ACF2 access.
Page 29 of 42
Fon’s Batch Procedures
b) To unsuspend the persons ID type in CH (user ID) NOSUSPEND (press enter). To exit out of the CLIST type “END” and press enter.
Use the following commands to reset Ids if the TSO ACFRESET feature is unavailable.
From the ISPF command line, enter TSO ACF. For the following commands, xxxxxx = User ID
L xxxxxx To list the ID CH xxxxxx PASSWORD(New Password) To reset password CH xxxxxx PSWD-VIO(0) To reset password violations to zero CH xxxxxx NOSUSPEND To unsuspend a suspended ID END To exit from the screen
3.7.1 Reset Password Guidelines: The user calls in and can supply their Employee Identification Number (EIN). Usually this number
is their Social Security number or employee ID. Verify that the NAME/EIN match on the ACF2 logonid record. If the customer can not provide you with this information, DO NOT reset the password. Forward the call on to the account help desk (if known) or to the Information Security help desk in Herndon at 703-742-1133 or 8-432-1133. It is the responsibility of the EDS account staff to reset passwords for users who can not supply this information.
The person calling in is authorized (privilege 3 or 4) in NACOS. The OAC day shift has the ability to sign on to NACOS and check for this privilege. Off hours, the call should be forwarded to the Herndon help desk or on-call person.
Users with the LEADER privilege have the ability to reset passwords for their account. They can also request password changes for their users when they are homesited on another site. Since the OAC has the capability to reset NETID passwords only on SAIPC1N, all calls for password resets for ID homesited outside of our network should be forwarded to Information Security.
Do NOT reset passwords for customers who call in on behalf of someone on their staff even if they know the EIN. ONLY the user or an authorized submitter in NACOS can request password resets. When in doubt, transfer the call to Information Security.
3.8VANTIVE Print Request TicketsOperators open Vantive tickets upon request for customers, to document problem calls, for any problem that requires a WRO service area or group to address, and for hard copy print requests. For detailed instructions on performing common Vantive functions, refer to the WRO Problem Management Procedures Manual (IPSQMSE0). Here are some general procedures for opening print request tickets.
1. Logon to the Vantive database.2. Open the contact locator screen under the file, open, contact.3. Search on the name of the person requesting the ticket. (Fields are case spacific).4. If no contact is found, you will need to create one.5. Verify that The Site tab is the location the client is calling from. If not change it.6. Make sure to save and refresh after the modification.7. After you have created or verified the contact record, click on the cases tab, then click New to open a
case.8. Make sure the ‘Agreement’ field reads: NAO generic or WR Mainframe. Either of these will allow
you to assign the ticket to the Woodland Service Delivery Center.9. Under the “Case Type” click on the “Move / add / change”.10. Note the jobname in the summary.11. Under ‘Group Assigned’ select 7738-WSDC OPS.12. Fill in all other fields required by Problem Management.13. Select ‘other services’ for the service class.14. Select ‘PLOT’ for the product.15. Click the ‘Events’ tab.
Page 30 of 42
Fon’s Batch Procedures
16. Click the Dispatched under ‘Events’.17. Record the ‘HD Event Time’. The amount of time in minutes you spent on this ticket.18. Find out if the request is for Micrographics or Paper, and if it is a respool or a special print request.19. Click on ‘File’, ‘Open’, ‘Company’.20. Type ‘EDS’ next to company and click search.21. Open EDS (CORPORATION).22. Click on the Comments Tab.23. Locate the corresponding information to your ticket. The template will appear in the ‘comment’
section and is grayed out, but you can still copy and paste from here to your ‘Detail’ under the ‘events tab’.
24. Fill in the information for the template.25. ‘Save’ and ‘Refresh (all pages)’
Once you have saved and refreshed you should se a notification at the bottom of the Vantive screen which means the case has been sent. System monitoring is no longer responsible for notifying the WSDC when a new ticket is sent to them.
3.9VANTIVE TicketsBatch Execution operators check VANTIVE tickets upon request only. At any time, a Batch Execution operator may receive a request to obtain the status of a VANTIVE ticket. VANTIVE tickets may be referenced by following the VANTIVE Tickets procedures located in the System Monitoring General Work Procedures Manual.
3.10 Send a Message to a TSO UserMVS systems allow users to send messages to each other. This is helpful when two users are working on a project and quick messages are necessary. It also helps to give a user, who is a long distance away, a quick message. There are several ways to send messages to other users (such as “TSO MSG”, OACQ, and SE). Not all methods are explained here, but any method can be used.
Using the “SE” Command : Quick Messages can be sent to a user with the “SE” TSO command. Examples:
From AOS type in: SE ‘ TEXT TO SEND’, user=(userid) or From ISPF type in: TSO SE,’TEXT TO SEND’U(userid).
Using the OACQ panel .
1. Log on to the same system as the receiving user.2. From the ISPF main menu type OACQ. Select option “M” (send message to TSO user).3. Move the cursor to the top of the screen under the “TSO ID” field.4. Enter the USERID to which the message will be sent.5. In the “Send message to selection #” field, enter the number assigned to the USERID typed under the
TSO ID field.6. Enter option “O” to send the message, or option “l” to have the message appear when the user logs
on.7. Enter “Y” in the “Ready to Send” field.8. Type the text and press enter.9. A message will appear verifying that the message was sent
3.11 Activate an EIDWhen a batch operator needs access to a dataset that is restricted, the emergency ID(EID) should be activated to gain access to certain datasets. This EID access is temporary and lasts for the current logon session. The EID panel tracks all users so that sensitive datasets can be more closely monitored.
The following steps are used to activate an EID.
Page 31 of 42
Fon’s Batch Procedures
1. Type ‘EID’ on the command line in ISPF.2. Password = TSO password.3. EID = 0SC1AOAC.4. Note the reason for activating your EID xxxxxxxxxxxxxx.5. On the command line type ‘A’ and press enter.6. You will receive a message ‘EID ACTIVE’ at the top right of your screen.
3.12 Change LogmodesThere are different types of logmodes to use for terminals. After signing onto the system, the operator may choose to change the terminal’s logmode. One way to change a terminal’s logmode is from the command line in the selection screen, enter *LM logmode (e.g. *LM C4B32793). Some logmode types that may be used are listed below.
C4B32793 LSX32703 D4A32783 SNX32702
3.13 Search for Data Using the TSO Search FacilityTSO users can scan multiple datasets for a given data string using the TSO Search Facility. The utility lists the results of the search in an output data set you specify in the panel. This data set will remain in your library and will be overwritten every time you use the utility.
1. Enter 3.14 from the primary option menu.2. Type the data string that is being searched for in the “search string” field.3. Type “N” in the “multiple strings” field on the search-for utility panel.
NOTE: Enclose the data string in quotes if there are imbedded blanks. All occurrences of the string will be found whether the matching string is in uppercase, lowercase or mixed case letters.
4. Enter the data set name where the desired search is to take place followed by an “*” (e.g., SYS4.SERESP*.
5. Enter the name of the data set where search results will go (default= userid.srchfor.list).
3.14 Prepare and Respond to Initial Program Loads (IPL)An IPL may occur under three conditions, scheduled, unscheduled, and system crash. In each of these situations, batch processing is interrupted. It is critical for the Batch Execution group to maintain constant communication with Resource Management and the affected customers during these periods. Refer to the following procedures for appropriate actions.
3.15 Prepare for Scheduled IPLsScheduled system IPLs are normally performed during a scheduled weekend maintenance window in accordance with customer requirements. Scheduled IPLs are required to perform routine system maintenance and install or upgrade hardware and software products. Batch Execution operators prepare batch processing to ensure minimal impact to the affected customers.
On each system, Resource Management operators begin draining initiators prior to a scheduled IPL. This is done to ensure that long running jobs do not start and subsequently require cancellation. Heavy batch systems require a longer time to drain than light batch systems. Because all of the initiators are closed, jobs will start backing up, awaiting execution.
1. Check the statistics on the jobs the operator is watching within CA-7 to ensure jobs that are running will not run into the IPL window.
2. Call the SE if running jobs will not complete before the IPL window.3. Single thread the jobs into the execution queue if necessary.4. Through SAM, increase the priority of the job.
/$TJxxx,P=n (xxx=job number, n=1-15, 15 being the highest priority)5. Check if the job has a system affinity (SYSAFF) or is required to run on a particular system.
Page 32 of 42
Fon’s Batch Procedures
6. Ensure that the job begins executing; otherwise take appropriate action (typically, working with the Resource Management Group to start and drain initiators).
7. Continue the process of single threading the jobs until the time of the IPL.8. Document all canceled jobs.9. Resource Management operators will bring the system down. If another system is available or when
the system comes back up for production, refer to ABEND resolution procedures for restart instructions on canceled jobs.
3.15.1 Respond to an Unscheduled IPLAn unscheduled IPL is normally caused by a software or hardware problem. The IPL is performed to apply changes to resolve a problem or to remove additions or changes that may have caused the problem. Personnel in the Resource Management Group shut down the system in a graceful manner. Batch Execution personnel assist with the system shut down to ensure minimal impact to the customers.
Resource Management will work with Batch Execution operators while canceling all active jobs running in the system.
1. Document all the canceled jobs within INFO Mgmt.2. Resource Management operators will bring the system down. If another system is available or when
the system comes back up for production, refer to ABEND resolution procedures for restart instructions
Additionally, during and after an unscheduled IPL the following steps should be taken to ensure that there is ample time to verify production cycle status’s:
Prior to the IPL, ensure that the Resource Management Group drains all class “H” initiators for all systems that share a spool image of CA-7..
1. After the unscheduled IPL, determine which jobs were submitted near the time of the problem by reviewing option 3 (the Active Jobs File).
2. Obtain the potentially affected jobs’ JES number in CA-7.3. Locate the potentially affected jobs in SAM.4. Review the SYSOUT for each job to determine if the job completed successfully.5. If any job did not complete successfully, refer to ABEND resolution procedures for restart
instructions.6. Through option 3 (the Active Jobs File), locate any job with the status of ended not OK and has a
status of “not submitted”. Jobs with this status are restarted from the top.7. If there are any ABENDs unrelated to the unscheduled IPL, refer to ABEND resolution procedures
for restart instructions.8. If any job has a status of “job disappeared” , the cycle monitor will need to review the SYSOUT to
determine whether a successful or unsuccessful completion occurred. If successful, force complete the job within CA-7. If unsuccessful, refer to ABEND resolution procedures for restart instructions.
9. If any job has a status of “Problem Reading SYSOUT”, do the following: Place the job(s) on hold Zoom the job and change the NJE flag to a Y Change the search counter to a 0022 Press PF3, and confirm your changes After confirming your changes, free the job
10. After statusing all cycles, contact the RMS group so that the JES class “H” initiators can be started.
3.15.2 Respond to a System CrashA system crash is an unexpected event that is caused by a software or hardware problem. The system is unavailable for batch processing during this time. Batch Execution operators provide notification and assistance to Ses during and after a system crash to ensure minimal impact to affected customers. After the problem has been detected and fixed, Resource Management operators will bring the system back up.
1. Ascertain the extent of the system crash from Resource Management (i.e. one system or one complex).
Page 33 of 42
Fon’s Batch Procedures
2. Refer to ABEND resolution procedures with the following considerations: Consider whether the job has SYSAFF or RESAFF constraints. The SE needs to determine if catalog maintenance needs to be performed prior to restarts, since
the job did not end gracefully. Jobs may be restarted on unaffected systems if possible. ABEND information must be obtained through SYSLOG, once the systems are back up.
JESLOG will not always provide the necessary information. ABEND resolution may have to wait until a system is available.
3. After the system crash, determine which jobs were submitted near the time of the problem by reviewing CA7.
4. If any job did not complete successfully, refer to ABEND resolution procedures for restart instructions.
3.16 Support Account MigrationsDuring an account migration of workload into the SSMC, the Batch Execution group is often required to participate in the project where batch monitoring will be necessary. Below are items that the Batch Execution point of contact should consider during a migration.
1. Develop Monitor turnover sheet or add to cycle monitor sheet if needed.2. Add necessary items to the appropriate on-line turnover member.3. Add information for the Account Procedure Manual
Identify critical path for cycles if needed. Information management if needed. ABEND notification if needed. Information on account specific requirements. Naming conventions of datasets pertaining to Batch Execution. Respool (who will be responsible)
4. Add new account training items to the appropriate training checklist.5. Add customer impact jobs to 80 if necessary.6. Send Customer EDS ON-CALL Users Guide and IRSBEXOC (SSMC ONCALL Implementation and
Support Guide for the SBU).
3.17 Documentation Change ControlRefer to the System Monitoring General Work Procedures Manual for the standard method of performing documentation change control within the System Monitoring organization.
Specifics relating to how Batch Execution accomplishes document review and approval are listed below:
Batch Execution document changes/additions/deletions may be proposed by entering the request into: SAIPC3A\SA300001\HOME\SAIPC\SYSTEM MONITORING\QMSCHG\BEXCHGR.DOC or by notifying the responsible Document Controller.
Completed QMSCHG requests are moved by the Document Controller into: SAIPC3A\SA300001\HOME\SAIPC\SYSTEM MONITORING\QMSCHG\BEXCHGC.DOC
Page 34 of 42
Fon’s Batch Procedures
APPENDIX A - QUALITY RECORDS
RECORD NAME **RET.PERIOD
DESCRIPTION
ICSBEX** - “Monitor Turnover Log Checklist”
1 month Monitor sheets that contain account specific information. Stored in the Batch Execution work area in the appropriate Monitor binder. The monitor sheets are filed under the index number indicating the day of the month.
ICSBEX70 - “Batch Execution Logon ID Checklist”
1 month Checklist to set up BEX logon Ids. Stored in the OAC work area.
ICSBEXTG.XLS Indef Batch Execution’s General Training Checklist Matrix. Stored in Public\Data\Forum\Qmsdoc\Prod\Training\MON.
ICSBEXTM.XLS Indef Batch Execution’s Monitor Training Checklist Matrix. Stored in Public\Data\Forum\Qmsdoc\Prod\Training\MON.
ICSQMS21 - “Document Controller Checklist”
The Last Revision applied by area Document Controller
Checklist used by the Batch Execution QMS Document Controller to update BEX QMS documentation. Stored hard copy in the BEX work area in the binder marked Document controller binder.
IMSBEX01 0 days ABEND Worksheet. Used to turnover any outstanding ABENDs. This has a 0 retention period. If the worksheet has no outstanding ABENDS documented the worksheet may be discarded..
IMSBEX02 - IMSBEX04 0 days BEX Work Procedures Quizzes for Months 1, 2, and 3. These have a 0 retention period. Once an individual takes the quiz and is signed off on their training checklist, this form may be discarded.
IMSMON01 The Last Revision of a document applied by area Document Controller
System Monitoring QMS operator listing for QMS approvals.
IRSMON01 - 0 days System Monitoring Terminology and Acronyms.IRSBEXOC Indef ONCALL documentation to send to Customers.Online Turnover Log 30 days Description of events that occurred each shift. Located and
stored on-line under TSO MON.SYS4.MORN(MSTRLOGx). (x=1-4 for M-Th and W for Fri-Sun)
30 days On-line system turnover information. Stored on-line in TSO.
** - At a minimum
Page 35 of 42
Fon’s Batch Procedures
APPENDIX B - Commonly Used Datasets / Filenames
Commonly Used Datasets DescriptionPSC1A.OACLB.CLIST Batch Execution CLISTPSC1A.OACLB.COMMENTS Concerns and Comments for shift improvementsPSC1A.OACLB.MSTRJCL Exec JCL library for cycle (SC1AOAC)PSC1A.OACLB.SEABEND.GxxxxV00 SSMC Daily ABEND ReportPSC1A.OACLB.TURNOVER Turnover logs for account cyclesPSC1A.OACSC.SCHEDULE Batch Execution Scheduling LibraryPSC1x.OACSC.TEMPCHN* Processed Temporary Change Libraries
x=System ID and *=Member name depending on the day of the week.SYS4.MORN(HOLDAYxx) Customer holiday scheduleSYS4.SERESP Responsible SEs for accountsTSC1A.OACLB.RESPOOL Respool JCL used by Batch ExecutionTSC1A.PCG.TEMPCHNG Temporary changes for cycles
S001 Data check on a tape / I/O error / Accidental switching of tape drives / ‘NO’ was replied to a WTOR / LRECL or BLKSIZE in the DCB does not match that of the dataset
S013 Invalid or no BLKSIZE / Invalid LRECL / DCB parameters conflict / Missing or misplaced SYSIN DD CARD / PDS member not found / Dataset not found / No directory allocation
S013-10 No blocksize for dummy dataset.- Correct the DCB and specify blocksize other than zeroS013-14 Dataset is not partitionedS013-18 Member not foundS013-20 RECFM=FB but the BLKSIZE is not a multiple of LRECL / RECFM=VB but the BLKSIZE
is not greater than the LRECLS03A Errors coding freemainsS03B File label DCB / PGM DCB disagreeS03D-04 Volume Serial not specifiedS03D-0C Incorrect DCB for indexed DSNS047 PGM not authorized see softwareS0C1 Operation Exception.S0C2 Privileged Operation ExceptionS0C4 Protection ExceptionS0C5 Addressing ExceptionS0C6 Specification ExceptionS0C7 Data ExceptionS0CB Dividing by ZeroS106-0C Space needed for control blocksS10A Insufficient regionS122 Operator canceled job with a dump / The system cancelled the job because it exceeded
allowable linesS137 I / O error on tape label end of volume processing (do a check tape to verify count and a mini
label)S13F Bad checkpoint restartS213 RC 30 file contention problem or an error in Format 1 DSCBS214 ABENDed during the CLOSE for a data set on a tape / Possible tape problem / Possible drive
problemS222 Job cancelled by operator or by the systemS237 Error during end of volume processing / Block count does not agree with the DCB block count
in trailer labelS322 CPU time expiredS413 I/O error where in the volume or unit specifications in the DD were not metS413-04 No device availableS413-34 VOL=SER not specifiedS413-1C Incorrect volume sequence numberS422 The specified Q space was exceeded. Too many DD names within one job step. Restart the
job and the DD name count will begin with the step that the job is being restarted on.S513-04 Duplicate device allocationS613 Label missing / Tape position error / Error writing tape mark of HDR (do a check tape to
verify count and a mini label)S637 I/O error during end-of- volume processing for a magnetic tape dataset / A concatenation
error / Error writing a tape mark / Tape positioning error / Reading label error Do a checktape to verify block count and a mini label.
S637-04 An I/O error occurred reading a tape label, writing a tape mark, or positioning a tape volume. The solution to fix this is to restart the job.
S637-OC Concatenation of datasets with wrong DCB
Page 37 of 42
Fon’s Batch Procedures
This is normally seen on sorts. Notify the SE of the situation and for any options.S714 Data check / Equipment check / Error writing a tape mark S804 Not enough coreS804-1 Region size specified was not enough to execute the instruction.S804-2 Zero or no region was specifiedS806 Program not foundS80A Not enough core / Insufficient virtual storageS813 Label DSN/DD DSN do not agree (do a check tape to verify count and a mini label)S837 Volume count exceededS837-04 Density ConflictS837-08 Volume not found in JFCBS878 Not enough region codedS913 Security Exception / Password protected dataset / ACF 2 ViolationSA13 Incorrect sequence number in HDR / Tape mark found instead of HDRSB37 No space available or specified space for ABENDing DD is not large enough for fileSD37 This is a space ABEND of a particular dataset that requires the dataset to be compressed or
expandedSE37 No space on volume. It may be necessary to contact DASD to request more space on a
volume.SC03 Files not closed at EOJ or at step change
Refer to common ABEND research and resolution procedures for additional Common ABEND codes.
For more information regarding ABEND codes, refer to the MVS / ESA Messages and Codes Manual, the MVS / ESA System Messages, or utilize the on-line Quick Reference Facility (TSO QW).
Page 38 of 42
Fon’s Batch Procedures
APPENDIX D - Batch Execution Training Progression
Listed Below is the Cycle progerssion a new Batch Execution Operator should follow. A new Batch Execution Operator should start off with the less complicated and less critical cycles and then work towards learning the more complicated and critical cycles. It is also important to watch new people when they are training and before they are checked off on items on the training checklist. This progression should be followed unless business needs arise that will not make it possible to follow this list.
1. Monitor 1 and 22. Monitor 5, 7 and 83. Monitor 64. Monitor 4 and 115. Monitor 11 and 126. Monitor 13 and 147. Monitor 9 and 10
