Running the Operational Codes for the Brahmaputra

Tom Hopson

Automatic (background) Job Processing: CRON

Cron is a time-base job scheduler in Unix-like computer operating systems. 'cron' is short for 'chronograph’. Cron enables users to schedule jobs (commands or shell scripts) to run automatically at a certain time or date. It is commonly used to perform system maintenance or administration, though its general purpose nature means that it can be used for other purposes, such as connecting to the Internet and downloading email.

Crontab file:

.---------------- minute (0 - 59) | .------------- hour (0 - 23)| | .---------- day of month (1 - 31)| | | .------- month (1 - 12) OR jan,feb,mar,apr ... | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat | | | | |* * * * * command to be executedThere are several special entries, most of which are just shortcuts, that can be used instead of specifying the full cron entry:Entry DescriptionEquivalent To@reboot Run once, at startup. None@yearly Run once a year 0 0 1 1 *@annually (same as @yearly) 0 0 1 1 *@monthly Run once a month 0 0 1 * *@weekly Run once a week 0 0 * * 0@daily Run once a day 0 0 * * *@midnight (same as @daily) 0 0 * * *@hourly Run once an hour 0 * * * *

Note: you may need to specify full paths of programs since SHELL (“dot” files) are not used by CRON

More on crontab file execution timing:Some examples …

Go to the directory

cd /home/administrator/Desktop

First steps using Linux commands:

CRON file example:

Create a file called, say, “cron_ex”, with:* * * * * echo "trying this out!" >> /home/administrator/Desktop/cron_test

Then type:>crontab cron_ex (to install the CRON job)>crontab -l (to see if it is working in the CRON environment>date (to make sure a minute has passed)

After a minute in your directory, you should see a file “cron_test” >cat cron_test (to see the output)

Then type>crontab -r (to remove)

Text edit: firstprog.pro

In the file, type:

print,'this is my program’end

Then, start idl Inside IDL, type:

.r firstprog

To exit IDL, type: exit… what do you get?

Next, we want to use cron to control a computer program –In this case, an IDL program. But lets start with the basics first …

CRON IDL example:

Create a file called, say, “cron_idl”, with:* * * * * /usr/local/bin/idl</home/administrator/Desktop/infile_IDL

Then create infile_IDL with:.rnew /home/administrator/Desktop/testcron.protestcronexit

And create testcron.pro withPRO testcronspawn,"echo 'trying this out!' > /home/administrator/Desktop/cron_test2"openw,1,’/home/administrator/Desktop/cron_test3'printf,1,'trying this out!'close,1end

Let’s run the program first to see what the outputshould look like:

Start IDL:>idl

To compile the procedure, type: >.r testcron

To run the procedure, type:>testcronAnd exit idl>exit

In a linux window, look at the file:>cat cron_test3

Now lets go back and have cron control this program …

CRON IDL example:

Create a file called, say, “cron_idl”, with:* * * * * /usr/local/bin/idl</home/administrator/Desktop/infile_IDL

Then create infile_IDL with:.rnew /home/administrator/Desktop/testcron.protestcronexit

And create testcron.pro withPRO testcronspawn,"echo 'trying this out!' >> /home/administrator/Desktop/cron_test2"openw,1,’/home/administrator/Desktop/cron_test3'printf,1,'trying this out!'close,1endAnd do as before to install … (then type >crontab -r to remove when finished)

CRON IDL example 2:

Now do as before with the file getRT.pro, to download satellite precip … first, let’s look at the file:;---------------------------------------------------------------; program to download both 3-hr TRMM data;---------------------------------------------------------------

PRO getRT,year,month,day,HISTDAYS

gpcp3hrdir='/Users/hopson/WorldBank/Training_CWC_June2012/Labs/Day2/CRON/'gpcp3hrprefix='3B42RT.'gpcp3hrsuffix='.bin.gz'gpcp3hrftpsite= $ 'ftp://trmmopen.gsfc.nasa.gov/pub/merged/mergeIRMicro/'TRITIM=8trihourly=['00','03','06','09','12','15','18','21']

; note: added "-r" on Dec 30, 2006 to stop multiple file creation (i.e. *.1, *.2, etc.); that occurred around the beginning of Dec, 2006;***wgetcmd='/usr/bin/wget -N 'wgetcmd='/usr/local/bin/wget -N '

; check last "HISTDAYS" days, including current day, to see if data existsif n_elements(year) eq 0 then begin HISTDAYS=1 ; number of past days to query if data retrieved ; get dates to check/download (earliest to latest) today=systime(/julian)endif else begin ; get dates to check/download (earliest to latest) today=JULDAY(month,day,year)endelse

datenam=STRARR(HISTDAYS)for i=0,HISTDAYS-1 do begin caldat,(today-i),month,day,year stryear=strtrim(year,2) strmonth=strtrim(month,2) if month LT 10 then strmonth='0'+strtrim(month,2) strday=strtrim(day,2) if day LT 10 then strday='0'+strtrim(day,2) ;***datenam(i)=stryear+strmonth+strday ; latest to earliest datenam(HISTDAYS-1-i)=stryear+strmonth+strday ; earliest to latestendfor

; do for current and past daysfor i=0,HISTDAYS-1 do begin cd,gpcp3hrdir spawn,wgetcmd+'"'+gpcp3hrftpsite+gpcp3hrprefix+datenam(i)+'*'+'"'endfor

RETURNend

CRON IDL example 2:

Now do as before with the file getRT.pro, and do as before to install …

… and you’re now downloading NASA TRMM satellite precipitation data automatically!

(then type >crontab -r to remove when finished)

Note about the “wget” command:

GNU Wget is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get, connotative of its primary function. It currently supports downloading via HTTP, HTTPS, and FTP protocols, the most popular TCP/IP-based protocols used for web browsing.

To run the forecasting codes:Go to the Desktop/practicalcd /home/administrator/Desktop/practical

Type the following in the “practical directory”>tar xspjSlf tarForecast.tbz2

Then go to the created Training/ directory

Modify line 20 of the program se_fcstwkarma_w2s.proto specify the working directory of the codes

Start up IDL, then type

>se_fcstwkarma_w2s,1 … and off it goes

When finished, look at the plots in the “scratch” directory