www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
UNIX Shell-Scripting Basics
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Agenda
What is a shell? A shell script?Introduction to bashRunning CommandsApplied Shell Programming
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
What is a shell?
%▌
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
What is a shell?
/bin/bash
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
What is a shell?
#!/bin/bash
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
What is a shell?
INPUT
shell
OUTPUT ERROR
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
What is a shell?
Any ProgramBut there are a few popular shells…
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Bourne Shells
/bin/sh/bin/bash
“Bourne-Again Shell”
Steve Bourne
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Other Common Shells
C Shell (/bin/csh)Turbo C Shell (/bin/tcsh)Korn Shell (/bin/ksh)
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
An aside: What do I mean by /bin ?
C Shell (/bin/csh)Turbo C Shell (/bin/tcsh)Korn Shell (/bin/ksh)
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
An aside: What do I mean by /bin ?
/bin, /usr/bin, /usr/local/bin /sbin, /usr/sbin, /usr/local/sbin /tmp /dev /home/borwicjh
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
What is a Shell Script?
A Text FileWith InstructionsExecutable
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
What is a Shell Script?
% cat > hello.sh <<MY_PROGRAM#!/bin/shecho ‘Hello, world’MY_PROGRAM% chmod +x hello.sh% ./hello.shHello, world
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
What is a Shell Script? A Text File
% cat > hello.sh <<MY_PROGRAM#!/bin/shecho ‘Hello, world’MY_PROGRAM% chmod +x hello.sh% ./hello.shHello, world
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
An aside: Redirection
cat > /tmp/myfile cat >> /tmp/myfile cat 2> /tmp/myerr cat < /tmp/myinput cat <<INPUTSome inputINPUT
cat > /tmp/x 2>&1
INPUT
env
OUTPUT ERROR
0
1 2
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
What is a Shell Script? How To Run
% cat > hello.sh <<MY_PROGRAM#!/bin/shecho ‘Hello, world’MY_PROGRAM% chmod +x hello.sh% ./hello.shHello, world
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
What is a Shell Script? What To Do
% cat > hello.sh <<MY_PROGRAM#!/bin/shecho ‘Hello, world’MY_PROGRAM% chmod +x hello.sh% ./hello.shHello, world
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
What is a Shell Script? Executable
% cat > hello.sh <<MY_PROGRAM#!/bin/shecho ‘Hello, world’MY_PROGRAM% chmod +x hello.sh% ./hello.shHello, world
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
What is a Shell Script? Running it
% cat > hello.sh <<MY_PROGRAM#!/bin/shecho ‘Hello, world’MY_PROGRAM% chmod +x hello.sh% ./hello.shHello, world
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Finding the program: PATH
% ./hello.shecho vs. /usr/bin/echo% echo $PATH/bin:/usr/bin:/usr/local/bin:/home/borwicjh/bin
% which echo/usr/bin/echo
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Variables and the Environment
% hello.shbash: hello.sh: Command not found
% PATH=“$PATH:.”% hello.shHello, world
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
An aside: Quoting
% echo ‘$USER’$USER% echo “$USER”borwicjh% echo “\”””% echo “deacnet\\sct”deacnet\sct% echo ‘\”’\”
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Variables and the Environment
% env[…variables passed to sub-programs…]% NEW_VAR=“Yes”% echo $NEW_VARYes% env[…PATH but not NEW_VAR…]% export NEW_VAR% env[…PATH and NEW_VAR…]
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Welcome to Shell Scripting!
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
How to Learn
man man bash man cat man man
man –k man –k manual
Learning the Bash Shell, 2nd Ed. “Bash Reference” Cardshttp://www.tldp.org/LDP/abs/html/
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Introduction to bash
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Continuing Lines: \
% echo This \Is \ A \Very \Long \ Command LineThis Is A Very Long Command Line%
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Exit Status
$?0 is True
% ls /does/not/exist% echo $?1% echo $?0
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Exit Status: exit
% cat > test.sh <<_TEST_exit 3_TEST_% chmod +x test.sh% ./test.sh% echo $?3
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: test
% test 1 -lt 10% echo $?0% test 1 == 10% echo $?1
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: test
test[ ]
[ 1 –lt 10 ] [[ ]]
[[ “this string” =~ “this” ]](( ))
(( 1 < 10 ))
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: test
[ -f /etc/passwd ] [ ! –f /etc/passwd ] [ -f /etc/passwd –a –f /etc/shadow ] [ -f /etc/passwd –o –f /etc/shadow ]
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
An aside: $(( )) for Math
% echo $(( 1 + 2 ))3% echo $(( 2 * 3 ))6% echo $(( 1 / 3 ))0
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: if
if somethingthen :# “elif” a contraction of “else if”:elif something-elsethen :elsethen :fi
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: if
if [ $USER –eq “borwicjh” ]then :# “elif” a contraction of “else if”:elif ls /etc/oratabthen :elsethen :fi
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: if
# see if a file existsif [ -e /etc/passwd ]then echo “/etc/passwd exists”else echo “/etc/passwd not found!”fi
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: for
for i in 1 2 3do echo $idone
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: for
for i in /*do echo “Listing $i:” ls -l $i readdone
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: for
for i in /*do echo “Listing $i:” ls -l $i readdone
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: for
for i in /*do echo “Listing $i:” ls -l $i readdone
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: C-style for
for (( expr1 ; expr2 ; expr3 ))do listdone
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: C-style for
LIMIT=10for (( a=1 ; a<=LIMIT ; a++ ))do echo –n “$a ”done
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: while
while somethingdo :
done
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Logic: while
a=0; LIMIT=10while [ "$a" -lt "$LIMIT" ]do echo -n "$a ” a=$(( a + 1 ))done
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Counters
COUNTER=0while [ -e “$FILE.COUNTER” ]do COUNTER=$(( COUNTER + 1))done
Note: race condition
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Reusing Code: “Sourcing”
% cat > /path/to/my/passwords <<_PW_FTP_USER=“sct”_PW_% echo $FTP_USER
% . /path/to/my/passwords% echo $FTP_USERsct%
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Variable Manipulation
% FILEPATH=/path/to/my/output.lis% echo $FILEPATH/path/to/my/output.lis% echo ${FILEPATH%.lis}/path/to/my/output% echo ${FILEPATH#*/}path/to/my/output.lis% echo ${FILEPATH##*/}output.lis
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
It takes a long time to become a bash guru…
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Running Programs
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Reasons for Running Programs
Check Return Code $?
Get Job Output OUTPUT=`echo “Hello”` OUTPUT=$(echo “Hello”)
Send Output Somewhere Redirection: <, > Pipes
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Pipes
Lots of Little Tools
echo “Hello” | \ wc -c
INPUT
echo
OUTPUT ERROR
0
1 2
INPUT
wc
OUTPUT ERROR
0
1 2
A Pipe!
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Email Notification
% echo “Message” | \mail –s “Here’s your message” \ [email protected]
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Dates
% DATESTRING=`date +%Y%m%d`% echo $DATESTRING20060125% man date
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
FTP the Hard Way
ftp –n –u server.wfu.edu <<_FTP_user username passwordput FILE_FTP_
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
FTP with wget
wget \ftp://user:[email protected]/file
wget –r \ftp://user:[email protected]/dir/
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
FTP with curl
curl –T upload-file \ -u username:password \ ftp://server.wfu.edu/dir/file
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Searching: grep
% grep rayra /etc/passwd% grep –r rayra /etc% grep –r RAYRA /etc% grep –ri RAYRA /etc% grep –rli rayra /etc
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Searching: find
% find /home/borwicjh \ -name ‘*.lis’[all files matching *.lis]% find /home/borwicjh \ -mtime -1 –name ‘*.lis’[*.lis, if modified within 24h]% man find
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Searching: locate
% locate .lis[files with .lis in path]% locate log[also finds “/var/log/messages”]
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Applied Shell Programming
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Make Your Life Easier
TAB completionControl+Rhistorycd -Study a UNIX Editor
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
pushd/popd
% cd /tmp% pushd /var/log/var/log /tmp% cd ..% pwd/var% popd/tmp
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Monitoring processes
psps –efps –u oracleps –C sshdman ps
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
“DOS” Mode Files
#!/usr/bin/bash^MFTP transfer in ASCII, ordos2unix infile > outfile
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
sqlplus
JOB=“ZZZTEST”PARAMS=“ZZZTEST_PARAMS”PARAMS_USER=“BORWICJH”sqlplus $BANNER_USER/$BANNER_PW << _EOF_set serveroutput onset sqlprompt ""
EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB', '$PARAMS', '$PARAMS_USER');
_EOF_
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
sqlplus
sqlplus $USER/$PASS @$FILE_SQL \ $ARG1 $ARG2 $ARG3if [ $? –ne 0 ]then exit 1fiif [ -e /file/sql/should/create ]then […use SQL-created file…]fi
Ask Amy Lamy!
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Passing Arguments
% cat > test.sh <<_TEST_echo “Your name is \$1 \$2”_TEST_% chmod +x test.sh% ./test.sh John Borwick ignore-this
Your name is John Borwick
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
INB Job Submission Template
$1: user ID$2: password$3: one-up number$4: process name$5: printer name
% /path/to/your/script $UI $PW \ $ONE_UP $JOB $PRNT
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Scheduling Jobs
% crontab -l0 0 * * * daily-midnight-job.sh0 * * * * hourly-job.sh* * * * * every-minute.sh0 1 * * 0 1AM-on-sunday.sh% EDITOR=vi crontab –e% man 5 crontab
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
www.wfu.edu/~borwicjh/presentations/UNIX%20Shell-Scripting%20Basics.ppt
Other Questions?
Shells and Shell ScriptsbashRunning Commandsbash and Banner in Practice