Unix day2 v1.3

Tech Mahindra Limited confidential© Tech Mahindra Limited 2008

UNIX

Day 2

CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 2

Objectives At the end of this session, you will be able to:

Understand regular expressions Understand the grep family of commands Understand UNIX Shell features and its environment Read/Write basic shell scripts using decision making constructs


Agenda: Day 2 Regular Expressions and grep

Unix Shell

Unix Shell Environment

Unix Shell Scripting


Regular Expressions and Grep


Regular Expression Often called a pattern, is an expression that describes a set

of strings

Example, a regular expression “amit” may match “amit”, “amita”,”amitabh”, “namit” etc…

Many UNIX tools, primarily grep,sed & awk make use of regular expressions in text processing


Regular Expressions (contd.) Regular Expressions can be divided into:

Basic regular expressions (BRE) Supported by grep

Extended regular expressions (ERE) Supported by grep –E or egrep


Basic Regular Expression

\ Quote the next metacharacter ^ Match the beginning of the line

. Match any character $ Match the end of the line

[] Character class

Special Operators


Extended Regular Expression

| Alternation

()

Grouping

Special Operators


Examples “a.g” matches aag, abg, a1g, etc

“a[pmt]g” matches apg, amg or atg

“a[^pmt]g” matches aag, abg but not apg or amg or atg

“^ftp” matches ftp at the beginning of a line

“tle$” matches tle at the end of a line

“^$” matches a line with nothing in it

“jelly|cream” matches either jelly or cream

“(eg|pe)gs” matches either eggs or pegs


Quantifiers: number of repeats of the previous character

* Match 0 or more times

+ Match 1 or more times

?

Match 1 or 0 times

BRE

ERE


Examples “adg*” ad followed by zero or more g characters

“.*” Any character, any number of times

“[qjk]” Either q or j or k

“[^qjk]” Neither q nor j nor k

“[a-z]” Anything from a to z inclusive

“[^a-z]” No lower case letters

“[a-zA-Z]” Any letter

“[a-z]+” Any non-zero sequence of lower case letters

“(da)+” Either da or dada or dadada or...


grep family fgrep: fast searching for fixed strings

$fgrep string file(s) It handles fixed character strings as text patterns Does not use regular expressions Faster than grep and egrep for searching text strings

Examplesfgrep “Ramesh” datalist If found, lists the line(s) containing Ramesh


grep family grep: is called as a global regular expression printer It searches for a given pattern in a file(s)

$ grep -[cvnl] [pattern] [files]

Option Description -c counts the total no of lines containing the pattern -v displays all the lines not containing the pattern -n displays lines with line number -l displays file names containing the pattern

Example: grep “Agg*[ra][ra]wal” datalist

It lists all lines where the pattern matches some text The possible matches for the text are:

Agrawal, Agarwal, Aggarwal, Aggrawal(and many more combinations possible)


grep family egrep: extended grep, supports both BRE as well as ERE

grep –E can also be used in the place of egrep Examples:

$ egrep ‘ (John|Johnathon) Smith ‘ employee.txt

This will search for John Smith as well as for Johnathon Smith

grep –E “(S|Sh)arma datalist Matches Sarma or Sharma in the text from datalist


Unix Shell


Shell Features Command Prompt Command Interpretation and Shell Meta Characters I/O Redirection Pipes Shell Programming Constructs Job Control Command History


Shell as a Command Interpreter

Shell prints the prompt on screen

User enters the command

Shell interprets the command

Shell waits till the command finishes

execution


Popular Unix Shells Bourne Shell

Korn Shell

C Shell

Bourne Again Shell


Bourne Shell The Bourne shell is the original UNIX shell program

It is very widely used

Invoked using sh or /bin/sh at the command prompt

The Bourne shell supports conditional branching in the form of if/then/else statements

In addition, the Bourne shell supports case statements and loops (for, while, and until)

The Bourne shell uses the $ as a prompt


Korn Shell The Korn shell is a newer variation of the Bourne shell.

It supports everything the Bourne shell does, and adds features not available in the Bourne shell.

Invoked using ksh or /bin/ksh at the command shell prompt.

The Korn shell was originally written by David Korn and is copyrighted by AT&T.

The programming structure of the Korn shell is very similar to that of the Bourne shell. The Korn shell, however, is more interactive.


C Shell The C shell is a very commonly used shell

Its programming structure closely resembles that of the programming language "C."

The C shell uses the "%" as a prompt

The C shell supports all of the features that the Bourne shell supports, and has a more natural syntax for programming

The C shell is more interactive than the Bourne shell, with additional features that are not available in older shells

The configuration of the C shell is controlled by the .rc and the .login files


Bourne-Again Shell The Bourne-Again shell is a variation of the Bourne shell

It is commonly used in Linux, but is widely available in other standard UNIX distributions

The Bourne Again shell is another modification of the Bourne shell, and uses the $ as a prompt

To start the Bourne Again shell, type "bash" at the shell prompt

The behavior and environment of the Bourne Again shell is controlled by the .bashrc file, which is a hidden file in your home directory




Environmental Variables The Unix system is controlled by a number of shell variables

that are separately set by the system - some during boot sequence, and some after logging in.

These variables are called system variables or environment variables.

The set statement displays the complete list of all these variables. These built-in variable names are defined in uppercase.


Environmental Variables PATH: is a variable that instructs the shell about the route it

should follow to locate any executable command

HOME: when you log in, UNIX normally places you in a directory named after your login name

MAIL: determines where all incoming mail addressed to the user is to be stored

PS1 and PS2: PS1 - your command prompt and PS2-Multi-line command string

SHELL: determines the type of shell that a user sees on logging inNote: There are other environmental variables also but we have discussed the important ones.


Profile .profile (.bash_profile in Linux): the script executed during

login time

The .profile must be located in your home directory, and it is executed after /etc/profile, the universal profile for all users

Universal environment settings are kept by the administrator in /etc/profile so that they are available to all users


Setting Environment Variables Assigning a value to a variable will set an environment

variable temporarily

For example: $ x=50

This example will set value 50 to the variable x


Setting Environment Variables The variable will not be available if you exit the shell. Add

this variable to configuration files e.g. “.profile” to set it permanently

If you no longer want a variable to be set, you can use the unset command to erase its value

For example:$ unset xThis command will cause x to have no value set


export By default, the values stored in shell variables are local to

the shell, i.e., they are available only in the shell in which they are defined. They are not passed on to a child shell.

But the shell can also export those variables recursively to all child processes so that, once defined, they are available globally. This is done with the export command.

$ x=hello $ export x $ sh $ echo $x hello


. (dot): Running commands from file If we have a list of commands in a file “sample”, we can execute it

using command: $ . sample

It is like executing a shell script, but with following difference: Standard shell scripts cause a new sub shell to be created to

run the script. The dot command uses the same shell to execute the

commands.

The dot command, however, creates no child process, so any changes it produces apply to the original shell

It also does not require the script to have executable permission




Shell Scripts Scripts are mostly written for developing interfaces viz. interfaces

to application servers or to database servers.

Scripts can be client side scripts like JavaScript, VBScript etc. or server side scripts like shell scripts, perl scripts etc.

When a group of Unix commands has to be executed regularly, it is stored in a file. All such files are called as shell scripts.

There is no restrictions on extension of these files, but conventionally extension .sh is used for a shell script.

Unlike compiled programs, shell scripts are interpreted at run time.

You can use the vi editor to create/edit the shell script.


Shell Scripts Shell scripting language provides following:

Scalar and array variables

Set of operators

Flow, Loop and case statements

Positional Parameters

here document

Functions


Executing shell script You can execute the shell scripts using either command sh

or by just typing shell script name at the prompt (make sure that you have execute permission)

$ sh test.sh $ test.sh $ ./test.sh


Shell Scripting User-created Shell Variables:

variable=value => assigns value to variable$variable => refers value of the variable

To display a variable: echo $var “$var” ‘$var’

the output: hello hello $var


Shell Scripting

read statement

To read a value in a variable form keyboard:read var

To read a value in a variable from a file:

read var < file1

It reads the first line from the file into variable var

read var1 var2 < file1It reads the first word into var1 and second word into var2 from file1


Shell Scripting expr command This command is used to evaluates expressions e.g.

expr 10 + 20 Operators

Arithmetic: + Add - Subtract/ Divide \* Multiply % Modulo


Shell Meta Characters Wildcard substitution: * ? []

Redirection: >, >>, <, 2>, <<

Piping: |

Command substitution: ` `

Sequential commands: semicolon (;)

Inserting comment: #

Command grouping: () parenthesis


test command test command:

used to conduct several tests on integers, strings, file attributes It produces no output so used with if/while where its exit status

is used

Examples:

test 15 –gt 10

This command compares 15 and 10 and gives a true exit status but does not produce any output.


test command

Comparing numbers: Operators:

Relational -lt less than -le less than equal to -gt greater than -ge greater than equal to –eq equal to –ne not equal to

Logical: -a And -o OR ! NOT


Test Command - Strings String comparison used by the command test:

String Comparison True if

string1 = string2 strings equal

string1 != string2 strings not equal

-n string string not null

-z string string is null


Test Command - Files File tests used by the command test

Test True if

-d file file exists and is a directory-e file file exists-f file file exists and is a regular file-r file file exists and is a readable-s file file exists and has a size > 0-w file file exists and is a writable-x file file exists and is a executable


Conditional Statement: if-then-else The if statement takes two-way decisions depending on the condition

if conditionthen

commands

fi

if conditionthen

commands

else

commands

fi

if conditionthen

commands

else if conditionthencommandsfi

fi

if conditionthen

commands

elif condition thencommands

else

commandsfi


Case Statement The statement matches an expression for more than one

alternative, and permits multi-way branching

case variable/expression/value invalue1) command1

command2 ;;value2) command3 ;; *) command4

esac


Example of Case Statement

echo “Enter the color”read colorcase $color in Red | red) echo “You have selected red color”

;;Blue | blue) echo “You have selected blue color”

;; *) echo “Sorry! Yet to add this color”

;; esac


Positional Parameters (command line arguments) One can pass the command line arguments to shell script

while execution

When arguments are specified with a shell script, they are assigned to variables called positional parameters

The first argument is read by the shell into the parameter $1, the second into the parameter $2, and so on

The $# represents total number of arguments passed to the script

The command is assigned to a variable $0

You can use these variables up to $9


Positional Parameters (command line arguments) The $* indicates all arguments, in a single variable,

separated by the first character in the environment variable IFS

The $@ is same as $* except when enclosed in double quotes

The “$@” works with string input


set set command assigns values to positional parameters:

$ set 23 532

The command assigns value 23 to the positional parameter $1, and 532 to $2

It also sets $#, $*


shift shift command shifts command line arguments to left

The shift command copies the contents of a positional parameter to its immediate lower numbered positional parameter. When called once, contents of $2 are copied to $1, $3 to $2 and so on.

$ shift 2 The command does two shifts i.e. $1=$3, $2=$4, and so on.

Using shift command we can pass more than 9 command line parameters to shell script


SummaryIn this session, we have covered:

Regular Expressions and grep

Unix Shell




Thank You

Date post:	27-Jan-2017
Category:	Education
Upload:	xavier-john
View:	197 times
Download:	0 times

Unix day2 v1.3

Education