Introduction to Perl · Introduction to Perl Perl’s Data Structures Perl’s Data Structures I...

Introduction to Perl


Scott Hazelhursthttp://www.bioinf.wits.ac.za/~scott/perl.pdf

August 2013

http://www.bioinf.wits.ac.za/~scott/perl.pdf


Introduction and Motivation


Practical Extraction and Report Language

I General language – Intended for systemsprogramming, scripting

I Popular with systems programmers & Webdevelopers

I Relatively new language (ca. 1990)

I Big language – support for concurrency and OO.

I Portable.



Perl powerful and flexible language:

I Many devotees

I Many criticisms of the language

Easy to write difficult-to-understand code.

I Focus on writing readable code (for yourself,others)



Objectives

I Describe basic features of Perl

I Write simple Perl programs

I Use basic matching facilities

Lots of resources:I BooksI http://cpan.mirror.ac.za/,www.cpan.org, www.perl.org

I perldoc, info perlAssume: knowledge of programming, C-likelanguage

http://cpan.mirror.ac.za/

www.cpan.org

www.perl.org


Perl’s Data Structures

Perl’s Data Structures I

Perl is an imperative language.

I State is represented by a set of variables.

I Program is an ordered sequence of commands.

I Computation is accomplished by the executionof these commands in the specified order

Will explore Perl’s OO features later.



Flexible (perhaps too flexible) language.

I Scalars: numbers, strings

I Arrays, lists

I Hash tables

I References

Variables have a special prefatory character toindicate to which genre of variable (%, $, @, . . . )



Feature Warning

I Variables automatically declared

I Variables given default values

I Implicit type coercion common

I Meaning dependent on context

Good idea to use the “use strict; use warnings” pragmas



Scalar types

Scalar types

All scalar variables have a $ as prefix: e.g. $x,$year$, $etc.

$c= 10;

$f = 9/5*$c+32;

$cname = ’Introduction to Perl’;

print "Course $cname: size $c\n";



String operations

Interpolation

I Inside a double-quoted string, Perl does variableinterpolation, and interprets escape charactersprint "Course $cname size \t $f\n;";

I Not done inside single-quoted strings.print ’Course $cname size \t $f\n’;



String operations

Interpolation

I Inside a double-quoted string, Perl does variableinterpolation, and interprets escape charactersprint "Course $cname size \t $f\n;";

I Not done inside single-quoted strings.print ’Course $cname size \t $f\n’;



String operations

Other string operations

Concatenation: .

$fruits= ’apples and pears’;

$base = ’pies’;

$full = $fruits . ’ make ’ . $base;

$full .= ’: ’ . fruits;

I Length of string: length. Substrings: substr

I Repeat operator: x

I Powerful & flexible string matching and processing –regular expressions.

I Type conversion to and from integers as necessary !!?!!



Artithmetic operations

Arithmetic OperationsThe basic Perl arithmetic operations are: +, -, *, /,%, **.

I Default: floating point arithmetic

I There are many arithmetic procedures: int,sqrt,. . .

Short-cuts: Perl has many C-like features. eg.

$x = 3;

$x += $y+1;

$x++;

--$y;



Truth and Logical operations


1. Any string is true except for the empty stringand ”0”;

2. Any number is true except for 0

3. Any reference is true

4. Any undefined value is false




Logical operators

High binding Low binding

&& and|| or! not

Short-circuit evaluation!

Logical expressions returns the last value evaluated.

e.g. 2 or 3 and 5 2 and (0 or 5) $a ||= 2




Relational operators

Comparison Numeric String

Equal, Not equal ==, != eq, neLess than (equal) <, <= lt, leGreater than (equal) >, >= gt, geComparison <=> cmpSmart matching ~~ ~~




File operators

Example Name Example Name

-e $a File Exists -T $a File is a text file-r $a Readable file -w $a Writable file-d $a File is a directory -f $a Regular file


Standard input and output


Default: file handle STDIN associated withkeyboard (input), STDOUT with console output.

I Input is line oriented.



Output

print command by default sends output toSTDOUT.

print "Hello";

same as

print STDOUT "Hello";



Input

A reference to <STDIN> waits for input fromconsole.

I data typed in is returned

print "Enter the temp in C: ";

$c = <STDIN>;

print "Temperature in F is ". (9/5*$c+32);


Control Structures

Making decisions

Control structures

Conditional

I if/else/elsif:if ($x < $y) {$x=1} else {$y=1};

I unless: unless ($ok) { die "Error in file I/O"; }

I no explicit switch/case

I implicit through use of short-circuit(-e "f.dat") or ($numf++)

I ternary conditional operator —cond ? ex1 : ex2


Control Structures

Making decisions

Example

unless (-e "data.dat")

{ die "File data.dat not found" };


Control Structures

Making decisions

Example

$c = $a cmp $b;

if ($c==0) {

print "The strings are the same.\n";

} elsif ($c <0) {

print "$a comes before $b\n";

} else {

print "$b comes before $a\n";

}


Control Structures

Loops

Loops

I while: while (cond} { ... }

I for statement: C-style:for (init; condition; change) { ... }

I foreach


Control Structures

Loops

Example while loop

Read in a list of numbers terminated by 0. Compute the sum.

$sum =0;

$num=<STDIN >;

while ($num != 0) {

$sum=$sum+$num;

$num=<STDIN >;

}

print "Sum is $sum\n";


Control Structures

Loops

use strict;

my $i;

my $j;

for ($i=1; $i <11; $i++) {

for ($j=0; $j<$i; $j++) {

print "*";

}

print "\n";

}


Control Structures

Loops

use strict;

my $i;

for ($i=1; $i <11; $i++) {

print "*" x $i;

print "\n";

}


Control Structures

Other flow of control. . .

Other flow of control. . .

I next, last continue and break

I scalar range operator: ..don’t use unless you’re an expert.

I gotodon’t use


Simple process control


Can execute Unix commands or executables usingthe backtick operator ‘

$today = ‘date "+%C%y%m%d"‘ ;

print "Today is $today\n";



Other system calls

system

Similar to backtick but returns return code

$x = system("ls");

print "Returned <$x >\n";

execSimilar to system, but does not wait for completion.


File operations

File operations

To read data from a file or write it to a file, use afile handle.

I Need to associate file handle with external file.

I File can be virtual – could be a pipe.


File operations

Files in general

Files in General

File handle is the Perl construct used to manipulatefiles

I Open it – associate with external file or pipe

I Use it

I Close it


File operations

Files in general

I open(DATAF, "figs.dat") read

I open(DATAF, "<figs.dat") read

I open(DATAF, ">figs.dat") write

I open(DATAF, ">>figs.dat") append

I open(DATAF, "| output-pipe-cmd"); setup output filter

I open(DATAF, "input-pipe-cmd| "); set upinput filter

To print to a file that is open for writing, use therelated file handle in the print statement.


File operations

Files in general









File operations

Files in general









File operations

Files in general









File operations

Files in general









File operations

Files in general









File operations

Files in general









File operations

Files in general









File operations

Input from files

Input from files

Suppose that INP is a file handle. A reference to<INP> does the following:

I Returns the next line of input;

I Consumes the input – in the case of a file,advances the file pointer to the next line of thefile.

I NB: end-of-line marker read in.


File operations

Input from files

for($i=0; $i <3; $i++) {

$x=<INP >;

print "**$x##"

}

Assuming that file contains apple, banana, cherry.

**apple

##**banana

##**cherry

##


File operations

Input from files

for($i=0; $i <3; $i++) {

$x=<INP >;

print "**$x##"

}

Assuming that file contains apple, banana, cherry.

**apple

##**banana

##**cherry

##


File operations

Input from files

To add up the numbers in a file with three numbers.

open(INP ,"nums.txt");

$x1 = <INP >; $x2 = <INP >; $x3 = <INP >;

print "Answer is " . $x1 + $x2 + $x3;

or

open(INP ,"nums.txt");

print "Answer is " . (<INP > + <INP > + <INP >);


File operations

Input from files

What’s the difference between

print "Answer is " . (<INP > + <INP > + <INP >);

and

print "Answer is . (<INP > + <INP > + <INP >) ";


File operations

Input from files

Add up odd numbers in file

print "Enter the file name: ";

$namef = <STDIN >;

open(DATAF , $namef );

while($x = <DATAF >) {

if ($x % 2) {

$sum = $sum + $x;

}

}

close(DATAF );

print "The sum is $sum\n";


File operations

Input from files

A common Perl idiom is

open(DATAF ,$namef) or die "Can’t open $namef";


File operations

Implicit operands

Implicit Operands

Perl uses implicit operands extensively.

I We are told that this is a feature.

I The main culprit: $_ or $ARGSet (among other places) by reference to a filehandle. Many operators use $ARG if not givenexplicit operator explicitly.

I The following prints out the contents of a file:while (<DATAF>) {print};


File operations

Implicit operands

Exercise

Write a Perl program that reads integers from a filecalled nums.dat and counts how many numbers are

I less than zero

I between zero and 10

I greater than or equal to 11

If there is no such file, your program should print anerror message and halt.


Arrays/lists

Arrays/lists

Array variables are prefixed by @

my @days;

@days = ("Sun", "Mon", "Tue", "Wed", "Thu", "Fri","Sat");

I Arrays indexed from 0

I Individual element are scalars: so $days[0]


Arrays/lists

Arrays/lists

Array variables are prefixed by @

my @days;


I Arrays indexed from 0I Individual element are scalars: so $days[0]


Arrays/lists

Common to use foreach


foreach $d (@days) {

print "Day $d\n";

}


Arrays/lists

Quote words

Short hand for arrays of words

@days = qw(Sun Mon Tue Wed Thu Fri Sat);


Arrays/lists

@ARGV – program’s arguments

@ARGV: Predefined variable array

I Contains the program’s arguments – valuespassed by the caller.

I If the program is run as follows:./example.pl apple pear 1 2 3

array @ARGV contains those valuesI $ARGV[0] is apple;I $ARGV[3] is 2;


Arrays/lists

Array semantics

Expressions are evaluated in scalar or list context.

I Similar or identical expressions can evaluate todifferent things in different contexts.

Context determined by operation:

I LHS of assignment determines context

I Operators or functions may determine context


Arrays/lists

Array semantics

Expressions are evaluated in scalar or list context.

I Similar or identical expressions can evaluate todifferent things in different contexts.

Context determined by operation:

I LHS of assignment determines context

I Operators or functions may determine context


Arrays/lists

Array contextI print @days yields SunMonTueWedThuFriSat

I @weekdays = @days[1..5] array $weekdays set toMon .. Fri

I @days[1] is a one element array

Scalar context

I $numdays = @days; sets $numdays to 7

I

$y = ("Sun","Mon","Tue","Wed","Thu","Fri","Sat")

sets $y to Sat

I $weekdays = $days[1..5] sets $weekdays to 5

I $#days is 6 – the highest index of @days


Arrays/lists




Scalar context


I


sets $y to Sat




Arrays/lists




Scalar context


I


sets $y to Sat




Arrays/lists




Scalar context


I


sets $y to Sat




Arrays/lists

Useful functions

I push : adds something to the right of the array

I pop : removes the rightmost element of thearray.

I shift/unshift

I sort: returns a sorted listby default string order ascending, but can bechanged.


Arrays/lists

Arrays are inherently 1D

I Need references to handle multi-dimensionalarrays


Arrays/lists

Asides

Asides on strings

I splitting

@x = split ";", "1;2;3";

foreach $s (@x) {

print "$s\n";

}

I chop, chomp.

I see regexes later


Hash tables

Hash tables

Unordered set of scalars which allows fastinformation retrieval:

I Elements accessed/indexed by a string valueassociated with it

I Variables prefixed by a %, e.g. %currencyI Individual elements are scalar:$currency["South Africa"]


Hash tables

$currency["South Africa"]="ZAR";

$currency["Britain"] = "GBP";


Hash tables

Example

%day2num = ("Sun",0, "Mon",1, "Tue",2, "Wed",3,

"Thu",4, "Fri",5, "Sat",6)

Better is

%day2num= ("Sun"=>0, "Mon"=>1, "Tue"=>2, "Wed"=>3,

"Thu"=>4, "Fri"=>5, "Sat"=>6)

Referred to as $day2num{"Wed"}


Hash tables

Useful hash operations

I keys returns list of keys in hash table.Order non-deterministic.

I values returns list values stored in the hashtable. Order of no significance.

I sort keys %table lists the keys inlexicographic order


Hash tables






Hash tables






Hash tables

Example

# read in from file

while ($name = <INP>) {

$reg = <INP>;

$carreg{$name} = $reg;

}

# print out in order of name

foreach $n (sort keys %carreg) {

print "Car reg of $n is $carreg{$n} \n";

}


Hash tables

Can also tell sort how to sort:

I sort {$a <=> $b} list sorts the list innumeric order.

I

sort {$table{$a} cmp $table{$b}} keys %table

sorts the keys so that corresponding hash tableelements are in order


Hash tables

#print out in order of car reg

foreach $n

(sort {$carreg{$a} cmp $carreg{$b}}

keys %carreg) {

print "$carreg{$n} owned by $n\n";

}


Procedures

Procedures

Perl has procedures but its parameter passingmechanism is poor.

I Suppose we have a procedure plus that addsup two numbers.

Called:

$x = &plus($a, $b);

where $a and $b are the parameters.

Call by value semantics


Procedures

Declaring a subroutine

To declare a subroutine: sub NAME BLOCK

e.g.

sub printhead {

print "Name Age Number Balance\n";

}

...

&printhead();


Procedures

Parameters

Procedures have one formal parameter: @_

I The values of the actual parameter are given tothe formal parameter (call-by-value).

I @_: a list local to the procedure.

sub plus { sub plus {

$x = $_[0]; my ($x,$y) = @_;

$y = $_[1]; return $x+$y;

return $x + $y; }

}


Procedures

A common idiom uses shift

sub plus {

$x = shift @_;

$y = shift @_;

return $x + $y;

}


Procedures

ExampleA procedure to add up a list:

sub addlist {$sum = 0;

foreach $num (@_) {$sum += $num};

return $sum;}

sub proclist {

my ($f1,$f2,@nums) = @_;

foreach $n (@nums) {$sum = $sum+$n; }

return ($f1*$f2*$sum);

}

$x = &addlist(1,2,3,5,6,9);

$y = &proclist(3,4,1,1,0,2);


Procedures

The following does not work (s

sub dotprod {

(@a,@b) = @_;

...

...

}

@x = (1,2,3);

@y = (4,5,6);

&dotprod(@x,@y);


Procedures

I Formal parameter is an unstructured list (possible dangerin passing multiple lists as parameters)

I Forward declarations: sub addlist;

Telling Perl that addlist is a procedure which will bedeclared elsewhere.

I Can have anonymous procedures that get assigned tovariables.

I Weakness in parameter system can be overcome by usingreferences.

I Default: all variables are global wherever defined. This iseven recognised by the Perl community to be a Bad ThingDeclare variables local inside procedures.Best way: prefix first use of variable in a procedure withwith my. Lexical scoping.


References

References

References are mechanisms for a variable to refer tosomething, e.g. (1) another variable; (2) a piece ofdata; (3); a function

Symbolic references

Can turn a string into a variable

$x = 123;

$y = "x";

$$y = 5;

print $x;


References

Hard referencesCan use the \ operator to dereference a variable,and -> to dereference.

@a = (10,20,30,40,50);

$x = \@a;

for $e ( @{$x} ) {

print "$e\n";

}

print $x->[0];


References

Call by reference

sub count {

my ($fname , $rcount) = @_;

chomp $fname;

unless (-T $fname) { return; }

open(FINP ,$fname );

while (<FINP >) {

$$rcount ++

}

close(FINP);

}


References

@files = ‘ls -1 $ARGV [0]‘;

$n = 0;

for $f (@files) {

&count($f ,\$n);

}

print "Total number of lines is $n\n";


References

Passing multiple arrays

sub dotprod {

($a,$b)=@_;

....

}

@x = (1,2,3);

@y = (4,5,6);

&dotprod(\@x,\@y);

Note that only scalars are passed.


References

Refs to anonymous arrays and hashes

Square brackets creates an array – returns areference

$a = [ 10, 20, 30, 40];

I Error: @a = [ 10, 20, 30, 40];

Curly braces creates a hash – returns a reference

$day = {"sun"=>0, "mon"=>1, "tues"=> .....}


Modules

Modules

A module is a collection of code, data structuresthat can be used as a library

I Often see it in OO programming

Magic word is use

use IO;

I To access something is in a module use ::. So,module::thing

I Modules can be nested.


Modules

Example

use IO:: Compress ::Bzip2;

IO:: Compress :: Bzip2::bzip2 ("do.pl", "do.pl.bz2")

Can also import specific things

use IO:: Compress ::Bzip2 qw(bzip2 $Bzip2Error );

unless (bzip2 ("do.plx", "do.pl.bz2")) {

warn "Compression failed returning : $Bzip2Error";

}


Modules

Example

use IO:: Compress ::Bzip2;

IO:: Compress :: Bzip2::bzip2 ("do.pl", "do.pl.bz2")

Can also import specific things

use IO:: Compress ::Bzip2 qw(bzip2 $Bzip2Error );

unless (bzip2 ("do.plx", "do.pl.bz2")) {

warn "Compression failed returning : $Bzip2Error";

}


Objects

Objects

Data structures which

I contain data, know functions that can apply tothem;

I anonymous

I accessed through references

I typically organised in classes


Objects

use Bio::Seq;

$seqio = Bio::SeqIO ->new(’-format ’ => ’embl’ , -file => ’myfile.dat’);

$seqobj = $seqio ->next_seq ();

$seqstr = $seqobj ->seq ();

$seqstr = $seqobj ->subseq (10 ,50);

@features = $seqobj ->get_SeqFeatures ();

foreach my $feat ( @features ) {

print "Feature ",$feat ->primary_tag ,

" starts ",$feat ->start ,

" ends ",

$feat ->end ," strand ",$feat ->strand ,"\n";

}


Regular expressions

Regular expressions, matching, and more

Perl’s regular expression support powerful

I concise way of describing a set of strings.

Typical: a command uses a regular expression toprocess some argument.

I Example: split uses a regex to split a string

$line = "Gauteng;Johannesburg GP 7";

@info = split("[ ;]", $line);

($prov, $capital,$reg, $pop) = split("[ ;] ", $line);


Regular expressions

Specifying Regular Expressions

I Most characters stand for themselves: F Fred 6312

I \ | ( ) [ { ^ $ * + ? . are metacharacters (havespecial meaning)

I escape with a backslash for the chars:Fred stands for the string (Fred)

I To group things together, use parentheses.

I To specify alternatives, use |

(green|red) apples stands for green apples or redapples


Regular expressions









Regular expressions









Regular expressions









Regular expressions









Regular expressions


I A list of characters in square brackets matches any of thecharacters.

I [YyNn] matches any of an upper or lower case “y” or“n”.

I [A-Za-z0-9] is all the alphanumeric characters


Regular expressions

I \n new line; \t tab; \s a whitespace;

I \d digit; \D non-digit;

I \w a word charater, \W a non-word character

I . anything but a \n

I m{3} stands for mmm(map){2,3} stands for mapmap or mapmapmap

I m* stands for 0 or more m’s

I m+ stands for 1 or more m’s

I Many others. Rules long and complex.


Regular expressions










Regular expressions










Regular expressions










Regular expressions










Regular expressions










Regular expressions










Regular expressions

Processing with regular expressions

Various commands that allow

I finding a pattern matching a regular expression in a string;

I extracting out the regular expression;

I substituting;

I or other modification

e.g. matching, substitution, translation, substitution.


Regular expressions

Splitting up input

split(pat,string)

Takes the input string and splits the string wherever thepattern pat occurs.

I returns a list of strings as a result

I can choose whether the split pattern is part of thereturned string or not

$x = split /\*|::/, $c processes the string $c, splits itwherever a * or :: appear, and returns the split list.So, if

$c="Dat: 23 * Mon: 11; LgA -632 ::: LgB -217* a",

$x= ?


Regular expressions

Splitting up input

split(pat,string)





$c="Dat: 23 * Mon: 11; LgA -632 ::: LgB -217* a",

$x= ?


Regular expressions

Splitting up input

split(pat,string)





$c="Dat: 23 * Mon: 11; LgA -632 ::: LgB -217* a",

$x= ?


Regular expressions

Matching

I m/regexpr/ returns true if the regular expression matches$_.

The m is optional in most cases (unless slash in string)m-/apple-

I To match a particular string use the binding operator: =~

$inp =~ m/(Pascal)|(\WC\W)|(C\+\+)/;

I A number of modifiers available: i (case insensitive) g(global)Count number of matches in a string:

while ($inp =~ m/(Pascal)|(\WC\W)|(C\+\+)/g) {

$wrd++

}


Regular expressions

Matching

I m/regexpr/ returns true if the regular expression matches$_.The m is optional in most cases (unless slash in string)m-/apple-


$inp =~ m/(Pascal)|(\WC\W)|(C\+\+)/;



$wrd++

}


Regular expressions

Matching



$inp =~ m/(Pascal)|(\WC\W)|(C\+\+)/;



$wrd++

}


Regular expressions

Matching



$inp =~ m/(Pascal)|(\WC\W)|(C\+\+)/;



$wrd++

}


Regular expressions

Extracting patterns*Can refer to and extract out subexpressions matched $1, $2,. . .

I To find a repeat m/\b(\w+)\s+\1/

I To extract out

while ($line = <INP>) {

$line =~ m/(\w+)\s+(\w+)/;

$name = $1;

$reg = $2;

$carreg{$name}=$reg;

}

Look for repeats:


Regular expressions

Substitution

s/regexprA/regexprB/ substitutes regular expression A byregular expression B. Default string processed is $ .s/Apple/Orange/; Replaces the first occurrence of Apple in$ with Orange

Similar modifiers to m: i, g,...

Use the binder operator to choose the string chosen:

$name = "Alan Turing";

$name =~ s/(\w)\w*/\1\./;

$c = ($name =~ s/(\w)\w*/\1\./);

$c = ($name =~ s/(\w)\w*/\1\./g);


Regular expressions

Translation

Use the tr or y operator.

I Operates on strings, rather than regular expressions.

tr/range1/range2/ replaces any character in range1 andreplaces it with the corresponding character in range2.

I $c =~ tr/a-z/A-Z/ change every character in $c touppercase.

I $cnt = $c =~ tr/a-z/a-z/ sets $cnt to the number oflower case letters in $c.


And more

And more

I maths functions

I more sophisticated I/O

I lots of built in variables: e.g. \$NR

I modules, OO

I IPC, networking

I process control

Date post:	18-Jun-2020
Category:	Documents
Upload:	others
View:	4 times
Download:	0 times

Introduction to Perl · Introduction to Perl Perl’s Data Structures Perl’s Data Structures I...

Documents