Essential Unix and Linux!Perl for Bioinformatics, 140.636!
F. Pineda
Generic computer architecture
Fig. 1.2 From “Designing Embedded Hardware”, 2nd Ed. by John Catsoulis
Memory
Storage
OS concepts
The file system
Devicese.g. Keyboard,
printerscreen, ethernet,
mouse, etc.
Operating system (e.g. Linux, Unix, MacOS
Shellinterprets typed commands
GUIinterprets mouse !
and keyboard events
Processese.g. editors,
word processors, spreadsheets,scripts, etc.
ApplicationsExcel, R, Perl scripts
n Unixn Linuxn OS X, MacOSn Windowsn CPM
The Operating System (OS) orchestrates and coordinates computing on the machine
Unix Phylogeny
Essential unix (operating system) concepts
1. A processEverything that “runs” under unix is a process, e.g. commands, spreadsheets, terminal windows, etc.Relevant commands: ps, top
2. A file & the hierarchical file systemdirectories(folders), data files, applications. On the disk, everything is a file. Even directories! Files are not processes!Relevant commands: ls, df
3. A shell A shell is a process. The shell accepts command lines and feed
them to the unix kernel. Command lines consist of commands or programs to execute along with information needed to execute them
ProcessesWhat processes am I running from my shells?!
ps
Which processes are using the most cpu?!top
What processes is everyone running from shells?!ps -a
What processes of all kinds are running?!ps -ax
File system concepts
/
bootbin dev etc home2 lib mnt opt proc root sbin tmp usr var vmlinux
140.636F08 faculty
stu08-01 stu08-02stu08-00 Stu08-03
bin local
fernando
private_htmlpublic_html
home
public
example.txt
The hierarchical file systemThe file system is a tree
/home2/140.636F08/stu08-00/private_html
Navigating the hierarchical file system
1. Navigation commandsn ls -- list directory contents n pwd -- prints the absolute path to the working directory n cd -- change directory to a directory specified by a pathn df -- free disk spacen du -- disk usage statistics
2. Pathsn Absolute path
n A fully qualified path must be specified from the root directoryn / is the symbol for the root directory
n Relative pathn Always defined relative to the working directoryn . is the symbol that means the working directoryn .. is the symbol that means the directory above the working directory
n Executing an executable that resides in the current working directoryn ./filename
Review: commands to get around the file system
n pwd the command that tells you where you are, i.e. print working directoryn ls the command that lists the contents !
of the current directoryn cd the command that changes the !
current directory. In other words ! it is how you move around the ! file system
Memorize these three navigation commands or die!
Files permissionsn to see what permissions a file has use ls
command with -l option:n ls -l filename
n to change file permissions use the chmod command (see linux quickstart notes)
read write execute
user
read write execute
group
read write execute
otherfiletype
d r w x r - x r - x
Basic Bash shell commands
simple commandsn Commands
n examples: clear, cd, ls, man, wcn Syntax: command [options] parameters !
n Optionsn An option is a way of telling Linux to perform a command in a
particular way (like pressing alt-key while clicking a mouse)n Typically a minus sign followed by one or more characters!
and sometimes an argument !
n Parametersn Often filenames or arguments for the command!
n Wildcardsn used to represent filenamesn ? matches any single charactern * matches any number of characters
some example commands
wc helloworld.plwc -l helloworld.plwc -l *.plls *.*
On-line help for commands
n man commandn command -helpn apropos keywords
Standard input & output
two very important Unix concepts
that you absolutely must understand
Standard I/On Commands are processes, so they have a default standard input and
a default standard output !!!!
n For commands executed in shell window:n standard input is connected to the keyboardn standard output is connected to the screen
processstandard input standard output
shellprocess
STDIN STDOUT
Keyboard device graphics device
When you invoke the Perl interpreter it is a process!
perlprocess
STDIN STDOUT
Keyboard device graphics device
Redirection and pipes
two more very important and related ideas
that you absolutely must understand
Standard i/o streams can be Redirected
n The standard output of a command can be redirected to a file by the “>” redirector
ls process
STDIN
Storage device
STDOUT
graphics device
ls -l
results.txt
STDOUT
file
ls -l > results.txt
Standard i/o streams can be Redirectedn a file can be redirected to the standard input of a command by the
“<“ redirector.
wcprocess
STDOUT
graphics device
STDINresults.txt
file
wc -l < results.txt
Keyboard device
STDIN
wc -l
a pipeline of processes
process1
file1.txt
STDOUT process2STDIN
file2.txt
STDOUT STDIN process3
a “pipeline” of processes!(without the intermediate files)
process1 process2STDIN process3STDOUT STDIN STDOUT STDIN
PipesThe standard output stream of one command can be piped to the �
standard input of another command by using thepipe “|” symbol. No intermediate file is required.
Example: What does this do?
ls -l | wc -l
ls -l wc -lSTDINSTDOUT STDIN
Let’s use perl to illustrate pipes!1st recall that the Perl interpreter is a process
perlprocess
STDIN STDOUT
Keyboard device graphics device
In the shell start the perl interpreter and feed it a text file of perl statements(examine the file of perl statements)
perl helloworld.txt
In the shell start the perl interpreter and feed it another text file of perl statements (examine the file of perl statements)
perl helloperl.txt
What does this command line do?
perl hello.txt | perl helloperl.txt
perl process
perl processSTDINSTDOUT STDIN STDOUT
demo…
Execution of unix commands and scripts
How the shell executes commands and applications
1. If the shell does not recognize the command name as one of it’s built-in commands, it starts a search for an executable file with the same name.
2. An executable file is either an application binary or a special kind of text file called a script
3. The shell searches through all the directories on the disk specified by the PATH shell variable (demo)
4. And executes the first executable that it finds with the specified name.
5. The file must have “execute permission in order to be executed, otherwise you will see a “permission error”
How the shell executes a “script”
1. If the executable is a text file, the shell treats it as a “script”. A script must have a special first line
1. The first line must start with #!, the rest of the first line is the path to an executable
2. The executable is launched3. The file itself is passed to the standard input (STDIN) of the
the executable. 2. Examples
helloworld.plhelloperl.pl
Equivalent command lines
./helloworld.pl | ./helloperl.pl
perl hello.txt | perl helloperl.txt
Perl scripts are the standard way of invoking the perl interpreter. You almost never see perl invoked “by hand”, we did it just to
illustrate what is going on “under the hood.”