Brief introduction to UNIX

A. Emerson

CINECA, High Performance Systems

Contents

Using Unix commands− command syntax

Getting Started– Getting help, identity, logging out

Files and directories− Making, renaming, deleting and copying.− Examining file contents.

Further file handling− File compression and making archives.− File permissions

Miscellaneous− Jobs and processes− Editing with vi.

Unix commands - usage

Unix commands are normally used in the form:

<command> <one or more options> <arguments>

Where the options are generally included with the – sign.

Example

ls -l –a /usr/local/bin/filesls -l –a /usr/local/bin/files

Single letter options can usually be combined:

ls -la /usr/local/bin/filesls -la /usr/local/bin/files

Unix commands - wildcards

If you want to do the command on multiple files you can use the * “wild card” character.

Examples

$ mv *.pl programs/perl/

$ ls data/*

$ mv *.pl programs/perl/

$ ls data/*

rm *.* rm *.*

Be careful with rm (which does a delete) because there is no way to undo it..

AARGH!

Getting StartedCommand

− man

Purpose− Gives you the manual page for a given command

Example$ man pwd

pwd(1) NAME

pwd - print working directory name

SYNOPSIS

pwd

DESCRIPTION

pwd prints the path name of the working (current) directory. pwd is both an explicit command (invoked as /usr/bin/pwd), as well as a builtin

$ man pwd

pwd(1) NAME

pwd - print working directory name

SYNOPSIS

pwd

DESCRIPTION

pwd prints the path name of the working (current) directory. pwd is both an explicit command (invoked as /usr/bin/pwd), as well as a builtin

Getting Started

Command− id

Purpose− Tells you your username (!) and what group you

belong to.

Example

$ id

uid=50083(aem0) gid=30(cineca)

$ id

uid=50083(aem0) gid=30(cineca)

Getting Started

Command− exit (or ctrl-d)

Purpose− Logs you out from the system

Example

$ exit$ exit

Files and directories

Command− mkdir

Purpose− Makes a directory

Common options− -p

creates all the sub-directories in a path if they don’t exist

Examples

$ mkdir perl-programs

$ mkdir –p 2003/jan/data

$ mkdir perl-programs

$ mkdir –p 2003/jan/data

Files and directories

Command− cd

Purpose− Changes directory. With no arguments changes to

home directory.

Examples

$ cd my_data

$ cd /usr/local/bin/programs

$ cd

$ cd my_data

$ cd /usr/local/bin/programs

$ cd

Files and directories

Command− mv

Purpose− Moves or renames a file or directory

Common options− -i

Asks confirmation before overwriting another file or directory

Examples

$ mv first.pl perl-programs/jan

$ mv blast.out blast.out.bak

$ mv *.seq sequence-dir

$ mv first.pl perl-programs/jan

$ mv blast.out blast.out.bak

$ mv *.seq sequence-dir

Files and directories

Command− rm

Purpose− Deletes or renames a file

Common options− -i

Asks confirmation first− -r

deletes all sub-directories of a directory (VERY DANGEROUS)

Examples

$ rm *.old

$ rm –i blast.pl

File blast.pl. Remove ? (yes/no)[no] :

$ rm *.old

$ rm –i blast.pl

File blast.pl. Remove ? (yes/no)[no] :

Files and directories

Command− cp

Purpose− Makes a copy of a file or directory

Common options− -i

Asks confirmation first if overwriting another file− -r

copies all files of all sub-directories of a directory

Examples

$ cp program.f90 program.f90.old

$ cp blastdir/*.out .

$ cp program.f90 program.f90.old

$ cp blastdir/*.out .

. means the current directory

Files and directoriesCommand

− lsPurpose

− Lists files and directoriesCommon options

− -t sort by modification time

− -l long format, gives all details of the file (very useful)

− -a

shows file beginning with . (not visible with just ls)

$ ls –lt

total 136

-rw-r--r-- 1 bioinf00 cineca 15678 May 20 2002 test.out

-rw-r--r-- 1 bioinf00 cineca 2939 May 20 2002 test.bas

-rw-r--r-- 1 bioinf00 cineca 53541 May 20 2002 prova.bas

$ ls –lt

total 136

-rw-r--r-- 1 bioinf00 cineca 15678 May 20 2002 test.out

-rw-r--r-- 1 bioinf00 cineca 2939 May 20 2002 test.bas

-rw-r--r-- 1 bioinf00 cineca 53541 May 20 2002 prova.bas

size in bytes

Files and directories

Command− more (traditional Unix), less (Linux)

Purpose− Allows you to view the contents of a file.

$ less fasta_1.seq

>THC479287

CAGAACAGTAGCTAAGAGTCAAACCATGCGTTTGAGTCTCAGCTCTGCT

CTCCACTTTACCTTTTGAAGGAGATCCGGACTACAAAGGAAAGGTCTTT

CTAATTTTTATCTTTTTTTTTTTTTAAACAGGTGAAGGTGCCGAGCTAT

AGAAATACAAAATAAAGATCACACATCAAGACTATCTACAAAAATTTAT

AGAAGAAAAGCATGCATATCATTAAACAAATAAAATACTTTTTATCACA

AGGAA

$ less fasta_1.seq

>THC479287

CAGAACAGTAGCTAAGAGTCAAACCATGCGTTTGAGTCTCAGCTCTGCT

CTCCACTTTACCTTTTGAAGGAGATCCGGACTACAAAGGAAAGGTCTTT

CTAATTTTTATCTTTTTTTTTTTTTAAACAGGTGAAGGTGCCGAGCTAT

AGAAATACAAAATAAAGATCACACATCAAGACTATCTACAAAAATTTAT

AGAAGAAAAGCATGCATATCATTAAACAAATAAAATACTTTTTATCACA

AGGAAfasta_1.seq (END)

Files and directories

Command− head, tail

Purpose− Allows you to view the first lines of a file (head) or the last lines

of a file (tail)

Common options− n

The number of lines to show. The default is 10.

$ head -5 seq.fasta

>THC479329 SWI/SNF complex 155 KDa subunit^^SWI/SNF complex 155 KDa subunit (BAF155)^^SWI/SNF relatedTTTTAGAATCCAGAAATGGTGTTCCATTTATTCACTGAAAAAGAGAGAGTTCATTCATTTTCTCCATTCTTGCCAAACTCCCTCCCCTCATTTTTTCCACACTGAGAAACATGTTTGTACAAAAACCACATATTATTCCCCCCCCTCTGGCTGAATTACAGGAATAAAACCAGATCAAAGACATGAAAAGAAAAAG

$ head -5 seq.fasta

>THC479329 SWI/SNF complex 155 KDa subunit^^SWI/SNF complex 155 KDa subunit (BAF155)^^SWI/SNF relatedTTTTAGAATCCAGAAATGGTGTTCCATTTATTCACTGAAAAAGAGAGAGTTCATTCATTTTCTCCATTCTTGCCAAACTCCCTCCCCTCATTTTTTCCACACTGAGAAACATGTTTGTACAAAAACCACATATTATTCCCCCCCCTCTGGCTGAATTACAGGAATAAAACCAGATCAAAGACATGAAAAGAAAAAG

Further file handlingCommand

− compress (standard UNIX), gzip (GNU version – faster)− uncompress, gunzip

Purpose− Compresses text files to save disk space

Common options− -v

verbose, gives % compression ratio

$ ls -l MAG500-rw-r--r-- 1 aem0 cineca 3249198 Jan 29 2002 MAG500$ gzip –v MAG500gzip -v MAG500MAG500: 70.1% -- replaced with MAG500.gz$$ ls –l MAG500.gz-rw-r--r-- 1 aem0 cineca 971325 Jan 29 2002 MAG500.gz

$ ls -l MAG500-rw-r--r-- 1 aem0 cineca 3249198 Jan 29 2002 MAG500$ gzip –v MAG500gzip -v MAG500MAG500: 70.1% -- replaced with MAG500.gz$$ ls –l MAG500.gz-rw-r--r-- 1 aem0 cineca 971325 Jan 29 2002 MAG500.gz

Making archivesCommand

− tarPurpose

− Creates an archive of files and directories. Many Unix software packages consist of a tree of sub-directories which can be difficult to transfer between different machines. tar can be used to create a single archive file which when untarred re-creates the original directory structure

Common options− -c

creates an archive− -f

uses a file for the archive (you can also use CDs,tapes, etc)

− -xextracts file from an archive

− -vverbose – tells the user what tar is doing

(RECOMMENDED)

Creating archives - Example

$ ls -Fblast/$ tar -cvf blast.tar blastblast/blast/results/blast/input/blast/input/input1.datblast/input/input2.datblast/input/input3.datblast/data/blast/data/blast1.outblast/data/blast2.outblast/data/blast3.outblast/data/blast4.outblast/data/blast5.outblast/data/data-old/blast-old.outblast/data/blast0.out

$ ls -Fblast/$ tar -cvf blast.tar blastblast/blast/results/blast/input/blast/input/input1.datblast/input/input2.datblast/input/input3.datblast/data/blast/data/blast1.outblast/data/blast2.outblast/data/blast3.outblast/data/blast4.outblast/data/blast5.outblast/data/data-old/blast-old.outblast/data/blast0.out

directory to archive

name of archive file

Extracting archives - Example

$ cd new-dir$ tar -xvf blast.tarblast/blast/results/blast/input/blast/input/input1.datblast/input/input2.datblast/input/input3.datblast/data/blast/data/blast1.outblast/data/blast2.outblast/data/blast3.outblast/data/blast4.outblast/data/blast5.outblast/data/data-old/blast/data/blast0.out

$ cd new-dir$ tar -xvf blast.tarblast/blast/results/blast/input/blast/input/input1.datblast/input/input2.datblast/input/input3.datblast/data/blast/data/blast1.outblast/data/blast2.outblast/data/blast3.outblast/data/blast4.outblast/data/blast5.outblast/data/data-old/blast/data/blast0.out

Often gzip is combined with tar archives to give files like

blast.tar.gz

or

blast.tgz

Sometimes called tarballs.

File permissions

$ ls -l blast.tar

-rwxr--r-- 1 aem0 cineca 30720 Mar 13 10:47 blast.tar

$ ls -l blast.tar

-rwxr--r-- 1 aem0 cineca 30720 Mar 13 10:47 blast.tar

File permissions

rwx r-- r---

owner group otherspecial

r - read access

w - write access (for a directory means files can be deleted in the directory, even if the files don’t have write access)

x - executable (searchable for a directory)

- permission not set

File permissionsCommand

− chmod

Purpose− Changes the permissions of a file or directory. Only the owner of

a file, or root, can change the permissions.

Common options− -R

changes all the permissions in a directory, including sub-directories

$ chmod u+x myprog.pl

$ chmod g+w,o-w seq.dat

$ chmod +r *.fasta

$ chmod 777 *.prog

$ chmod u+x myprog.pl

$ chmod g+w,o-w seq.dat

$ chmod +r *.fasta

$ chmod 777 *.prog

make file executable for owner

write access for group, no write for others

add read access for all

octal notation, here = +rwx for all

Running programs

To run a program which is an executable file, just type the name of the file:

$ my_prog.pl$ my_prog.pl

However, like this the terminal cannot be used until the program finishes. Add an & to return control to the user (running in the background):

$ my_prog.pl &

[1] 31705

$ my_prog.pl &

[1] 31705

Now the user can do other things, logout and go home, etc.Note that Unix assigns a job number and a process number to the running program.

Running programs

To see what programs are running you can use the jobs command:

But this only applies to programs run during the same session (the same shell). To see all programs (processes) being run use the ps command:

$ jobs

[1] + Running myprog.pl

$ jobs

[1] + Running myprog.pl

$ ps –u aem0

31705 pts/3 00:00:40 myprog.pl29775 pts/3 00:00:00 tcsh31738 pts/3 00:00:00 ps

$ ps –u aem0

31705 pts/3 00:00:40 myprog.pl29775 pts/3 00:00:00 tcsh31738 pts/3 00:00:00 ps

Editing with vi

vi is the standard Unix text editor and is present on every Unix system.

$ vi myprog.pl$ vi myprog.pl

vi has 3 modes:

1. Command mode− For manipulating and moving through the the text

2. Line mode− For special commands and interacting with Unix.

3. Insert mode− For entering text, i.e. writing programs, entering

data, etc.

Editing with vi

Command mode – the usual and initial mode (i.e. when starting vi)

Commands include− ←↑↓→ arrow keys move the cursor− hjkl same as arrow keys− x delete a character− dw delete a word− dd delete a line− 3dd delete 3 lines− u undo previous change− ZZ exit vi, saving changes

Editing with vi

Line mode – entered by typing :, / , ? or ! .

Commands include− :q! save file, discarding changes− :q quit− :e filename edit a new file − :w filename write with new filename− :wq write file and quit− :!cmd run Unix command− /string look for string

RETURN executes command and returns to command mode

Editing with vi

Insert mode – entered by typing any of the following in command mode

− a append after cursor− i insert before cursor− o open line below− O open line above− Rtext replace with text

to exit insert mode, and return to command mode, type <ESC>.