Linux command line basics I: files and

folders YanbinYin

1

http://cys.bios.niu.edu/yyin/teach/PBB/Yin-chapter1.pdfhttp://korflab.ucdavis.edu/Unix_and_Perl/current.html

UseVirtualprivatenetwork(VPN)toconnecttoourLinuxseverfromhome

UseInternetExplorerorFirefoxGotohttps://secure.niu.eduPutinyourZidandpassword

ThenextpagewillinstallCISCOAnyConnect SecureMobility softwareItwillaskyoutoinstallJAVAifyoudon’thaveityet

Afterthatyouhaveaniconattherightbottomcorner, showing theVPNsoftware

Andyouwillbeabletoaccessthe10.157.217.4 Linuxserver(Ser)

3

UnderstandLinux/Unixandshell

Filesystem

Basicshellcommands

Commands fortextprocessing

Pipe:chainmultiplecommands togettextprocessingpipeline

Shellone-liner

There'salwaysmorethanonewaytodoit

4

Putty,ssh secureshellclient,bothareinstalledoncomputersofMO444

5

SecureShell(SSH) isacryptographicnetworkprotocol forsecuredatacommunication,remote command-line login, remotecommandexecution,andothersecurenetworkservicesbetweentwonetworkedcomputers

Whatisssh?

6

gototheleftbottom,search“putty”

PuttheIPaddress:10.157.217.4

HitOpen

7

HitYes

PutyourstudentIDhere(e.g.z01553886)andhitenter

Waituntilyouseethefollowing, thenputyourpassword(thesameasyourstudentID)Yourpasswordwillbeinvisible whileyouaretyping.Hitenterafteryouaredone

YourusernameandpasswordarebothyourstudentID

8

Ifyouhaveputyourpasswordcorrectly,youwillseethis,meaningyouarelogged innow:

IfyoudidNOTputyourpasswordcorrectly,youwillseethis:Justdoitagain

Aplaintextworld

Everything istext,noimages

Good:Minimalconsumptionfromgraphics, allresources(CPU,RAM,harddrive)savedforcomputation

Bad:Userunfriendly

9

z01553986@ser:~$

Username Machinename Yourhome(colon, tilde)

PromptsignCursorsitsrightafterprompt

Whatyouseeafteryousuccessfullylogged in

10

MAChasitsbuilt-in terminal.Searchterminalapplication

Typeinthispart,hitenterandthenyes,putpassword,andthenyouareconnected

11

Ubuntu hasitsGUIandterminalandifyouwanttoconnecttoanotherLinuxserver

Aftertheterminalisopened, type:ssh z01553986@10.157.217.4

12

Tosummarize:howtologin

WINDOWS

Linux Ser server MAC

puttyTerminal:ssh yyin@10.157.217.4

Terminal:ssh yyin@10.157.217.4

Terminal:ssh yyin@gaea.niu.edussh yinyb@zcluster.rcc.uga.edu…

Tologout, typelogout, exitorCtrl+d

UseVirtualprivatenetwork(VPN)toconnecttoourLinuxseverfromhome

UseInternetExplorerorFirefoxGotohttps://secure.niu.eduPutinyourZidandpassword

ThenextpagewillinstallCISCOAnyConnect SecureMobility softwareItwillaskyoutoinstallJAVAifyoudon’thaveityet

Afterthatyouhaveaniconattherightbottomcorner, showing theVPNsoftware

Andyouwillbeabletoaccessthe10.157.217.4 Linuxserver(Ser)

14

15

InthegraphicalinterfaceofWindows,LinuxandMAC,youclickyourmouse tomakesomethinghappen.

However,undercommandlineterminal(orconsole)interface,youalwaystypeinacommandusingthekeyboard andhitEntertoletsomethinghappen.

ThecommandyoutypedinisinterpretedandexecutedbytheShell,theLinuxsoftwareandinterfacetoconnecttheusersandtheOS.

Theoutputofthecommandisprintedonthescreenbydefaultorwrittentoafile.(togetherwitherrormsg ifany)

Interactivecommandlineterminal

16

Theoperatingsystem

Terminalistheinterface/Shellwhereusertalks(bytypingacommand) tothekerneltooperatehardware

Whatisshell?

17

CommandsCommandscanbeasingleword/letterorafewwordsseparatedbyspace;alwayshitEnterafteryoufinishedtyping

Thefirstword/letterisoftenaShellcommandoranexternalprogramoryourownscriptname

Otherwordscanbecommandoptions,files,foldersetc.

Examples:

yyin@ser:~$ pwd/home/yyin

yyin@ser:~$ ls /disk4/z01553986examples.desktop

Forallofyou,yourhomeisat/disk?/yourid

?couldbe1,2,3,4

e.g./disk4/z01553986

http://korflab.ucdavis.edu/Unix_and_Perl/unix_and_perl_v3.1.1.pdf

Commandsyoumustlearn

19

yyin@ser:~$ ss: command not found

yyin@ser:~$ '>>> ^C

Ifyoutypesomenot-existingcommand

Ifyoumistakenlytypeinaquoteandhitenter,yougetstucked

Ctrl+c willgetyououtofthere

Ctrl+d willlogyouout

Forpeoplewhohavenoexperienceinterminal:youarelikeinadarkroomalone.WhereamI,whatisthatsound, whatdidIjustbump into?Youcan’tsee,youhavetogetusedtousingyourothersenses…

Noshortcuts, justlearnbyusing

20

Everythingisafileorafolder/directoryinLinux

Afoldercontainsfilesor/andotherfolders(subdirectories)

Asubdirectorycancontainotherfilesorfolders

Sothefilesystemisatree-likestructure(multi-furcation)

/

/home /bin/media /usr /etc

/media/DATAPART1/media/DATAPART2/media/DATAPART3…

/home/nmcginn/home/hscao/home/yyin…

…

…

/home/yyin/book…

root

2nd level

3rd level

4th level

21http://edu.isb-sib.ch/mod/resource/view.php?id=306

22

Sinceyouareinafilesystem,knowingwhereyouareisveryimportant

Rememberpwd isthecommandtofindwhereyouare(workingdirectory)

Ifyouissueacommandwhichexpectafile/folder name,youneedtotellShellthelocationofthefile/folder, whichiscalledpath

Withoutspecifying thepath,Shellwillthink thefile/folder isinthecurrentfolder.Ifitcan’tfind thefile, itreportsanerroryyin@ser:~$ pwd/home/yyin

yyin@ser:~$ ls /disk4/z01553986examples.desktop

http://askubuntu.com/questions/17299/what-do-the-different-colors-mean-in-the-terminal

23

File/folder/programnamesCasesensitive!!!File and file are two different files

Allowtocontainletters,numbers,underscore(_),dot(.),dash(-),plus(+)butnottouseotherspecialcharactersandspaces

Example:[yyin@gaea fungal]$ lsall.hmmlib_1.75.fungal-dockerin.domain.fa.nobac.n2+.fa.id.source.nrgi.ps.col.lab

Forprograms,betterusethecorrectfileextension(.sh,.pl,.c,.py),e.g.run-blast.sh

24

Homefolder(dir)

AfteryouloginaremoteLinuxmachineoryouopenedaterminalinyourUbuntuorMACcomputer,youareatyourhome

Youcancreateordownloadfolders,files,bioinformaticssoftwares

Makeitorganizedandcleanbycreatingfoldersthathavemeaningfulnames,suchastools,data,work,project,scriptsetc.

Youcanalsocreatesubfolders,e.g.blastundertools,orproject1underproject

Don’tputeverythinginyourhomedirectory;youwilleasilyaccumulatetoomanyfiles/folders/programs/scriptsthatyouwillhaveaheadachetofindorremember

25

Directory(folder)commands

pwd findoutwhereyouare(yourcurrentdirectoryorworkingdirectory)

cd changedirectory

Remember?Saveyoufromsometyping

Don’needtotypeeverything inAlwaysuseTABkeytoauto-completeaword!!!!!!!!!!

Uparrowkeytobringyourpreviouscommands

26

RelativepathandabsolutepathFindoutwhereyouare:yyin@ser:~$ cd /home/yyin

yyin@ser:~$ pwd/home/yyin

IfIwanttochangedirectoryto/home/

yyin@ser:~$ cd ..yyin@ser:/home$

IfIwanttogobacktomyhome

yyin@ser:/home$ cd yyinyyin@ser:~$

Icanalsousetheabsolutepathtogoback

yyin@ser:/home$ cd /home/yyinyyin@ser:~$

Absolutepathorfullpath

Relativepath,relativetoyourcurrentdir:/home

27

Specialdenotations:

~or~/ yourhome

.or./yourcurrentdirectory

..or../thedir onelevelabove

../..?

Try:

cd ~

cd ..pwd

cd ../..pwd

Ifyouthinkyouarelostinthefilesystemanddon’tknowwhereyouare,alwaysrun

pwd

Fromanywheretogobacktoyouhome

cdcd ~

tilde

28

Moredirectorycommands

mkdir createadirectory

rmdir deleteanemptydirectory(havenosubdirectoriesorfiles)

Atyourhome, try

mkdir bioinfo

Remembercasesensitiveandnospecialcharactersandspace!!!

29

Listcommand

ls listwhatfilesanddirectoriesarethereinafolder

Try

lsls –l list in long listing formatls –la list all including hidden files/foldersls –lt list according to modification timels –ltr list according to time in reverse orderls .. list one level upls –l /home/yyin list things under a given folderls –l | less if there are too many files to display in

one page, use pipe and less to show page by page (will explain shortly)have to type q to exit less

Theseareoptions, usedtoenrich thefunctionalities ofacommand

30

Themanualcommand

man ls

Ifthemanualismorethanonepage,hitspaceorPgDn keytopagedown,PgUp keyoru topageup

Toexitthemanpage,hitq

ManfollowedbyanyLinuxcommandtodisplaythemanualofthatcommand

man pwdman cd…

31

Permission UserandGroupsize

Modification time

dmeansfolder

ls –lt /

32

FileandfolderpermissioncanbechangedLet’slookatoneofthefilesundermyhome:

Thefirstone“-“meansitisafile;itwillbe“d”ifitisafolder (directory).

The following ninecolumns indicatethepermission ofread(r),write(w)andexecute(x)grantedfortheuser (firstthreecolumns), thegroup(middle three)andothers(lastthree).“-“meansnopermission

-rwxrwxrwx the file could be read, write, and execute by anybody

-r-------- can only be read by the owner

yyin@ser:~$ chmod go-r /home/yyin/Unix_and_Perl_course/Data/GenBank/E.coli.genbankyyin@ser:~$ ls -l /home/yyin/Unix_and_Perl_course/Data/GenBank/E.coli.genbank-rw-r--r-- 1 yyin yanbin 10602328 Oct 9 2008 /home/yyin/Unix_and_Perl_course/Data/GenBank/E.coli.genbank

Nowyoucannotreadthatfileinmyhome

-rw-r--r--

33

HowtoviewfilesInorderforyoutoreadmyfile:

yyin@ser:~$ chmod go+r /home/yyin/Unix_and_Perl_course/Data/GenBank/E.coli.genbank

yyin@ser:~$ ls -l /home/yyin/Unix_and_Perl_course/Data/GenBank/E.coli.genbank-rw-r--r-- 1 yyin yanbin 10602328 Oct 9 2008 /home/yyin/Unix_and_Perl_course/Data/GenBank/E.coli.genbank

Atyourhome, try

head /home/yyin/Unix_and_Perl_course/Data/GenBank/E.coli.genbank

Similarcommands:

morelesstailcat

UnlikeinWindows,textfilesofanysizecanbeviewed

Onlytextfilescanbeviewed

Therearealsobinaryfiles,zipped filesandtarredfilesthatcannotbeviewed.Forexample,- Executables (e.g.,blast,samtools,bwa,bowtie)- Datainbinaryformat(e.g,BAMfiles,indexfilesforBWAorBowtie,formattedBLASTdatabases)- Compressedfiles(usually*.gz,*.zip,*.bz2,…,butextensionsnotnecessary)

34

Howtocopyandmovefiles/foldersAtyouhome, try

cp /home/yyin/Unix_and_Perl_course/Data/GenBank/E.coli.genbank .

Rememberwhatdot(.)means?

Nowtry,

lsmv E.coli.genbank bioinfolsmv bioinfo/E.coli.genbank .ls

Whatabout?

mv E.coli.genbank bioinf

Whatifyouwanttocopyafolder?

cp –r bioinfo bioinfo2

Youcreatedthisfolderinyourhomemomentsago

Difference?mv file foldermv file file

35

Howtodeletefiles/folders?

rm file remove filerm folder remove empty folderrm –rf folder remove not empty folder

Forexample,

rm –rf bioinfo2

WARNING:deletedthingscannotberecoveredinLinux!!!

Tobeaskedbeforedeletion,

yyin@ser:~$ rm -i bioinfrm: remove regular file `bioinf'? y

Homework#6Createafolderunderyourhomecalledhw6;pwd tofindoutwhereyouare

Changedirectorytohw6;pwd tofindoutwhereyouare

CopytheentirefolderUnix_and_Perl_course frommyhometoyourworkingfolder(thefolderwhereyoucurrentlyare). Googleormanorcheckmyslidesifyoudonotknowhowtocopyafolder.

Changedirectorybacktoyourhome

Usels toexplorethefolderUnix_and_Perl_course andthesub-foldersintheretolocatetheAt_genes.gff fileandtheunix_and_perl_v3.1.1.pdffile.Tellmewhatisthesizeandcreationdateofthesefiles.

ChangethepermissionoftheAt_genes.gff filesothateveryusercaneditthisfile.

36

Writeareport(inwordorppt)toincludealltheoperations/commands andscreenshots.

Dueon11/03(sendbyemail)Officehour:Tue,ThuandFri2-4pm,MO325AOremail:yyin@niu.edu

Nextclass:commandsforcontrollingfiles

37