Unix tutorial - advanced - uibk.ac.attf/lehre/ss07/bs/vorlesungen/Unix tutorial - advanced.pdf · 1...

1

Advanced UNIX Tutorial

Shelling, Otheus

For University of Innsbruck, March 2007

• NOTE: NOT THE FINAL VERSION ! !

Topics Covered• Why Unix?• Brief History• Architecture• Inter-process Communication

• Filesystem• Regular Expressions• Program Execution

• PATH to enlightenment• Dynamic Linking

• The Shells• BASH Programming• Shell Startup• Short-cuts

• Cool Commands

root:/ $

Why UNIX? And Why Not?

• High Performance• Fine-grain control over hardware

• Fine-grain control over OS

• WYSIWYG .. Not WYSIAYG

• Windows has “caught up” in performance

• Windows is easier to administrate

• Windows systems don’t require fine-grain control

2

Flavors, Versions, DistributionsFlavors

UNIX comes in various Flavors• System V, BSD, Linux

DistributionsA particular Flavor will be packaged and marketed through various Distributions

• FreeBSD, RedHat, GentooVersions

A particular Distribution will be released with a specific Version number

• RedHat EL4 US, Solaris 10 (2.10) Other version designations

The kernel and various software tools and applications each have their own version numbering

• X11R6, Linux 2.6, Perl 5.6

UNIX Architecture – System LevelDevices -

managed by drivers

Drivers -interact through an API with the Kernel

Kernel -manages core resources• Processes• Memory (physical and virtual)

• Interprocesscommunication (IPC)

presents to users: • system calls• device files

UNIX Architecture – User LevelUser:

interacts with an application through a device

Application:commands

ls, qsub,run-time environment

bash, perl, python, Java, PHP, mozilla

subsystemX Windows, SGE

Shared libraries:glibc, gtk

Each application is in its own process and has its own private data. The code can be shared across instances.Each process runs under the permissions of the user that started it – “effective user id”The application and its libraries make system calls and operates on files.

3

Inter-Process Communication

Inter-Process communication (IPC)• Semaphores• Shared Memory• Filesystem• Pipes• Signals

*otheus* ~/eg$ ls -1

example

example2

exists

myexample

ozzie

*otheus* ~/eg$ ls -1 >files

*otheus* ~/eg$ sort -k 1.2,1.4 files

files

example

example2

exists

myexample

ozzie

*otheus* ~/eg$

Inter-Process communication (IPC)• Semaphores• Shared Memory• Filesystem• Pipes

– Unnamed– Named / FIFOs– Network Sockets

• RPC• Signals

*otheus* ~/eg$ ls -1 exampleexample2

existsfilesmyexample

ozzie*otheus* ~/eg$ ls -1 | sort -k 1.2,1.4 filesexampleexample2

existsmyexampleozzie

*otheus* ~/eg$

4



• RPC• Signals

*otheus* ~/eg$ mkfifo newfifo *otheus* ~/eg$ *otheus* ~/eg$ ls -l newfifoprw-r--r-- 1 otheus dps 0 Mar 28 19:52 newfifo

*otheus* ~/eg$ { while true; > do echo $RANDOM >>/home/otheus/eg/newfifo ; > sleep 5; done } &[1] 15533

*otheus* ~/eg$ ls -l newfifo prw-r--r-- 1 otheus dps 0 Mar 28 19:52 newfifo

*otheus* ~/eg$ date ; cat newfifo ; dateWed Mar 28 19:52:49 CEST 200732102Wed Mar 28 19:52:49 CEST 2007

*otheus* ~/eg$ date ; cat newfifo ; dateWed Mar 28 19:52:51 CEST 200726331Wed Mar 28 19:52:54 CEST 2007

*otheus* ~/eg$



2-way pipes over a network

• RPC• Signals

*otheus* ~/eg$ sudo netstat -anp |> grep -E "(181):(3306|80|22) " | > head -3

tcp 0 0 138.232.66.181:22 0.0.0.0:* LISTEN 4176/sshd

tcp 0 48 138.232.66.181:22 192.168.64.52:2131 ESTABLISHED 9343/5

tcp 0 0 138.232.66.181:3306 138.232.66.164:49325 TIME_WAIT -

Inter-Process communication (IPC)• Semaphores• Shared Memory• Filesystem• Pipes • RPC

– RPC process– Portmapperprocess

– YP/NIS & NFS

• Signals

*otheus* ~/eg$ ps -e -o uid,pid,cmd |> grep rpc

29 3338 rpc.statd0 3599 rpc.idmapd0 4093 rpc.yppasswdd -e chsh -e chfn0 4274 rpc.rquotad0 4312 [rpciod]0 4323 rpc.mountd

201 17112 grep rpc

*otheus* ~/eg$ rpcinfo -p | uniq -f 4

program vers proto port100000 2 tcp 111 portmapper100024 1 udp 32768 status100007 2 udp 856 ypbind100004 2 udp 845 ypserv100009 1 udp 878 yppasswdd100011 1 udp 640 rquotad100003 2 udp 2049 nfs100021 1 udp 32774 nlockmgr100005 1 udp 686 mountd

5

Inter-Process communication (IPC)• Semaphores• Shared Memory

• Filesystem• Pipes • RPC• Signals

– Only within a process “family”

– Processes “0”, “-1”

– SIGHUP– SIGTERM– SIGKILL– SIGSEGV

*otheus* ~/eg$ kill -l | head -81) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL5) SIGTRAP 6) SIGABRT 7) SIGBUS 8) SIGFPE9) SIGKILL 10) SIGUSR1 11) SIGSEGV 12) SIGUSR213) SIGPIPE 14) SIGALRM 15) SIGTERM 17) SIGCHLD18) SIGCONT 19) SIGSTOP 20) SIGTSTP 21) SIGTTIN22) SIGTTOU 23) SIGURG 24) SIGXCPU 25) SIGXFSZ26) SIGVTALRM 27) SIGPROF 28) SIGWINCH 29) SIGIO30) SIGPWR 31) SIGSYS 34) SIGRTMIN 35) SIGRTMI

*otheus* ~/eg$ echo $$ 2432*otheus* ~/eg$ /bin/ash

*\u* \w$ echo $$18201*\u* \w$ kill -HUP 18201Hangup

*otheus* ~/eg$ echo $$ 2432

*otheus* ~/eg$ trapped &[1] 18664*otheus* ~/eg$ kill -1 18664 ; sleep 1Caught signal!*otheus* ~/eg$ kill 18664 ; sleep 1 [1]+ Terminated trapped*otheus* ~/eg$

UNIX Filesystem

UNIX Filesystem – Tree Structure• Resembles a tree

– File-types– Mounting drives– Breaking structure

• Promiscuous• Permissions

6


– File-types– Mounting drives– Breaking structure



– File-types– Mounting drives– Breaking structure• Soft-links• Loopback mounts• “Bind” mounts





7




UNIX Filesystem – Very Promiscuous• Resembles a tree• Promiscuous:

Nearly everything in UNIX can be accessed through the filesystemDevicesProcesses (Linux & Solaris) via /proc filesystemKernel stats and configuration (Linux: /proc)

Exception: Network interfaces

• Permissions

UNIX Filesystem – Very Promiscuous• Resembles a tree• Promiscuous

– Devices– Processes– Kernel stats and configuration

• Permissions

MAKEDEV loop5 ram14 skip tty2XOR loop6 ram15 stderr tty3arpd loop7 ram2 stdin tty4cdrom lp0 ram3 stdout tty5cdrom1 lp1 ram4 systty tty6console lp2 ram5 tap0 tty7core lp3 ram6 tap1 tty8cpu mapper ram7 tap10 tty9cpu0 mcelog ram8 tap11 urandomcpu1 md0 ram9 tap12 usersockcpu2 mem ramdisk tap13 vcscpu3 mice random tap14 vcs1dnrtmsg mosal rawctl tap15 vcs2dvd net root tap2 vcs3fd nflog route tap3 vcs4floppy null route6 tap4 vcs5full parport0 rtc tap5 vcs6fwmonitor parport1 scd0 tap6 vcsahda parport2 sda tap7 vcsa1hw_random parport3 sdb tap8 vcsa2initctl port sdc tap9 vcsa3input ppp sdd tcpdiag vcsa4ip6_fw ptmx sde ts_ua0 vcsa5js0 pts sde1 ts_ua1 vcsa6kmsg ram sde2 ts_ua2 vipkllog ram0 sde3 ts_ua3 xfrmloop0 ram1 sdf ts_ua4 zeroloop1 ram10 sdf1 ts_ua5loop2 ram11 sdf2 ts_ua6loop3 ram12 sdf3 tty0loop4 ram13 shm tty1

8


– Devices– Processes– Kernel stats and configuration

• Permissions

*otheus* /proc$ ls -Cd [1-2]*

1 14642 17888 21007 21942 258

10 14644 17889 21008 21943 2619

10377 14657 17890 21009 2251 2620

10395 15 17892 21010 2269 2647

11 155 17894 21017 2270 2648

11639 17473 19570 21019 2312 2649

12 17476 2 21020 2313 2650

13 17478 20998 21021 240 2658

13956 17511 20999 21022 241 27265

13964 17871 21000 2121 2431 286

14 17884 21002 21646 2432

1433 17887 21003 21941 257

*otheus* /proc$ cat 1/cmdline; echo

init [4]

*otheus* /proc$ ls -F 1

attr/ environ maps stat wchan

auxv exe@ mem statm

cmdline fd/ mounts status

cwd@ loginuid root@ task/

*otheus* /proc$


– Devices– Processes – Kernel stats and configuration

• Permissions

*otheus* /proc$ ls -CdF [a-z]*acpi/ iomem net/buddyinfo ioports partitionsbus/ irq/ pcicmdline kallsyms scsi/cpuinfo kcore self@crypto key-users slabinfodevices keys statdiskstats kmsg swapsdma loadavg sys/driver/ locks sysrq-triggerexecdomains mdstat sysvipc/fb meminfo topspin@filesystems misc tty/fs/ modules uptimeide/ mounts@ versioninfiniband/ mpt/ vmstatinterrupts mtrr*otheus* /proc$ cat uptime704398.43 593929.58*otheus* /proc$ uptime20:56:58 up 8 days, 3:39, 7 users,

load average: 1.72, 1.75, 1.67*otheus* /proc$

UNIX Filesystem – Security• Resembles a tree• Promiscuous• Permissions

– 3 Classes:• User• Group• Other (World)

– 4 bits:• Read• Write

• Execute• Special

9


– 3 Classes:• User• Group• Other (World)

– 4 bits:• Read• Write• Execute• Special



– Directories– Set-uid

– Set-gid



– Directories

– Set-uid– Set-gid

10

Regular Expressions

Regular Expressions – Quick Overview

Regular Expressions – Examples

11



Program Execution

12

Program Execution - Overview

Program Execution – Example

Program Execution – Example

13

The Shells

The Shells• Run applications and “commands”– Interactively– “Batch mode”– In parallel

• Interact with Files

• Manage environment used by applications

The Shells• Run applications and “commands”

• Interact with Files– “Send” files to commands.

– “Send” command’s output to files.

• Manage environment used by applications

14

The Shells• Run applications and “commands”

• Interact with Files

• Manage environment used by applications– Environment variables

– File creation mask

The Shells• Flavors

– Bourne (sh)• Korn (ksh)• Bourne-Again (bash)

• Almquist (ash)• “Z” shell (zsh)

– “C” Shell (csh)• tcsh

– “Exotic”• fish• zoidberg

– GUI• Konqueror

– Text-GUI• Midnight Commander (mc)

The ShellsMan’s Best Friend• help (BASH only)• man

more or lessman –kman (Solaris)man (GNU/BSD)

• info

15

The ShellsMan’s Best Friend• help (BASH only)• man• info

Hyperlink viewer– CTRL-H– --vi-keys

File: info.info, Node: Top, Next: Getting St\arted, Up: (dir)

Info: An Introduction*********************

The GNU Project distributes most of its on-lin\e manuals in the "Infoformat", which you read using an "Info reader"\. You are probably usingan Info reader to read this now.

If you are new to the Info reader and want \to learn how to use it,type the command `h' now. It brings you to a \programmed instruction--zz-Info: (info.info.gz)Top, 35 lines --Top---Basic Commands in Info Windows******************************

C-x 0 Quit this help.C-x C-c Quit Info altogether.h Invoke the Info tutorial.

Selecting other nodes:----------------------n Move to the "next" node of this no\de.p Move to the "previous" node of thi\s node.u Move "up" from this node.m Pick menu item specified by name.-----Info: *Info Help*, 763 lines --Top--------No cross references in this node.

The Shells – Using Aliases• What is it?• Creating an Alias• Automatic aliasing (bash)

hash

type / which

unalias

alias

The Shells – Using Aliases• What is it?• Creating an Alias• Automatic aliasing (bash)

hash

type / which

unalias

alias

16

Advanced Shell Programming

Advanced Shell Programming with BASH• Working Directory• Variable Creation & Expansion

• Job Control• File Manipulation• File Specification

• Negative Logic• Execution Control• Functions

exec

fg / bg

eval

export

set / echo

cd

pwd




exec

fg / bg

eval

export

set / echo

cd

pwd

17




ls

kill

jobs

exec

fg / bg

eval

export


• Job Control• File Redirection and Manipulation– Listing Files– Viewing Files

• File Specification


rm /cp / mv

grep

head / tail

more / less

cat

ls

exec


• Job Control• File Manipulation

– standard streams– Copying, moving, renaming, new dir



newgrp

id / groups

chmod / chown

mkdir / rmdir

rm /cp /mv

grep

head / tail

18


• Job Control• File Manipulation

– Changing owner– Changing permissions



umask

newgrp

id / groups

chmod / chown

mkdir / rmdir


• Job Control• File Redirection• File Specification– File globs


exec

fg / bg

xargs

find

umask

newgrp

id / groups


• Job Control• File Redirection• File Specification– find and xargs


exec

fg / bg

xargs

find

umask

newgrp

id / groups

19


• Job Control• File Redirection• File Specification


for VAR in .. do .. done

case W in .. esac

if .. fi

true / false

test

eval



• Negative Logic• Execution Control

– if/elif/else/fi

• Functions

while.. do.. done


case W in .. esac

if .. fi

true / false




– case WORD in ... esac

• Functions

while.. do.. done


case W in .. esac

if .. fi

true / false

20




– case WORD in ... esac

• Functions

while.. do.. done


case W in .. esac

if .. fi

true / false




– for VAR in ... done

• Functions

while.. do.. done


case W in .. esac

if .. fi

true / false




– while ... do ... done

• Functions

while.. do.. done


case W in .. esac

if .. fi

true / false

21




– break, next, exit– ||, &&– type, which, whence

• Functions

|| and &&

exit

next

break

while.. do.. done




– break, next, exit– ||, &&– type, which, whence

• Functions

whence

which

type




– Declaring, returning

declare

command

return

22




– Using– Overriding builtins & commands

declare

builtin

command




– Using– Overriding builtins & commands

type

builtin

command

Shell Startup

23

Shell Startup (bash)

Shell Startup (tcsh)

Cool Commands

24

Cool Commands

Cool Commands

Cool Commands

25

Complex Examples

Complex Examples

Complex Examples

26

Complex Examples

Complex Examples

Complex Examples

Date post:	30-Jan-2020
Category:	Documents
Upload:	others
View:	18 times
Download:	0 times

Unix tutorial - advanced - uibk.ac.attf/lehre/ss07/bs/vorlesungen/Unix tutorial - advanced.pdf · 1...

Documents