Date post: | 22-Dec-2015 |
Category: |
Documents |
Upload: | delphia-dixon |
View: | 216 times |
Download: | 0 times |
Intermediate UnixIntermediate Unix
Presented July 29th, 2001 by:
“Robin” R. Battey ([email protected])
Evgeny Roubinchtein ([email protected])
with tips, suggestions, and corrections by:
Hannah Tang ([email protected])
http://www.cs.washington.edu/people/acm/tutorials/
The “File System”The “File System”
• Under UNIX, (almost) everything is a “file”:– Normal files– Directories– Hardware– Sockets– Pipes
• Things that are not files:– Users– Groups– Processes
OwnershipOwnership• Files have two owners
– Every file has exactly one user owner– Every file has exactly one group owner
• Everyone is a user– Users are in at least one group
• Processes have owners, too (known as an “id”)– Every process has exactly one user id– Every process has at least on group id
• Users and groups are really just numbers with names– Every username is mapped to a single numeric “uid”– Every groupname is mapped to a single numeric “gid”
Who am I?Who am I?
• Commands that tell you who you are:– whoami displays your username– id displays your username and groups
• Commands that tell you who others are:– finger [<name>] displays info for <name>– id [<username>] displays info for <username>
• Commands that change who you are:– su <username> “switch user” to <username> – login login as a different user
File PermissionsFile Permissions
• Every file has three access levels:– user (the user owner of the file)
– group (the group owner of the file)
– other (everyone else)
• At each level, there are three access types:– read (looking at the contents)
– write (altering the contents)
– execute (executing the contents)
Strange ThingsStrange Things
• There are three “strange” permissions:– setuid (run program as user owner)– setgid (run program as group owner)– text (stay in swap after executing)
• Directories act differently– “write” (creating/deleting files)– “execute” (cd-ing to that directory)– “setuid” (ignored)– “setgid” (created files have same group owner)– “text” (deletion restricted to user owned files)
Examining PermissionsExamining Permissions
• A “long” ls listing shows file permissions:
[zanfur@odin zanfur]$ iduid=8774(zanfur) gid=100(users) groups=100(users),101(mp3)[zanfur@odin zanfur]$ ls -ltotal 524-rwxr-xr-x 1 zanfur users 512668 Jul 31 00:18 bashprw-r--r-- 1 zanfur users 0 Jul 31 00:24 fifo-rw-r--r-- 1 zanfur users 0 Jul 31 00:18 filedrwxr-xr-x 2 zanfur users 4096 Jul 31 00:16 normaldrwxrws--T 2 zanfur mp3 4096 Jul 31 00:14 shareddrwxrwxrwt 2 zanfur users 4096 Jul 31 00:14 tmpdrwxrwxr-x 2 zanfur www 4096 Jul 31 00:15 www[zanfur@odin zanfur]$ _
Permissions Listing BreakdownPermissions Listing Breakdown
Field Description Valid Values
?--------- file type - (normal file), d (directory), p (fifo), b, c, s
-?-------- user read - (no read permissions), r (read permissions)
--?------- user write - (no write permissions), w (write permissions)
---?------ user execute - (no exec), x (exec), S (setuid), s (both)
----?----- group read - (no read permissions), r (read permissions)
-----?---- group write - (no write permissions), w (write permissions)
------?--- group execute - (no exec), x (exec), S (setgid), s (both)
-------?-- other read - (no read permissions), r (read permissions)
--------?- other write - (no write permissions), w (write permissions)
---------? other execute - (no exec), x (exec), T (text), t (both)
What You Can Do With PermissionsWhat You Can Do With Permissions
Permission File Directory
r (read) Read a file List files in …
w (write) Write a file Create a file in …
Rename a file in …
Delete a file in …
x (execute) Execute a file (eg shell script)
Read a file in …
Write to a file in …
Execute a file/shell script in …
Changing OwnershipChanging Ownership
• Changing user ownership:– The command is “change owner”:
chown <username> <filename>– but, you can only do it if you are root– so just copy it instead
• Changing group ownership:– The command is “change group”:
chgrp <groupname> <filename>– but, you can only do it if you are in group <groupname>
– and you must be the user owner of the file
Changing PermissionsChanging Permissions• The “change mode” command:
chmod <level><op><permissions>[,…] <filename>
<level> string of: u, g, o, a (user, group, other, all)<op> one of +, -, = (gets, loses, equals)<permissions> string of: r, w, x, s, t, u, g, o
(read, write, execute, set-id, text,same as user, same as group, same as
other),
• Examples:chmod u+rwx,go-w foobarchmod g=u,+t temp/chmod u=rwx,g=rwxs,o= shared/
Process ManagementProcess Management
• What can you do with it?– Start programs in the background– Run more than one program per terminal– Kill bad and/or crashing programs– Suspend programs mid-execution– List all jobs running in a shell– Move foreground jobs to the background– More …
Three States of a ProcessThree States of a Process• Foreground
– Attached to keyboard– Outputs to the screen– Shell waits until the process ends
• Background, running– Not attached to keyboard– Might output to the screen– Shell immediately gives you another prompt
• Background, suspended– Paused mid-execution– Can be resumed in background or foreground
Background ProcessesBackground Processes• Listing jobs:
– jobs lists background “jobs” and job #’s– ps lists processes and their process id (“pid”)– %<job#> expands to the process id of the job
• Stopping foreground jobs– Press ^Z (Ctrl-Z) in the terminal window
• Starting a process in the background– Append a & character to the command line– Examples: ls –lR > ls-lR.out &
xemacs my_program.cc &
• Resuming a stopped job– In the foreground: fg [<pid>]– In the background: bg [<pid>]
Killing ProcessesKilling Processes• The “kill” command:
kill [-<signal>] <pid>Send <signal> to process <pid>
• The “killall” command:killall [-<signal>] <command>Send <signal> to all processes that start with <command>
• Useful signals (kill –l for the complete list):TERM the default, “terminate”, kills things nicelyKILL will kill anything, but not nicelyHUP “hangup”, used to reload configurationsSTOP stops (suspends) a running process
What are environment variables?What are environment variables?
• The environment is an array of strings of the formNAME=VALUE
• Each process (program) has its own copy of the environment
• Processes started by the shell get a (“deep”) copy of the shell’s current environment
• Each process can examine and modify its own (and only its own) environment
• The “meaning” of environment variables is merely a matter of established conventions
Viewing your shell’s environmentViewing your shell’s environment
• To view a single environment variable’s value:echo $<name>
• For example:echo $HOME
will display/homes/iws/evgenyr
• To view all environment variables at once:– tcsh/csh/bash: printenv– sh/ksh: set
Setting environment variablesSetting environment variables
• tcsh/csh:– setenv <name> <value>– Example: setenv PRINTER ps329
• bash/ksh:– export <name>=<value>– There is no space on either side of ‘=’!
– Example: export PRINTER=ps329
• sh:– <name>=<value>; export <name>
!
Appending to environment variablesAppending to environment variables
• Appending /uns/bin/ to your PATH– bash/ksh: export PATH=$PATH:/uns/bin
– tcsh/csh: setenv PATH=$PATH:/uns/bin
• Prepending is similar:– bash/ksh: export INFOPATH=/uns/info:$INFOPATH
– tcsh/csh: setenv INFOPATH /uns/info:$INFOPATH
Common environment variablesCommon environment variables
• The “meaning” of environment variables is purely a matter of convention.
• However, these environment variables are quite common:– PATH, INFOPATH, MANPATH, EDITOR, VISUAL, PAGER, HOME, MAIL, USER
• The man page for a program will usually list the environment variables the program pays attention to.
Useful shell features Useful shell features (a preview of upcoming attractions)(a preview of upcoming attractions)
• Aliases
• Redirecting input and output
• Command substitution
• Scripts
Shell as a work-saver: aliasesShell as a work-saver: aliases• Aliases: textual substitution, similar to
C-preprocessor macros.• Syntax:
– bash/ksh: alias <text>=’<replacement>’
– tcsh/cshalias <text> ’<replacement>’
• When you type <text>, the shell “substitutes” <replacement>
• Typing alias by itself lists all your current aliases• unalias <text> removes the alias• Check the csh/tcsh man page for extra features
AliasAlias examples examples
• Always print postscript files double-sided– tcsh/csh:alias lpr ’lpr –Zduplex’
– bash/ksh: alias lpr=’lpr –Zduplex’
• Collect a few tree-friendly options to enscript:– tcsh/csh:alias print \ ’enscript –2rhB –SDuplex=DuplexNoTumble’
Line continuation character, just like in C/C++!
What is input/output redirection?What is input/output redirection?
• Normally, a program’s standard output is displayed on user’s terminal, and its standard input comes from the keyboard.
• Redirecting the output of a program means asking the shell to put the program’s output (stdout [C++’s cout]) into a file.
• Redirecting the input of a program means asking the shell to feed a file as the program’s standard input (stdin [C++’s cin]).
• Note: redirection works with files.
Why redirect program’s output?Why redirect program’s output?
• You may want to save output and examine it at your leisure:– if the program generates a lot of output
– if the program takes a long time to run
• You may want to present the output to another person
• Having the program write to standard output may make the program simpler to write
Standard output vs Standard errorStandard output vs Standard error
• By convention, “normal” output of a program is sent to standard output (stdout [C++’s cout]), while debugging or error output is sent to standard error (stderr [C++’s cerr]).
How to redirect program’s output?How to redirect program’s output?• To redirect just the standard output:<program> > <FILE>
• To redirect just the standard error:sh/ksh/bash: <program> 2> <FILE>csh/tcsh: ( <program> > STDOUT ) >& STDERR
• To redirect both standard output and standard error:
csh/tcsh/bash: <program> >& <FILE>sh/ksh/bash: <program> > <FILE> 2>&1
> vs. >>> vs. >>
• Both > and >> will create the output file, if it doesn’t already exist
• If the file does exist, then:– Using > to redirect output will overwrite the output
file:•ls > newlisting•printenv > my_environment
– Using >> to redirect output will append to the output file•cat ch1 ch2 ch3 > book•cat ch4 ch5 ch6 >> book
Why redirect program’s input?Why redirect program’s input?
• To run the program repeatedly with the same (or similar input)
• Having the program read from standard input may make the program simpler to write.
How to redirect program’s input?How to redirect program’s input?
• Simple!<program> < <FILE>
• Examplesort < my_grades.txthead < really_long_book.txt
PipingPiping
• Piping is connecting programs together by using the output of one program as the input to the next.
• Syntax:<program1> | <program2> | … | <programN>
• A simple example (view a sorted file-listing a page at a time):ls | sort | less
• Note: piping deals with the input/output of programs (that is, stdin, stdout, stderr)
Why piping?Why piping?
• Because “the whole is bigger than the sum of its parts.
• By combining Unix utilities in a pipeline, you can build tools “on-the-fly” as you need them.
Piping examplesPiping examples
• How many .c files are in this directory?ls *.c | wc –l
• What files were modified most recently?ls –t | head
• What processes am I running?ps auxw | grep <mylogin>
• Make an alias for the above, for other Linux boxen too:alias myps=’ ps auxw | grep `id –un`’
TheThe catcat utilityutility
• Especially useful: cat– When used with pipes, “copies” stdin to stdout
•cat can be used to redirect out of a pipe!
• Example: make a file containing currently running processes ps aux | grep evgenyr | cat > my_processes
– When used with a file, “copies” the file into stdout•cat can be used to place a file’s contents into a pipe!
• Example: list all the items in a list in alphabetical ordercat my_grocery_list.txt | sort | uniq
Command substitutionCommand substitution(aka “what’s up with the backquotes?”)(aka “what’s up with the backquotes?”)
• Command substitution means that the shell substitutes a command’s output for the command itself.
• To get the shell to perform command substitution, enclose the command in backquotes (`)
Shell as a work-saver: scriptsShell as a work-saver: scripts• Instead of typing the same series of commands over and
over again, put them in a file and have the shell execute the (commands in the) file!
• The file must have execute permission• The first line of the file should be:#! <YOURSHELL> (e.g., /bin/bash)
• There must be no space (or any other character) between ‘#’ and ‘!’.
• Whether to put space after ‘!’ is a matter of style• Shell also has control flow, tests, etc. The shell tutorial
on the ACM web page goes into much more detail.
Intermezzo: quotingIntermezzo: quoting
• Problem: some characters are special to the shell (*, ?), but you would like to pass those characters to the programs the shell runs as-is.
• So you quote the character(s):– A single character: by prefixing it with a \ (backslash)
– A string of characters: by enclosing them in• Single quotes: no characters are special. None
• Double quotes: some characters (notably $ and `) are still special. You’ll need to prefix those with a backslash to pass to the program as-is.
Quoting examplesQuoting examples• List all the files in the current directory:ls *
• List the file or directory called ‘*’ (or complain if it’s not there):ls \*
• Print $HOME:echo ’$HOME’
• Print the path to your home directory:echo ”$HOME”
• Print `id –un`: echo ’`id –un`’
• Print your login:echo ”`id –un`”
Next stop: utilitiesNext stop: utilities
• (No, not electricity, water, and sewer)• diff and patch• grep• find and xargs
diffdiff and and patchpatch
• You have two versions of a file; how do you see what’s changed between the two versions?– diff shows the differences between two files; you’ll
want to use the –u or –c option to diff to produce output in human-friendly format:diff –u my_file my_file.orig | less
– patch applies differences to a filediff –u my_file my_file.orig > my_patchpatch my_file < mypatch
my_file is now the same as my_file.orig
grep grep – search for patterns– search for patterns
• grep searches for patterns in texts:grep <string> <FILE>
• grep uses regular expressions to describe the string(s) to search for
• In a regular expression, most characters are treated as-is, but a few are magic– Ordinary characters just represent themselves– Magic characters do special things
grepgrep’s magic characters’s magic characters
• A few different kinds of magic characters:– Some characters “anchor” (parts of) a regular
expressions to specific places in the string:^ - The beginning of a line$ - The end of a line
– A dot (.) matches “any one character whasoever” (except for newline)
– [ ] form a character class:•[aeiou] – any single vowel•[a-zA-z0-9] – Any single letter or digit•[^0-9] – Any single character that isn’t a digit
Here, ‘^’means “not”
Even more of Even more of grepgrep’s magic characters’s magic characters
• Quantifiers say how many times the preceeding “thing” should be repeated:– * means “zero or more times”– ? Means “zero or one time”
Frequently used Frequently used grepgrep options options
-i : do case-insensitive search
-n : print line numbers
-v : print lines that do not match
-l : only list the files from which output would have been printed
grep grep examplesexamples
• Print all non-blank lines:grep –v ’^$’ my_file.txt
• Print all the lines on which my_function is called (or declared):grep –n ’my_function *(’ my_code.c
• Show all the xterms I am running:ps auxw | grep ”`id –un` .*xterm”
findfind
• Traverse the tree(s) rooted at <PATHs> , and for each “thing” found, evaluate the <EXPRESSION> until the result of <EXPRESSION> is known.
• The evaluation of <EXPRESSION> “short-circuits”, just like expressions in C/C++.– false && some_expression can never be true
• Options apply to the entire find command, not the individual expression
find <PATHS> <OPTIONS> <EXPRESSION>
Components of a Components of a findfind expression expression
• An expression is zero or more primaries connected by operators
• Two kinds of primaries:– Tests: just return true or false– Actions: do something, in addition to returning true or false– Operators work just as they do in C/C++.
•! / -not•-a / -and•-o / -or•, (comma)
– Parentheses are used for grouping, just as in C/C++.
findfind examples examples• Print all the *.c* and *.h* files in and below the current directory
find . –name ’*.[ch]*’ –a –print
• Show line numbers myfunction calls in the above *.c* filesfind . –name ’*.c*’ | xargs grep –n ’myfunction.*(’
• Change permissions on files under your home directory so they’re inaccessible to everyone but you (except for files in the www directory)cd; find . –path ”./www*” –prune –o –exec chmod go-rwx {} \;
• How big are my *.c* files?expr `find –name ’*.c*’ –printf ”%k + ”` 0
• Read the find manual (info find) for more examples
xargsxargs: combine arguments: combine arguments
• Feeds standard input as arguments to <COMMAND>• Often used with find
(find … | xargs grep)• But it doesn’t have to be used with find:cat filelist | xargs cat {} > concatenated
xargs <OPTION> <COMMAND> <INITIAL_ARGS>
Finding more information – at CSEFinding more information – at CSE
• Use info and man• Peruse the “see also” section of the man pages
• Peruse info’s i (search index) command
• The uw-cs.lab-help group
• Look at your .login, .cshrc, etc. to see how support has set things up
Finding more info – on the webFinding more info – on the web• http://www.faqs.org (comp.unix questions FAQ)• http://www.google.com• http://www.deja.com• The support web page:
http://www.cs.washington.edu/lab/ about/ugradcomputing.html
• (Shameless plug) the ACM tutorials page:http://www.cs.washington.edu/people/acm/tutorials/