Date post: | 28-Dec-2015 |
Category: |
Documents |
Upload: | jacob-murphy |
View: | 216 times |
Download: | 1 times |
Why is UNIX so popular?
• Platform Support– Unix runs on virtually an hardware platform
– Designed as a portable Operating System
• Toolset(s)– Unix has many powerful development tools
• Shell Programming Languages (sh,csh,ksh,tcsh)
• C language
• “filters” like awk, grep, sed, find, pr
• “pipe” process output to glue filters together
– Open Source Community
Networking
• Unix has ALWAYS supported TCP/IP
• Very powerful internet capabilities– telnet, ftp, NFS, …– most unix apps support networking– Even the smallest Unix/linux system can
support full-scale Internet apps.– Great development environment– REMOTE ACCESS via Telnet
“Free” UNIX software
• UNIX (.e. Freebsd, Linux)
• Languages (GNU Compilers, Java etc..)
• Web development environments– PhP– PERL (ModPerl)– Apache– FastCGI– NCSA web server
Other Reasons ….
• UNIX is the development environment for almost all computer science departments
• If you have a computer science degree > 1980 you know Unix…
• =====> Many, many UNIX programmers
• Supported by all major vendors– SUN. IBM, HP, SGI, DELL, Unisys etc.
UNIX OS features
• Process oriented environment• Hierarchical file system integrated with
virtual memory system (what does that mean?)
• Small kernel size allows easy “porting” to new environments (like CRUSOE)
• But full set of capabilities allow it to be used on very large systems
Recent Additions - Linux/Beowulf
• Linux growth has made it even easier to learn Unix
• Linux modifications allow it to scale to super computer size
• Beowulf Project - NASA Ames– Modify Linux so it can run on hundreds or
even thousands of cooperating systems
Beowulf
• Configuration– 1 Master computer
• Used for job scheduling, data output etc..
– Many identical Beowulf machines all connected by a separate private, high speed network switch (sometimes multiple networks)
– Can be used for almost any type of processing that can be done in parallel (computation, web serving, database etc.).
UNIX problems
• Command language(s) powerful, but cryptic and terse– Designed to minimize typing– Designed by Ph.D. computer scientists at Bell
labs who believe that everyone understands regular expressions
• Steep learning curve, not for novices...
Huh???
• "There are two major products to come out of Berkeley: LSD and UNIX. We don't believe this to be a coincidence."
• -- Unknown
• What’s the matter with this quote????
UNIX as a web platform
• UNIX quickly became dominant web platform because of its scalability and large set of tools available
• Many development choices available
Some UNIX Web environments
• Apache Web server - most popular web server– Server side toolkits
• CGI (PERL, C, PYTHON …)
• “FASTCGI”– link application directly into Apache so that it is always
running
• MODPERL - FASTCGI version of PERL
• PHP – Server side, imbedded HTML scripting language
• Java servlets, Java Server Pages etc..
Other Web Servers
• Netscape/Sun/Iplanet server(Stern)
• IBM Websphere
• NCSA
• ….
• ALL support the same toolsets
Other UNIX Tools - X
• Xwindows environment– Client server graphical programming
environment– Originally developed to allow scientists to
display results graphically even if the application (Bubble chamber) was running on another computer
– Quickly became a standard on many platforms, but especially UNIX
Xwindows User Interfaces
• Graphical input was supported in early versions of X• This allowed the development of Windows based
command environments for UNIX, as well as many applications (SAS, SPSS etc.)
• This now allowed UNIX to support remote command and windows based interfaces
• Great environment for systems programming support (especially if you have hundreds of web servers)
Conclusion
• UNIX is a powerful and robust environment
• Managers need to know it’s capabilities
• Here are some examples...
General UNIX commands
• cd Changes working directory • cp Copies a file • history Lists history of previous commands• ln Creates a link • ls Lists contents of directory • mkdir Creates a new directory • mv Moves or renames files • pwd Prints working directory name • rm Removes files and links • rmdir Removes a directory
• !! (Bang operator) Repeats previous command
UNIX File Manipulation
• apropos Locates commands by keyword lookup• awk Pattern scanning and processing language • cat Concatenates a file• cmp Compares two files • diff Displays differences between pairs of text files • find Finds files • grep, egrep, fgrep Searches a file for a pattern • head Gives first few lines of file • man Finds and displays manual information • more, page, less Filters used to display files on a screen • sed Edits a stream • tail Displays the last part of a file • troff Text formatting and typesetting
Process commands
• bg Places a stopped process in the background
• fg Places a stopped process in the foreground
• jobs Gives a list of current processes created from this
• shell
• kill Sends a signal to the specified process
• ps Gives process status
• & Puts a command in the background when appended to the command
Miscellaneous Commands
• chmod Changes file permissions
• clear Clears terminal screen
• csh Invokes a shell (command interpreter) with C-like
• syntax
• date Displays the date
• du Summarizes disk usage
• hostname Sets or displays name of the current host system
• passwd Changes login password
• script Makes transcript of a terminal sessions
• sh Invokes the Bourne shell
• umask Sets/gets default file permission mask
• which Locates a program file, including aliases and paths
But that’s not all folks…
• Pipes – UNIX supports routing the output of one
command or program (stdout or standard output) to the input of another (stdin or standard input)
– This allows the rapid construction of simple? systems using combinations of existing commands (like our homework)
– This concept was carried forward into DOS
UNIX Pipes and Redirection
• Pipe symbol is | – so proga | progb
• runs program a and program b simultaneously, piping the output (stdout) of proga in to the input (stdin) of progb
– so ps -eaf | grep nwhite will list all of my processes running on unix
– ps -eaf | grep nwhite |tee lpr -Pm9is2 • will also print a copy in the IS department printer• (what do you think “tee” does?)
Redirection
• Symbols <, >, >> allow you to “redirect” the output or input of a program (including shell procedures)
• so proga< inputfile– Starts proga up and “feeds” inputfile to it (instead of
user input)
• proga <inputfile >outputfile– uses inputfile as input and outputfile as output
• >> appends to file
Unix Shells
• Shell is the name of any Unix command that provides a command oriented interface to the user.
• Lot’s of shells– Why?
• 25% of all MS students think they want to write their own shell..
Common shells
• (sh) Bourne - the original
• (csh) C shell - modelled after “c” programming language syntax
• (tcsh) - C shell on steroids
• (ksh) - Korn shell - very powerful
• (bsh) - BASH
• You get the picture..
Common Elements
• All support the automation of shell commands, but use (slightly ) different syntaxes
• All are very powerful, since they all support pipes and filters and
• All can be used to build sophisticated systems
• Novice users will see little difference
But how do I tell Unix what shell to use?
• First line of file can be used to tell Unix what program to run to process this command
• so…
• #! /bin/sh– will run the sh command (bourne shell)
• #! /usr/local/bin/tcsh– Will run the t shell ...
Bourne Shell - Environment variables
• CDPATH The search path for the cd command• HOME The subdirectory that becomes current upon login and as a default for cd• LANG Determines the locale to use when LC_ALL does not specify one• LC_ALL Determines the locale to be used to override any previously set values• LC_COLLATE Defines the collating sequence to use when sorting• LC_CTYPE Determines the locale for the interpretation of sequence of bytes• LC_MESSAGES Determines the language in which messages should be written• LIBPATH The search path for shared libraries• LOGNAME Your login name, marked read-only by the /etc/profile script• MAIL The path name of the file used by the mail system to detect the arrival of new mail• MAILCHECK The number of seconds that the shell lets elapse before checking for new mail• MAILPATH A list of programs separated by colons where the shell will notify you of new mail• MAILMSG The mail notification message• PATH The search path for commands separated by colons• PS1 The string to be used as the primary system prompt• PS2 The value of the secondary prompt (when the shell finds a new-line character)• IFS Characters that are to be used as internal field separators• SHACCT The shell writes an accounting record in the file for each shell script executed.• SHELL The path name of the shell (should be exported by the $HOME/.profile script• TIMEOUT The number of minutes the shell remains inactive before it exits.
Setting Environment Variables
• MYVAR=blahblah
• export MYVAR {make it global}
• ….
• Later … cd $MYVAR
• or
• echo my name is $MYVAR
Quotation marks
• Using Quotation Marks• There are three different kind of quotation marks that you
can place around variables, ' (single quotes), " (double quotes), and ` (backquotes). Each performs a completely different function.
Double Quotes• Double Quotes: are the weakest of the three ways of quoting. Everything between
double quotes is still interpreted by the shell. For example, let's consider the following piece of code:
• name=yourself• foo="hello there $name" • The shell will interpret $name and $foo will have hello there yourself stored in it. If
you wanted to, you can• precede a special character with a backslash to prevent the shell from interpreting
metacharacters. So if you did • name=yourself• foo="hello there \$name" • the shell will not interpret $name and you will have hello there $name stored in $foo. • The most common usage of double quotes is to store text with whitespaces in it. For
example • foo="This is a variable with a bunch of whitespace"
Single Quotes
• Single Quotes: If you had done the same thing with single quotes, namely
• name=yourself• foo='hello there $name' • the text between the quotes will *not* be
interpreted by the shell and you'll end up with hello there $name stored in $foo. Single quotes are often used to hide special characters (called metacharacters) such as *, ?, etc. from the shell.
Back Quotes
• BackQuotes: Let's consider the following line in a Bourne shell script
• foo=`ls -l | fgrep Jul`
• The backquotes do something completely different here. The text between the backquotes (ls -l | fgrep Jul) is executed and the results of the execution are stored in the variable $foo.
• Backquotes are most often used when you want to capture the output of a command in a variable.
Great on-line course
• http://www.uwsg.iu.edu/edcert/session3/shell/
• Nice coverage of Bourne Shell
Command Variables
• Variables $1 $2 $3 … $* and $#• are set from the parameters on the command
line• so• myprog foo1 foo2 foo3• $1=foo1• $2=foo2• $3=foo3
Builtin Variables
• A number of variables are automatically set in the shell. They usually contain state information of one kind or the other:
• $# The number of arguments supplied
• $* All arguments passed to the shell on the command line
• $@ Similar to $*
• $- Options supplied to the shell
• $? The value returned by the last executed command
• $$ The PID of this shell
• $! The PID of the last command started with &
Shell Programming - case
• case string1 in
• str1) commands;;
• str2) commands;;
• *) commands;;
• esac
• string1 is compared to str1 and str2. If there is a match, the commands up to ;; are executed. If there is no match, the commands after the asterisk are executed.
For statement
• for var1 [ in list ]• do• commands• done • The for loop executes once for each item in the list.
The list can be a variable containing several words separated by spaces or it can be a list of values typed directly into the statement. Each time through the loop, the variable var1 is assigned the current item in the list until the last one is reached. [ in list ] is optional. If it is not present, the for loop is executed once for each item in the variable var1.
• Example– For I in 1 2 3 4 5; do echo $I; done
if Statement
• if [ expression ];• then• Commands• elif [ expression2 ];• then• Commands• else• Commands• fi• The expression following if is computed and, if it returns true,
the commands following the first then are executed. If expression returns false, expression2 is computed and follows the same logic. If neither expression nor expression2 return true, the commands following else are executed. Both elif and else are optional.
While statement
• while [ expression ]• do• commands• done• A while command repeatedly executes
the expression and, if the last command in the expression is true, executes the commands in the do list; otherwise the loop terminates.
Test Statement
• The test command is available in Bourne shell to test various things. The usage is • test expression • or• [ expression ] • Here is a list of tests that are allowed: • Integer operators • int1 -eq int2 True if int1 is equal to int2 • int1 -ge int2 True if int1 is greater than or equal to int2 • int1 -gt int2 True if int1 is greater than int2 • int1 -le int2 True if int1 is less than or equal to int2 • int1 -lt int2 True if int1 is less than int2 • int1 -ne int2 True if int1 is not equal to int2 •
Test - String/ File Operators
• String operators • str1 = str2 True if str1 is identical to str2 • str1 != str2 True if str1 is not identical to str2 • str True if str is not null • -n str True if the length of str is greater than zero • -z str True if the length of str is equal to zero • File operators • -d filename True if directory filename exists • -f filename True if file filename exists • -r filename True if filename is readable by the process • -s filename True if filename has a nonzero size • -w filename True if filename is a is writeable by the process • -x filename True if filename is executable •
Test - logical operators
• Logical operators
• ! expr True if expr is not true
• expr1 -a expr2 True if both expr1 and expr2 are true
• expr1 -o expr2 True if either expr1 or expr2 is true
• Example:
• if [ -f /bin/sh ]; then
• echo ”yes, we have the Bourne shell"
• fi
Filename matching
• * match any string, including the null string
• ? match any single character
• [ccc] match any of the characters in ccc
• [a-e1-5] equivalent to [abcde12345]
• "... " match ... exactly; quotes protect metacharacters
• \c match c literally
• / matched only by an explicit / in the expression; in case, matched like any other character
• . as the first character of a filename, is matched only by an explicit . in the expression
• a|b (in case expressions only) match either a or b
Putting it all together
• #!/usr/bin/sh
• #
• # Sample Bourne Shell script to list a directory on the
• # Stern Web Server
• #
• # Norman White, November 2000
• #
• # Only works for Stern grads
• # Only lists the contents of their websys subdirectory
• # Assumes the input is on the command line
• #
• # in the form
• http://www.stern.nyu.edu/~nwhite/lsgrad.cgi?fchar=a&userid=aab211
• # or is passed from the web server as a "GET" action in a cgi program
• #
• # part 1 - Generate intro html to put in front of output
• echo "Content-type: text/html"
• echo
• echo "<HTML><HEAD>"
• echo "<TITLE>Websys Directory Listing</TITLE>"
• echo "</HEAD> <BODY>"
• echo "<pre>"
• # remove next few lines for working script
• echo The arguments to the script are: $*
• # Environment Variable QUERY_STRING contains form fields
• echo QUERY_STRING contains $QUERY_STRING
• # uncomment below for help in debugging
• #set
• # Tricky stuff, use sed to get rid of the & between fields
• # so you have a line like:
• # set fchar=a userid=aab211
• # this sets local shell variables for each of the
• # form field values
• # so they can be used to generate the directory path
• # See http://www.stern.nyu.edu/~nwhite/websys/lsgrad.html for an example
• form to
• # invoke this script
• # note, this script MUST be named .cgi in order to be run
• # on the Stern web server
• # also, you need to do a chmod o+rx on it so it is executable
• # The "eval" line creates a shell variable for each form field
• # by issuing a
• # field1=value1 field2=value2 etc
• # The values of these variables can now be used later in the script
• # by putting a $ in front of them
• # "eval line echos the input line and pipes it to a sed program that changes
• # & to blank, hence fchar=a&userid=aab211 becomes
• # fchar=a userid=aab211
• eval ` ( echo $QUERY_STRING | sed -e "s/\&/ /g") `
• # Note need the { } around userid because of the trailing :
• echo Doing directory listing for user ${userid}:\<p\>
• ls -m /export/grad/$fchar/$userid/public_html/websys/
• echo "</pre>"
• echo "</BODY></HTML>"
So how do we do optional parts?
• Clickable list – Hint how do you get a clickable list of files…
• Grad vs undergrad - how can you tell?