Introduction to Unix – CS 21
Lecture 13
Lecture Overview Finding files and programs
which whereis find xargs
Putting it all together for some complex programs
What Do You Do When You Want To Locate A File? Option 1: ls
Can be very time consuming Option 2: Other programs
which Used to see what you are running
whereis Used to see where programs are located
find A general tool to find all sorts of files in the
directory
which Usage: which PROGRAMNAME Tells you what executable is being
run when you run a program Useful to tell when more than one
executable exists (i.e. one in /bin and one in /sbin)
Only searches your PATH
Example Of which
whereis Usage: whereis PROGRAMNAME Locates the source, executable,
and man page for a command Searches set paths
Example Of whereis
find Locate files based on any number
of factors More than just the name
When it was modified What permissions it has What size it is
Running find Usage: find [path] [expression] A little different from all other
programs you’ve run Instead of a letter, options are the
complete word Example:
Instead of –n, you use -name
Expression Options To find -name “pattern”
Matches the name of a file -iname
Matches the name of a file but is case-insensitive -empty
Matches empty files -type
d f l
Examples
More Expression Options -atime N
File was last accessed N days ago -mtime N
File was last changed N days ago +N, -N
+N = More than N days ago -N = Less than N days ago
Examples
Combining Expression Options -o
Or -a
And \( \)
Grouping
Action Commands -print
Simply prints out the name of the file that was found
Most common action -exec
Executes a command -ok
Executes a command, but prompts the user first
The exec Option – A Powerful But Dangerous Option -exec COMMAND \; {} gets replaced with the name of
the file that was found Example:
find . –name “*.txt” –exec echo {} \; Same as find . –name “*.txt” -print
Examples
When Would You Ever Do This? The –exec option allows you to
perform some command on all files that match a certain criteria
Example: Someone cracked into your system and
you want to check all recently modified files for any changes against a backup
find –mtime –2 –exec diff {} .snapshot/{}
Problem The exec command will run the
command once for every file that matches A lot of overhead May take a lot longer than you’d like
Solution: xargs
The xargs Command Take values from standard in and
convert them into command line parameters
Usage: xargs COMMAND Example:
xargs grep pattern
What?
File1.txtFile2.txt
…
Program
stdin xargs Program File1.txt File2.txt …
Why? When a program outputs a list that
will be used as input to another program
Used mainly in pipes with find Example:
find . –name *.txt | xargs grep ‘Jason’ find . –name *.txt –exec grep ‘Jason’ {} \; Calls grep many, many times less than
the second call and takes less time
Another xargs example Problem: You want to kill all of your
processes First, list all your processes
ps aux | grep login Next, select all the PIDs
cut –d’ ‘ –f2 Now pass all those PIDs to kill ps aux | grep login | cut –d’ ‘ –f2 | xargs kill -9
A Simple Cleanup Script
#!/bin/bash ps aux | grep login | cut –d’ ‘ –f2 | xargs
kill -9exit 0
>./exitScript.sh &
When Would You Use Bash Versus Awk? What is bash good at?
Finding and manipulating files and directories What is bash bad at?
Manipulating data inside files What is awk good at?
Manipulating data inside files What is awk bad at?
Finding and manipulating files and directories
Building A Complex Program Start small
Always try to get something working before you get something working well
Build up Add functionality one part at a time
and test as you go
Building A Complex Program In Bash Starting small – #!/bin/bash exit 0 Slowly add functionality #!/bin/bash read input echo $input exit 0
Building Up A Complex Program – In awk Start small { print $0 } Slowly add functionality/Pattern/ { print $0 }
Hints Break the problem down into smaller
components Try printing first – always print as you go Insert temporary files or variables if needed Example:
If you are required to process any number of command line parameters one at a time, start by handling just one
After you can handle one, try two Generalize after you have gotten the first part
down
Example
Computing Problem Find all files we haven’t changed in
2 days and move them to a folder called oldStuff
Should we use bash or awk? Bash shell programming would be
better
Break It Down Find all files we haven’t changed in
two days find ~ –mtime +2 -print
Move each file to the oldStuff folder mv file ~/oldStuff/
Put It Together We need some way to connect the
first part with the second part How about a variable?
Store the list of file names and then go through them one by one
var=`command`
Script Solution
#!/bin/bashfilesToMove = `find ~ -mtime +2 –
print`for f in $filesToMovedo mv $f ~/oldStuff/doneexit 0
Another Computing Problem Remove all files that have the
phrase “Remove Me” located on the first line
Which should we use, bash or awk? We are looking inside files, but we
aren’t manipulating the data, so we should use bash.
Break It Down Find all files
find . –name “*” -print Get the first line
head -1 Look for “Remove Me”
grep “Remove Me” Remove the file if “Remove Me” was
there rm file
Put It Together Variables to pass information along
List of all files Temporary files or variables to
store results and test Results from grep
One Script Solution #!/bin/bash allFiles=`find . –name “*” –print` for f in $allFiles do head –1 $f | grep “Remove Me” > .tmpFile if [ -s .tmpFile ] ; then rm –i $f fi done rm –f .tmpFile exit 0
Another Computing Problem Tax calculator
Given a receipt, calculate the tax Receipt is a file containing the name
of the product and price of the product on one line
Pork Tenderloin:10.85
Should we use bash or awk? awk would be better in this case
Break It Down Calculate the tax on an individual
item Get the item
$2 Multiply it by the tax
$2 * .0775
Sum up all the taxes
Put It Together To keep a sum, we need to store
the running total in some location How about a variable?
totalTax = totalTax + newTax We need to initialize the tax in the
beginning and print out the tax at the end
Awk ScriptNF == 2 { totalTax = totalTax + $2 * .0775}BEGIN { totalTax=0}END { print totalTax}
Problem 4 Given a file that contains
inventory, print out all items that are out of stock File format: ITEM/PRICE/QUANTITY Example: Needlenose pliers/2.50/4
Should we use bash or awk? Awk is better this time since we are
checking the data
Script Solution
{ if ($3 == 0) {
print $1 }
}
Next Time Quiz #2
Bash and awk programs sed
Perl Programming Introduction