+ All Categories
Home > Documents > L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer...

L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer...

Date post: 01-Feb-2018
Category:
Upload: dangque
View: 228 times
Download: 0 times
Share this document with a friend
25
Notes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands and file management techniques. You may or may not end up doing the majority of your scientific computing in a UNIX environment, however it is useful to be familiar with the basic structure of UNIX and some basic commands to help navigate through files. UNIX is by far the most common operating system for the workstations that dominate today's scientific computing. There are many different versions of UNIX. In this class we will be using either: - X11 on the Macs (an Apple version of UNIX) - CygWin Xwindows on PCs (a windows version of UNIX) Some of us still work some on Sun Workstations, which use Solaris (also UNIX). There is also a free version of UNIX, called LINUX (pronounced lynn' exs), that will run on both PCs and Macs and has been getting a lot of attention lately….. I am by no means an expert on UNIX; several of you will probably learn more than me by the time you have completed your degree. I have learned only enough to get by and could benefit from learning more! So I’m just going to outline the basics here for the benefit of those students who have not been exposed much to UNIX. Information on the history of Unix can be found on Wikipedia at: http://en.wikipedia.org/wiki/Unix Quick Guide Table of Contents Opening up a Unix terminal 2 Basic Commands 5 Files & Editing 9 Wildcards 11 Help pages 11 SSH, SCP, SFTP 12 Miscellaneous commands 15 Advanced commands 17 Extra stuff 20 U U NIX NIX I I NTRODUCTION NTRODUCTION
Transcript
Page 1: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

Notes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD)

The purpose of this handout is to provide you with basic Unix commands and file management techniques. You may or may not end up doing the majority of your scientific computing in a UNIX environment, however it is useful to be familiar with the basic structure of UNIX and some basic commands to help navigate through files. UNIX is by far the most common operating system for the workstations that dominate today's scientific computing. There are many different versions of UNIX. In this class we will be using either: - X11 on the Macs (an Apple version of UNIX) - CygWin Xwindows on PCs (a windows version of UNIX) Some of us still work some on Sun Workstations, which use Solaris (also UNIX). There is also a free version of UNIX, called LINUX (pronounced lynn' exs), that will run on both PCs and Macs and has been getting a lot of attention lately….. I am by no means an expert on UNIX; several of you will probably learn more than me by the time you have completed your degree. I have learned only enough to get by and could benefit from learning more! So I’m just going to outline the basics here for the benefit of those students who have not been exposed much to UNIX. Information on the history of Unix can be found on Wikipedia at: http://en.wikipedia.org/wiki/Unix

Quick Guide Table of Contents

Opening up a Unix terminal 2 Basic Commands 5 Files & Editing 9 Wildcards 11 Help pages 11 SSH, SCP, SFTP 12 Miscellaneous commands 15 Advanced commands 17 Extra stuff 20

UU N I X N I X II N T R O D U C T I O NN T R O D U C T I O N

Page 2: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

2

Lingo, definitions, and shortcuts Command-line: computer lingo for using a computer terminal window to type in commands that execute programs and applications, instead of a point-n-click operation (called a graphical user interface, or GUI, pronounced “gooey”) with a mouse. Terminal window: the standard text-based window where you will be typing UNIX commands. Standard escape/quit commands: Ctrl-C (escape, stop), Ctrl-D (quit)

Opening (and quitting) UNIX on a Mac or PC

Macs: To use UNIX on the Macs, you will need to bring up a regular

text-based window rather than the standard Mac windows. This can be done by running the "X11" program, which is normally in Macintosh HD/Applications/Utilities. During your first time using X11, you should drag the X11 icon to your Desktop dock so that you can easily access the program in the future.

- To open a new UNIX terminal, go to X11 menu Applications Terminal. - To exit your UNIX terminal (and close the window), type “exit”. - To quit X11, go to X11 menu X11 Quit. X11 can be downloaded for free from Apple if you do not already have it installed on your computer. Google: X11

PCs: To use UNIX on PCs, you will need to launch the Cygwin program,

which runs Xwindows. - Double click on the Cygwin icon on your desktop - In the terminal window, type “startxwin.bat” to start Xwindows

Page 3: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

3

Basic File & Directory Structure

There is a basic file structure within UNIX that sits on top of / or “slash”, or also called “root”. You may be familiar with the “/” symbol, as it is now the common symbol in website addresses – this is the same basic file structure notation in UNIX. Sitting on top of “root” are lower directories (the can be anything though) like pub, Users, etc. An example of a tour of directories on a Mac (PC’s will be different) might be: /Users/yourname/Desktop/somedirectory /Applications/MSOffice/….. /sw Let’s assume that you have successfully opened up a UNIX terminal on your computer. Your will now be located in your home directory. UNIX uses a directory tree structure, similar to the "folders" used by PCs but without all the fancy icons. Your home directory is often show as “~”. Many of the other directories like /bin /var /sbin and so-forth are not to be edited, and files within are pretty much off limits unless you really know what you are doing, and you have root or “sudo” permission.

Prompts

Unix uses a symbol called a cursor “prompt”, at the beginning of each window. This is where you will begin typing commands. This symbol varies from computer to computer, but typical symbols you might see are:

% $ : > Normally you will have a cursor prompt that will also tell you what machine you are running on. On a network computer, this might look like: geology12345 > geo234geo5 %

The notes that follow will represent a cursor prompt with the “%”.

Page 4: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

4

Navigating to Class Workspace (//geobase/GEO4315)

Mount the class workspace directory.

Before you begin working, you will need to connect to, (or “mount”) the //geobase/GEO4315 server

Macs: In your Finder Menu Go Connect to Server Enter Server Address: smb://geobase/GEO4315 Click Connect button If prompted, enter your Miners login or username & password Navigate to your own folder

PCs: Start My Computer Tools menu Map Network Drive In Folder, enter: \\geobase\GEO4315 If prompted, enter your Miners login or username & password Navigate to your own folder

Navigating to your directory in a UNIX window

Macs: % cd /Volumes/GEO4315 % ls % cd yourfolder

PCs: % cd \cygdrive\z\ % ls % cd yourfolder

Page 5: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

5

BASIC UNIX COMMANDS

----------------------------------------------------------------------------------------------------------- pwd To find out where you are in the directory structure, enter "pwd" at any time, which stands for "Print Working Directory." % pwd (type this, computer will return: /directory/directory) ----------------------------------------------------------------------------------------------------------- ls You can “list” the contents of your current directory with the "ls" command: % ls (I'm not showing you the output because it's too messy!) There usually are options for UNIX commands that can make them more useful for what you want. For example, "ls" simply lists the file names in your directory. If you want to find out how big they are or when they were last modified, then use "ls -l" where the "l" flag stands for the long output option. Sometimes you will also have important “invisible” files that begin with a “.__”. These files will not appear if you just enter "ls". To make them appear, enter "ls -a", where the "a" flag stands for the "all file" option. You can also combine flags: % ls -la ----------------------------------------------------------------------------------------------------------- cd To change your directory, use: % cd directoryname This will move you to a new directory. Note that this directory name must be in your current directory or path. OR you can give the full path name for directoryname:

Page 6: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

6

% cd /Volumes/GEO4315/bkonter will get me to my directory no matter where I am on the system. Alternatively, one can go to one's home directory by entering % cd or % cd ~/ You can also go directly to subdirectories by entering: % cd ~/dir1/dir2 This will get you to directory dir2, located in dir1 in your home directory. Naturally this does the same thing as: % cd % cd dir1 % cd dir2 You can go back up one directory level by entering % cd .. You can go back up and into a different directory by entering: % cd ../dir3 ----------------------------------------------------------------------------------------------------------- mkdir To create a new directory, use the "make directory" command: % mkdir nameofdirectory It is often convenient to use a different naming convention for your directories in order to distinguish them from your files. Some people use all capital letters for the directory names. This makes them more visible, but has the disadvantage of slowing down typing their names (yes, UNIX is case sensitive!). The file names should NOT contain spaces or any of the following special characters:

Page 7: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

7

& * \ | [ ] {} $ <> ( ) # ? ‘ " / ; ^ ! ~ %

If you don't want to use special names for directories or if you find yourself in somebody else's directory where they don't do this, you can use the "ls -F" command: % ls -F This will add a "/" suffix to the list directory names and a "*" prefix to executable file names (programs). ----------------------------------------------------------------------------------------------------------- rmdir To remove a directory, use the "remove directory" command: % rmdir nameofdirectory The directory must first be empty for this to work. To recursively remove a directory and its contents use: rm -r dirname Use this with extreme caution to avoid accidentally deleting more than you intended! ----------------------------------------------------------------------------------------------------------- rm To get rid of a file, use the "remove" command: % rm filename ----------------------------------------------------------------------------------------------------------- mv To change the name of a file, use the "move" command: % mv filename1 filename2 mv can also be used to move files between directories:

Page 8: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

8

% mv filename dirname This will move filename into directory dirname (assuming dirname already exists!) where it will have the same name. Note that this does the same thing as % mv filename dirname/filename For convenience, you can leave off the /filename if the file is to keep the same name. A note of caution: in the short version, if dirname does not exist as a directory, then the name of filename will be changed to dirname. ----------------------------------------------------------------------------------------------------------- cp To make a “copy” of a file, type: % cp filename1 filename2 This makes a copy of filename1 called filename2. % cp –R directory1 directory2 This copies an entire directory (and its contents) to another location. To prevent overwriting a file with a name that already exists, use: % cp -i filename1 filename2 This will first ask if you really want to do this if filename2 already exists. You can copy files to different directories in the same way as the mv command works.

Page 9: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

9

FILES AND EDITING

Files can be simple ascii (text) files or they can be binary files, which are often the executable versions of computer programs. To create and modify text files, we use a text editor. Vi: One standard UNIX editor is called Vi and is still used by some of the old time programmers. If you know all of its tricks it can be an extremely powerful editor. If you know vi or decide that you want to learn it--more power to you. You will not lose any points around here. However, vi is not mouse or window friendly and is not favored by most students today. Most of us have forgotten most of the vi that we once knew. Enough about Vi….. Nedit: The UNIX text editor program we will be using in this class is called Nedit. It is freeware, and runs on both Macs and PC UNIX systems. Name conventions: UNIX file names are case sensitive. By convention, the type of file is often indicated by ".type", for example: earthquakes.dat a simple table of data figure1.ps for a Postscript file figure1.gif for a GIF file testprog.m for a MatLab script testprog.f for Fortran77 program source code testprog.c for C program source code You may wish to create your own naming convention to keep track of your files. When used with wildcards (see below) this will make it easy to list all files of a particular type. WARNING: Do not use the dash character ("-") in file names; this may cause all kinds of problems for you. Use ”.” or “_” to separate the words. NEVER USE BLANKS IN FILE OR DIRECTORY NAMES!!! (we know this is common on Macs and PCs but you will eventually have big problems reading these files with your programs) ----------------------------------------------------------------------------------------------------------- Creating/editing a file To open an Nedit window and create a new file or modify an existing file, type

Page 10: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

10

% nedit filename & It is pretty self-explanatory. It brings the file up in a separate window and you can use the mouse and the scroll arrows. With Nedit, the cut and paste buttons on the keyboard work with this. The “&” symbol part of the command you entered puts the window “in the background”, which means it allows you to work in the Nedit window, but also returns the cursor prompt in your UNIX terminal, so that you can carry on with other tasks. If you forget to include the &, type “CTRL-Z” and “bg”. Also, if you do not specify a file name, Nedit simply opens up a new, blank file.

Page 11: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

11

WILDCARDS UNIX commands become much more powerful when they are used with the wildcard character "*", which can take on any ascii string of characters. For example, suppose you wanted to list all files in your directory ending with ".jpg”, the suffix used to identify JPEG image files. Simply enter: % ls *.jpg You could move all of these image files into a subdirectory called myjpgdir by entering: % mv *.jpg myjpgdir You might have a bunch of old plot files called myplot1, myplot2, etc. You could delete all of these at once by entering: % rm myplot* For obvious reasons, be very careful when using "*" with the rm command. Always look carefully at what you have typed before hitting the return key when you are deleting files using wildcards.

HELP (MAN) PAGES UNIX help (manual) pages UNIX has an online set of manuals that may be accessed with the "man" command. For example, suppose you forgot what "pwd" does. You could type % man pwd and you would get a description of the pwd command. One annoying aspect of standard UNIX is that the man command output uses a form of the VI editor rather than the normal window output, which permits scrolling up and down in the window. When you enter "man pwd" you will get a page of output on the screen with a colon (":") at the bottom. If you enter the spacebar, you will then get the next page of output, etc. But you can't scroll backwards using the window scroll arrows. To go backwards, enter "b" at the colon prompt. However, if you go past the final page, you will be dumped out of the man pages completely.

Page 12: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

12

SSH, SCP, and SFTP (remotely logging in & transferring files between machines)

note: ssh, scp, and sftp commands are not currently working

on the PCs ----------------------------------------------------------------------------------------------------------- SSH (secure remote login) to www.geo.utep.edu Ssh is the command you should use for logging into a remote computer (i.e., a computer other than the one you are sitting at). To remotely log in to the Department UNIX server (where your webpage materials), use the following command: % ssh [email protected] You will need to enter your UNIX password, and then use the cd command to change directories to /pub/username to navigate to your web files. Here, you can remove unnecessary files, re-name them, move them to different directories, etc. % cd /pub/username % ls % (enter commands like mv, rm, cp, etc. here) If you ever get error messages such as “You do not have permission to view this file”, or “Forbidden …”, you will need to update the permissions on your files. The standard way to do this for files that you want others to see on the internet, and that you also want to be able to change, is to type the following command in your /pub/username folder on www.geo.utep.edu: % chmod 755 * or % chmod 755 dirname

Page 13: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

13

----------------------------------------------------------------------------------------------------------- SCP (secure remote copy) to www.geo.utep.edu Any easy way to copy over a file or directory is to use the scp command: % scp filename [email protected]:/pub/bkonter/ will copy filename from the current directory to another directory in bkonter’s account on www.geo.utep.edu (after first prompting for the password). Note that you can replace the www.geo.utep.edu name (or IP address) with that of another computer. If you want to copy a directory, you need to use the -r (recursive) option, i.e., % scp -r dirname [email protected]:/pub/bkonter/dirname Note that one can also copy files from the remote computer to the current directory, i.e. scp -r [email protected]:/pub/bkonter/dirname local_dirname In this case, the remote directory was on /pub/bkonter/dirname on www.geo.utep.edu, and I wanted to copy it to my local computer and name it loca_dirname. ----------------------------------------------------------------------------------------------------------- SFTP (secure file transfer) You often will want to get files from another computer or transfer files from your computer to another computer. Another way of doing this is the sftp command. % sftp [email protected] Once within sftp, you will get a sftp> prompt. You then can use the "cd" command to get to the directory that you want and "ls" to see the file names on the remote computer. On www.geo.utep.edu, you’ll need to cd to /pub/username/……. Finally, you will use "mget" to bring the desired file to your own computer:

Page 14: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

14

% mget filename_that_i_want_to_get Or, if you want to transfer a file to the remote computer, use the “mput” command: % mput filename_that_i_want_to_send To exit from the sftp dialog, type "quit" to exit. If you want to get or put all of the files in the directory, use the command % mget * or % mput * and you will be prompted for each file name (this is called interactive move). If you don't want to be prompted, you can turn off the interactive mode by entering a “-I” when you first invoke sftp: % sftp -i [email protected]

Page 15: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

15

MISCELLANEOUS COMMANDS ----------------------------------------------------------------------------------------------------------- history (h) Unix keeps track of your previous commands. To see them, enter "history" (for history) and it will list your last 30 commands. ----------------------------------------------------------------------------------------------------------- Shortcut for repeating commands To repeat the very last command, use the up or down arrow keys on your keyboard to scroll through your previous list of commands Also note that if you want to copy a command from some file in Nedit or notes into your terminal window, experiment with highlighting the command with the mouse, then hold down the keyboard “option”, “control”, or “propeller” keys and click in the window you want the command to paste into. Every computer platform and keyboard is different (hence the use of the term “experiment”), however usually all of them simulate some kind of shortcut like this for copy and pasting. Mac X11 users: this is typically done by holding down the option key, however you will need to check your X11 preferences (input panel – emulate three button mouse). ----------------------------------------------------------------------------------------------------------- Quick view of the start or end of a file If you want to see what the beginning or end of a file looks like, use the "head" or "tail" command: % head filename --- lists the first 10 lines of the file % tail filename --- lists the last 10 lines of the file ----------------------------------------------------------------------------------------------------------- Viewing contents of files To see the contents of a file one page at a time, use the "more" command" % more filename (note: this command doesn’t work on PCs yet) and then hit the spacebar to advance one page at a time. ----------------------------------------------------------------------------------------------------------- Finding files

Page 16: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

16

If you lose track of where a file is, you can use the "find" command: % find . -name filename -print This will look in the current directory (that's what the "." is for) and below for the file named "filename" and then print where it is. If you only know some of the characters of the file, you can use a wildcard: find . -name 'map*' -print This will find all files that begin with "map" and print them on your screen. Note that you MUST enclose 'map*' in apostrophes for this to work. ----------------------------------------------------------------------------------------------------------- Word count (wc) To count the number of words or lines in a file, use: % wc –w filename (number of words) % wc -l filename (number of lines) ----------------------------------------------------------------------------------------------------------- Previewing a postscript file/image To preview a Postscript file (denoted ___.ps) on the screen use the following commands. Note that ghostscript, ghostview, and pageview are programs that will need to be installed on your computer before you can execute them. % ghostscript filename.ps or % ghostview filename.ps or % pageview filename.ps I usually use ghostscript first as it seems faster, but ghostview is more reliable for all types of Postscript files. (***Note: Macs do not always support pageview!!)

Page 17: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

17

ADVANCED UNIX COMMANDS The following section contains extra commands within UNIX that you may find useful. You can go pretty far with only 20 commands or so. However there are much more powerful things you can do. If you really want to be a UNIX guru, try reading up on the “man” pages for pipes, grep, awk, sed, etc. Then you will be able to write custom scripts to do all kinds of neat things. Here are some examples: ----------------------------------------------------------------------------------------------------------- sort UNIX has many powerful utility programs. One of these is the "sort" command to sort files in alphabetical or numerical order. Example: % sort +4 -n -b -r file1 -o file2 This sorts file1 and outputs (“-o”) the results to file2. The following options are used in this example: +4 skip first 4 fields (leave out to use beginning of line) -n numerical order (default is alphabetical order) -b ignore leading blanks -r reverse order (leave out for standard order) ----------------------------------------------------------------------------------------------------------- grep The grep command searches for and prints lines that match a pattern that you specify. % grep dziewonski filename This will print every line from file "filename" that contains the string "dziewonski". % grep -v dziewonski filename This will print every line from file "filename" that does NOT contain the string "dziewonski". % grep dziewonski filename > dz.lines This works as above except the lines containing dziewonski are written to the file dz.lines rather than printed to the screen. % grep dziewonski *

Page 18: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

18

This lists lines containing dziewonski from ALL files within your current directory. However, you may really want just the file names of the files that contain the string dziewonski, in which case the following command will work better: % grep -l dziewonski * This lists the file names of the files that contain the string "dziewonski". ----------------------------------------------------------------------------------------------------------- concatenate (cat) To concatenate (pull together) and print file contents, use the cat command: % cat file1 file2 This will print the lumped contents of both files to the screen. % cat file1 file2 > files1_2 This will print the lumped contents of both files to a new file: files1_2. ----------------------------------------------------------------------------------------------------------- awk To scan and grab contents of one file, use the awk command: % awk '{print $5,$3,$1}' file1 > file2 File1 has 5 columns of data (denoted as $1 $2 $3 $4 $5). This command copies columns 5, 3, and 1 to file2, in that order, omitting the 2nd and 4th columns. % awk '{print $2, $1*(-10)}' file1> file2 This command switches columns 1 and 2 from file1 and multiplies the original 1st column numbers by -10. ----------------------------------------------------------------------------------------------------------- sed Called the “stream editor”, the sed command can substitute strings of characters in a file:

Page 19: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

19

% sed 's/Peter/Paul/' file1 > file2 This copies file1 to file2, substituting "Paul" for the first "Peter" on each line. % sed 's/Peter/Paul/g' file1 > file2 This copies file1 to file2, substituting "Paul" for the every "Peter" on each line (note the "g" flag for global substitution). % sed 's/^/Paul says /' > file2 This inserts the prefix "Paul says " at the beginning of each line. Note that "^" means start of line

Page 20: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

20

AND FOR THOSE OF YOU THAT PLAN TO BECOME EXPERTS… ----------------------------------------------------------------------------------------------------------- UNIX Shells UNIX comes in different flavors. These flavors are guided by what is called a “Shell”. For this class, I am going to assume that you are running what is called the C-shell ("csh" or "tcsh"). It is possible that your PC or Mac is running something called "bash", which many people think is better than csh. The default C-shells for Macs changed from tcsh to bash between Jaguar and Panther (OS X 10.2 to 10.3). However because these notes are based on C-shell, you should make sure your Mac is running csh or tcsh (if you think bash is better, than you probably are more of a UNIX expert than me and don't need to be taking this part of the course anyway!). To find out what type of shell you terminal is running, you can type % printenv within your UNIX shell and it will tell you lots of interesting things, including what your SHELL (or XTERM SHELL on Macs) is. In some terminal windows, your prompt may also indicate the shell. To change to a csh or tcsh shell, type: % tcsh (or csh) ----------------------------------------------------------------------------------------------------------- THE .login and .cshrc FILES In your home directory, you can have a files called .login and .cshrc (or .tcshrc or .bash) that are executed whenever you login. These files are used to define and customize your environment. You may have default .login and .cshrc files set up by the System Administrator when you first log on, but you can modify them to do what you want. To inspect your .tcshrc file, go to your home directory and either open the file in an editor (Nedit) or type “more .tcshrc”.

Page 21: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

21

One of the most important things in the .tcshrc file is the "set path" command. This lists all the directories that the computer should look in when you try to run a program. For example, if you type "matlab" the computer needs to know where to look to find the matlab program. If you get a "Command not found" message, then you don't have the right directories listed in your .tcshrc file under "set path". Sometimes the System Administrator will start you with a .tcshrc (or .bashrc) file that does not list your current directory under the "set path" command. In this case, you will not even be able to run a program that is sitting right in your current directory. How humiliating! Don't feel inadequate, just make sure you have a '.' listed in your "set path" command. For example, here are some of Bridget’s set path commands on her Mac: set path=($path .) set path=($path /usr/X11R6/bin ) set path=($path /sw/bin ) set path=($path /sw/sbin ) set path=($path /Applications ) set path=($path /Applications/Utilities ) set path=($path /Applications/MATLAB6p5p1/bin) set path=($path /Developer/Applications ) Notice that there are tons of places to check for executables. The most important one, however, is the “.” at the beginning which stands for the current directory. You can always look at other people's .tcshrc files if you have trouble with yours. As a grad student, I borrowed my advisor’s .tcshrc file to get started, and there were lots of things in this file that I didn't really understand but just figured they must be doing something! If you make any changes to your .tcshrc file (or your .login file), they won't take place until your next login. Alternatively you can enter: source .tcshrc to make the changes immediately. BUT, you will have to do this separately for each window that you have open. For command flags that you use often, you might want to make aliases of these in your .tcshrc file. Many people prefer the "-i" option for mv and cp that they make it the default option by defining an alias so that "mv" and "cp"

Page 22: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

22

become "mv -i" and "cp -i". I recommend that you do this--it is likely to save you some grief in the future: alias cp 'cp -i' alias mv 'mv -i' ----------------------------------------------------------------------------------------------------------- FILE COMPRESSION Often the files that you retrieve will be compressed. Files are compressed using the UNIX "compress" command: % compress filename This will change the name to filename.Z which tells you that it is compressed. This is useful to save disk space when you will not be using the file for awhile. To get back to the original file, use uncompress: % uncompress filename You can compress a whole bunch of files by using wildcards: % compress *.ps will compress all of your Postscript files, assuming you use the .ps suffix convention for these files. These can be uncompressed with "uncompress *.ps" as you would expect. An alternative compression method (not standard UNIX but usually available) is invoked with the gzip command: % gzip filename This changes the name to filename.gz with the reverse operation: % gunzip filename The gunzip command will also decompress .Z files (but the uncompress command will not decompress .gz files). You may find it useful to use compression yourself by compressing files that you do not use very often in order to save space. I often do this when I want

Page 23: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

23

to get some disk space but am too nervous to delete the files and too lazy to write them to a backup drive. ----------------------------------------------------------------------------------------------------------- USING THE TAR COMMAND Often you may want to save or retrieve an entire directory of files. This is most easily done using the "tar" command. If you are within the directory containing the files that you wish to save, then enter: % tar -cvf ../archive.tar . The arguments are as follows: -c create tar archive -v verify by printing file names to screen -f output file name will follow ../archive.tar name for tar file (../ to put in next level up) . tar every file in current directory Alternatively you can save the entire directory and its contents from the level above the directory: % tar -cvf programs.tar programs.dir The tar file can then be ftp’d to another machine. For even more efficiency you may wish to compress the tar file first. The files can be retrieved as follows: % tar -xvf archive.tar This will put all of the files in the archive into the current directory. Finally, tar is also commonly used to read/write data to tape, in which case the file name (e.g, ../archive, programs.tar, etc.) in the above examples is replaced with the name for the tape device. You must be logged on to the machine that the tape drive is connected to (ssh to this computer).

Page 24: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

24

----------------------------------------------------------------------------------------------------------- top (note: this command doesn’t work on PCs) To find out what jobs are running on your machine, you can use the "top" command to list the most active jobs: % top This will take over your window and update the results continuously until you enter q for quit. The Process ID (PID) is listed, together will the username, the niceness, the faction of the CPU being used and other useful information. top is interactive and you can input various commands (? for help, u to see only one user. etc.). You can also kill jobs from within the top program. ----------------------------------------------------------------------------------------------------------- df, du To find out how much disk space is available use df (disk free): % df -h This will not list all the disks on the system unless they have been mounted. Just go to the disk that you are interested in and retry "df" if it does not appear the first time. The -h flag works on Macs and SGIs and writes the blocks out in human units like Gbytes, not 512kilobyte blocks! To see how much space you are using, the best command is % du -ks * This will list the disk usage of each of your subdirectories. It is a good idea to go through your directories once a month or so to delete unnecessary files and/or compress large files that you don't use very often. ----------------------------------------------------------------------------------------------------------- Process status (ps) This command will help you inspect what programs are currently running on your computer. This is particularly helpful if your computer seems slow – it

Page 25: L8 Unix Notes - UTEP Geological · PDF fileNotes compiled by B. Konter (UTEP) and P. Shearer and G. Kent (UCSD) The purpose of this handout is to provide you with basic Unix commands

COMPUTER APPLICATIONS IN THE GEOSCIENCES

25

may be using its CPU for other (perhaps unnecessary) jobs. % ps -eaf This lists all processes that are running on your machine ----------------------------------------------------------------------------------------------------------- Kill process (kill) To terminate a job or process, use the kill command: % kill PID This kills a job with process ID number PID (obtained from the top or ps command). This is useful for runaway jobs. For stubborn jobs, use the -9 option: % kill -9 PID % diff file1 file2 This lists all differences between file1 and file2. This is useful if you have made some changes to a file but cannot remember exactly what they are.


Recommended