UNIX_DAY1_V1.3

UNIX / LINUX

CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 3

Objectives At the end of the session, you will be able to:

Understand Unix File System Use Unix basic and filter commands Read and write shell scripts Read and write awk scripts Understand process management in UNIX Understand the general utilities e.g. gzip, ftp & tar


Agenda Introduction to Unix Navigating the File System General Purpose Commands Pipes and Filters Shell The vi Editor Shell scripting AWK scripting Process Management General Unix Utilities

UNIX

Day 1


Objectives At the end of the session, you will be able to:

Define UNIX Operating System Understand UNIX O.S. Components e.g. Kernel & Shell Understand Unix File System Execute Unix file related general commands Understand file access permissions Understand Pipe and redirection Understand filter command Edit a file using vi editor


Agenda: Day 1 Unix Overview

Unix File System

Unix Commands

Vi editor

Unix Overview


History of Unix Developed in AT&T Bell Labs by Ken Thomson as a single

user OS in 1969

Initially written in assembly language

Developed as multi-user OS later

Rewritten in C in 1973

Licensed to university for educational purposes in 1974

POSIX (Portable Operating System for Unix) was developed POSIX is the standard from IEEE.

One of the POSIX guidelines is towards making system calls interface across all UNIX flavors, same.


Unix Flavors AIX by IBM

Solaris by Sun Microsystems

HP-UX by Hewlett-Packard Company

IRIX by Silicon Graphics, Inc.

FreeBSD by FreeBSD Group

GNU/Linux by Open Source Movement

MacOS by Apple Computer, Inc.

SCO Unix by The Santa Cruz Operation Inc.


Features of Unix Multi-user

Multi-tasking

Portable

Interactive

Shell

Security

Hierarchical File System


Unix Architecture

Hardware Kernel

Shell & Unix Utilities

Other Application Program

13

UNIX Onion Architecture (1 of 2)

13

Shell

Shell

User ProcessShell

User ProcessShell

Everything below the system call interface and above the physical hardware is the KERNEL.

It provides file system, CPU scheduling, memory management, and other OS functions using system calls.

14

UNIX Onion Architecture (2 of 2)

Shell

Shell

User ProcessShell

User ProcessShell

The system call interface layer converts a process running in user mode to a protected kernel mode process.

System call provides a programming interface that allows user programs to access kernel functions.

15

User Application and Kernel Interaction

Kernel controls the underlying hardware. The kernel provides low-level device, memory, and processor management functions.

Shell is a command interpreter of UNIX.

Utilities are set of UNIX commands and Programs.

Hardware

Kernel

The Operating System

ShellUtility

Applications

User

Applications are different softwares run on UNIX.

Users enter UNIX commands or run different software applications.

16

Knowledge Check (1 of 3) Which of the following are true about the operating system?

1. It manages memory.2. It manages processor.3. It manages Input/Output devices.

17

Knowledge Check (2 of 3)

The operating system comprises a set of software packages that can be used to manage interactions with the hardware. The elements that are generally included in this set of software are listed below. Which is the odd one out?

1. Input/Output Devices2. Shell3. File System4. Kernel

18

Knowledge Check (3 of 3) What are system calls? Describe their uses.

System Call:It is used to invoke an operating system routine. The routine is stored in the kernel.

Uses: (This is not exhaustive)1.Through system calls many low level jobs like read, write, open,

exec, kill etc can be executed efficiently.2.System calls provide the interface between a process and the

kernel. 3.The OS routines have highest level of privilege. When a system

call is triggered, the system enters a higher privilege level, and the routine is executed.

4.System calls provide an opportunity to execute a specific set of instructions which the interrupting program has no direct control over.

Unix File System


Root Directory Structure

cat

ls

user2 user3user1

progrm1

progrm2

bin boot home etc usr dev lib

Desktop

start

/

Directory

File


1./bin stores external command files for the ordinary user

2. /boot stores OS kernel information3. /home stores user home directories (

/export/home for Solaris)4. /etc stores the OS configuration files5. /dev stores the device files6. /lib stores the library files


File Basics Everything on Unix is a file. File structure is hierarchical like

an upside down tree.

File is just a sequence of bytes.

The meaning of the bytes depends solely on the programs that interpret the file.

The format of a file is determined by the programs that use it.


File Attributes Name (such as “test.log”)

An Owner (such as “sandhya”)

Access Rights (such as read, write, execute)

Other attributes (such as date of creation)


File Types Every item in a UNIX file system belongs to one of the three

possible types:

1. Ordinary/Regular files

1. Directory files

1. Device/Special files


Ordinary File Contains text, data, or program information

Cannot contain another file or directory

Can be thought of as one-dimensional array of bytes


Directory File Contains directory(s) and/or file(s) within it.

Has one line for each item contained within the directory.

Each line in a directory file contains only the name of the item, and a numerical reference to the location of the item, called inode number.

Inode number is an index to a table known as the inode table. Inode stores all information about the file except its name.


Device File Physical devices (printers, terminals etc) are represented

as “files”.

Two types of device files:1. Character Special2. Block Special


Links Links make the same file available in multiple directories at

the same time Two types of Links:

1. Hard Link2. Soft Link or Symbolic Link

1. Hard Link A hard link is another name given to the existing file These names share the same inode

2. Soft Link A soft link to a file has a separate inode than the file It stores the target file’s path in its inode


Inode Inode is a data structure containing useful information about

an item in the Unix File System.

Inodes reside on disk and do not have names. Instead, they have indices (numbers) indicating their positions in the array of inodes as shown in the next slide.


Inode


Pathname Every item in the file system with a name can be specified

with a pathname.

Pathname represents the path to the entry from the root of the file system. By following this path, the system can find the inode of the referenced entry.

Pathnames can be absolute or relative.


Unix Users Super User

Owner

Group

Others


Unix Users Superuser

Can also be referred to as a System Administrator Has an overall authority on Unix OS Responsible for OS maintenance, backup and recovery, user

management etc. Superuser login is root and prompt is #

Owner Is a user who creates a file For every Unix file there can be only one owner File owner can assign the file permissions to group and other

users


Unix Users Group

In Unix, groups can be formed based on area of work Superuser can create a group and assign members to it Owner of a file can decide what permissions to be given to

group members

Others User who is not a owner and does not belong to any specific

group is referred to as other user Owner of a file can decide what permissions to be given to

other users

Unix Commands


Basic Commands $ pwd

Shows working i.e current directory

$ whoShows who is logged on

$ who am iShows the login name of the current user


Basic Commands $ man command formats and displays online manual pages

The manual pages are divided into logical grouping of the commands called the sections. Sections are numbered from 1 through 9. For example, commands are 1, system calls are 2, library function are 3, file formats are 5, & management commands are 8.

If section is specified, man only looks in that section of the manual.

The command man 2 open displays the manual for the system call open.


Basic Commands $ date: command prints or sets the system date and time

For example,$ date$ date –r TestFileDisplays the last modification time of the file “TestFile”

This command can be used with suitable format specifiers as argument. Each format is preceded by a + symbol, followed by the % operator, and a single character describing the format.


Basic Commands

Sequence Interpretation

%a abbreviated weekday name (Mon .. Sun)%b or %h abbreviated month name (Jan .. Dec) %d day of month (01 .. 31) %r time (12- hour) %T time (24- hour) %y last two digits of year (00 .. 99) %D date (mm/dd/yy)

e. g. $ date “+Today is %a %m %Y”


Basic Commandsecho “Hi $USER \n Welcome to Unix”

echo ‘Hi $USER \n Welcome to Unix’

echo Hi $USER \n Welcome to Unix

cleartput clear

tput cup <row_coordinate> <column_coordinate>tput cup 15 20


Basic Commands

$ which <command_name> Shows the path of the specified command

$ type <command_name> Shows the type of the specified command – shell built-in

or file path if it is external

$ file list1 Shows the type of the file whether regular file or

directory file or some other file


Basic Commands $ cal

Shows the calendar of the current month/year

$ bc 2.3+5.4

Shows the mathematical calculations involving integers as well as floats

bc is a filter command

$ write user1 Lets to write a message to the other user; user1 in this

example


File Management Commands$ ls

command to list files and directories

Option Description-l list in long format-C multicolumn output

-F indicates type of file by /, * -R recursive listing of all subdirectories encountered

-a list all files including hidden files-i List all files along with i-node number


Wild Card Characters * - zero or more characters – e.g. ls m* lists all the files

starting with m

? – single Character – e.g. ls m? lists all the files starting with m and having one character after that

[]- any character from all the characters within [] e.g. ls [aeiou]* lists all files starting with a or e or i or o or u

- Specifies range

! – works as not operator e.g. ls [!x-z]* will list all the files not starting with any character from the range x-z


Redirection

Redirection

Output Redirection

Error Redirection

Input Redirection


Input Redirection Input Redirection specifies that the standard input, instead

of coming from the standard input file, comes from a disk file

$cat < filename e.g. cat < data1 e.g. cat 0< data1


Output Redirection The output redirection specifies that the standard output of

a given command is written to the disk file, instead of monitor

$ cat filename1 > filename2 e.g. cat data1 > data2 e.g. cat data1 1 > data2


Error Redirection The error redirection specifies that an error is written on the

standard error-msg file, instead of the monitor

$ cat filename 2>error-msg file e.g. cat data4 2>error-msg


File Management Commands cat

To concatenate files To display contents of one or more ordinary files To create an ordinary file

$ cat file1 file2 … It displays contents of all files specified on the command line

one below the other

$ cat > file It creates a new file by accepting text from the standard input Press CTRL-d to save and exit the file


File Management Commands mkdir [-p] dirname

Makes a directory of a given dirname The dirname can contain the full path prefix of the directory to

be created More than one directories can be created at the same time When executed with –p option, it does not give any error if the

directory dirname already exists When executed with option –p, it makes parent directories in

the path, if needed (if any parent directory in the path is not available)

$ mkdir dir1 dir2 dir3 Makes directories dir1 dir2 and dir3 in the current directory


File Management Commands rmdir dirname

Removes an empty directory

Avoids the chances of accidental removal of a directory with some useful data files.

$rmdir dir1 dir2 dir3 removes directories dir1 dir2 and dir3 from the current

directory, given that they are empty


File Management Commands cd [directory]

changes working directory to the directory, if specified; otherwise to the home directory

cd .. moves to the parent directory

cd or cd ~ changes to the home directory


File Management Commands cp command copies files and directories

$ cp –i file1 file2 Copies file1 to file2 -i - informs user before overwriting, if file2 exists

$ cp file1 file2 … dest_directory Copies multiple files in the specified existing directory

$ cp -r directory1 directory2 … dest_directory Recursively copies files from directory1, directory2 etc. to the

dest_directoryNote: Shell Meta-characters can also be used with “cp”


File Management Commands mv command changes name of the file or moves the file to

the specified destination path

$ mv file1 new-file Renames file1 as new-file

$ mv file1 file2 … dest_directory Moves multiple files to the specified existing directory

$ mv directory1 directory2 … dest_directory Moves one or more directory subtrees to an existing or new

dest_directory Note: Shell Meta-characters can also be used with “cp”


File Management Commands rm command is used for deleting unwanted files/directories

$ rm [-i] file … It is interactive removal (option –i) of specified files

$ rm -r directory … It is recursive deletion of all the files within the specified

directories and also the directories themselves

Note: Shell Meta-characters can also be used with “cp”


Creating Links ln file1 file2

Creates a hard link to existing file file1 with the name file2 in the current directory

Both names point to the same inode

ln –s file1 file3 file3 is a soft / symbolic link to file1 here Removal of file1 affects file3

ls –i file1 file2 file3 Inode number of the file file1 and file2 are same but inode of

the file3 is different


Finding Files and Directories find command is used for finding files and directories in the

specified file tree based upon certain criteria

Command Syntax:$ find path criteria actione.g.

$ find /usr -name “ymess*” –print

Finds files recursively in the directory in path The path can be absolute or relative Multiple directories can be written in the path In the criteria, wild cards are allowed


Finding files by name:

find /home/user1 -name “*.sh” -print

finds all files with .sh extension in /home/user1 and its subdirectories

Finding files by type:

find /usr –type d –print

finds all directories in /usr and its subdirectories

find /var –type f –print

finds all regular files in /var and its subdirectories

Finding Files and Directories


Finding Files and Directories Find files and take some other action than printing

find . –name “*.sh” –exec cp {} {}.bak \;

Finds all files with the extension .sh, in the current directory and its subdirectories, and then prepares a copy of each file with .bak extension


File Comparison cmp command compares two files and reports the location

of first mismatch

$ cmp file1 file2file1 file2 differ: byte 1, line 1


File Comparison comm compares two sorted files and produces result in

three colums:Lines: Unique to file1 Unique to file2 Common

$ comm file1 file2 Anil 12 2000

Ajit 12 2000 Dinesh 27 1400

Sunil 12 2000


File Comparison diff command compares two files and proposes the changes

in first file to make two files identical

$ cat file1Anil 12 2000Ajit 12 2000Sunil 12 2000

$ cat file2 Anil 12 2000Dinesh 27 1400

$diff file1 file2 2,3c2< Ajit 12 2000< Sunil 12 2000---> Dinesh 27 1400


File Access Permissions chmod command is used to change the permissions on a file

for owner, group and otherschmod <permission><filename>

Symbolic Method: Code Meaning a all

u userg groupo other+ add- remove= assign

chmod u+x, g-w, o+r, o-wx sample


File Access Permissions Absolute value Method: Code Meaning 4 Read 2 Write 1 Execute $ chmod 754 sample


Pipes It is used to pass the output of one Unix command as an input

to another Unix command e.g. $ ls >temp

$ more temp

ls |

moreInput Output

| $ ls | more

pipe


Text Manipulation Commands: Filters A filter is a SHELL command which takes an input ( either

from a standard input device or a file ) , processes it and send the processed out put to a standard output device or a file

At run time, the system supplies data to the filter as standard input. This standard input file can not be altered by the program.


Filters wc: counts lines, words and characters $ wc -[wlc] [filename]

head: displays first ‘n’ lines, horizontal slicing $ head -[n] [filename]

tail: displays last ‘n’ lines, horizontal slicing$ tail -[n] [filename]

split: divides files horizontally$ split -[n] [filename]


Filters cut: cuts file vertically either column wise/field wise

$ cut -[cfd] [filename] -c columns/characters -f field number -d field delimiter/separator

$ cut -c2-5 sample cuts columns 2 to 5 from the file sample


Filters $ cut -c1,2 file

Cuts 1st and 2nd characters from file

$ cut -d” “ -f1-2 file Cuts first 2 fields of the file

$ cut -c2- names Starts cutting from 2nd char to the end of line


Filters paste: merges lines of specified files and display onto

standard output $ paste -d[field separator] [list of files]

sort: ordering text files $ sort filename


Filters uniq: removes duplicate lines from a sorted file

$ uniq –[dcf] [file]-d only print duplicate lines-c prefix lines with number of occurences-f2 avoid comparing first two fields

nl: Displays file contents along with line nos. $ nl –[options] [files]


Filters tee: takes input from some command and generates two

outputs. One is redirected to a file and other to standard output or next command

e.g. $who | tee tmp

who tee

tmp


Filters tr: translating Characters

$ tr [options] < [file]-d: deletes specified characters-cd: do not delete specified characters-s: substitute multiple occurrences of a character

by single occurrence.

$ tr “abc” “ABC” < samp: replaces all occurrences of a with A, b with B, c with C


Command Substitution One command can be used as an argument to another

command

$echo The date today is `date` $echo “There are `ls |wc –l` files”

vi Editor


The vi editor The vi editor is a screen-based editor which lets a user

create new files or edit existing files

A key concept in vi is combining a certain action with a movement

vi is extremely powerful in moving around within (or between) files


The vi editor A vi session begins by invoking the command “vi” with a

filename$ vi [filename]

You can start vi without a filename, but when you want to save your work, you will have to tell vi which filename to save it into.

The last line in the screen is reserved for some commands that you can enter to act on the text. This line is also used by the system to display messages.


Modes of Operation The three different modes of operations are:

Command mode: This is the default mode where you can pass the commands to act on the text, using most of the keys of the keyboard

You can switch to this mode using “Esc” key

Insert (Input) mode: To enter the text, you have to enter into input mode. Press key “i” to enter into insert mode from command mode

You can switch to command mode by pressing “Esc” key

ex mode or line mode: You have to save your file or switch to another file or make a global substitution in the file. You then have to use ex mode, where you can enter the instruction in the last line of the screen. To enter into this mode, press “Esc” key followed by “:”


Text Insertion Commands

Command Description

i inserts text before cursor position a Appends text after cursor position I inserts text at beginning of line A Appends text after end of line o opens line below current line to insert text O opens line above current line to insert text


Cursor Movement Commands

Command Description

h left by one characterl right by one characterk up by one linej down by one linew right by one wordb left by one word0 or ^ beginning of line$ end of line


Cursor Movement Commands

Command Description

ctrl+u move up half windowctrl+b move up full windowctrl+d move down half windowctrl+f move down full windowctrl+e scroll downctrl+y scroll up


Text Search Commands

Command Description

/text searches and highlights the text downwards n moves between highlighted text * searches the identical text on which the

cursor was ?text searches the text upwards


Text Deletion Commands

Command Description

x character under cursorX character before cursor[n]dw delete n wordsd0 beginning to cursor positiond$ or D cursor position to end of line[n]dd n lines from current linep p will put (paste) deleted lines to current

cursor position


Text Yanking Commands

Command Description

y character y0 beginning to cursor position y$ cursor position to end of line [n]yw copy n words [n]yy copy n lines from current cursor position

p p will put (paste) copied lines tocurrent cursor position


Text Manipulation Commands

Command Description

nc [space] overwrites next n characters with spacec0 overwrites the portion between

beginning of line to cursor position c$ overwrites the portion between

cursor to end of line cw overwrites current word

:%s/pattern1/pattern2/g globally replaces pattern1 with pattern2 on the specified lines

:%s/pattern1/pattern2/gc globally replaces pattern1 with pattern2 on the specified lines interactively


Text Related Commands

Command Description

ab <abb> <longword> set abbreviation for a long worduna string unset abbreviation>> right shifting a line<< left shifting a lineR replace characters starting with

current character till Esc is pressed


File Related Commands

Command Description

ZZ or :wq save and exit:w save & continue editing:q! quit without saving:r filnam I nsert contents of file filnam :[addr1,addr2]w filname write the lines between line number addr1 and line number addr2 in the file filename



Command Description

“a4yy copy 4 lines into buffer “ap paste contents of buffer a to current cursor position. Maximum 26 buffers are available buffer having names “a” to “z”ctrl-v select particular columns “byy copy selected into buffer b “bp paste contents of buffer b to current cursor position



Command Description

1,$s/source/target/ substitute string source by string target from line number 1to las lineu undo last change on the lineU undo last changes on the lineCtrl-R redo the undone changes:e edit multiple files :e# return to previous file


Visual Mode commands

Command Description

v_u converts the character to lower case v_U converts the character to upper case :sp splitting window Ctrl-w toggle between windows <Ctrl-w>j moves to lower window <Ctrl-w>k moves to upper window


Customizing vi set commands

Command Description

:set all displays all set option:set autoindent/ai does automatic indentation:set number/nu shows all line duly numbered:set showmatch helps to locate matching brackets:set tabstop=5 sets tab=5 for display:set ic ignore case while pattern matching

When the string “no” is prefixed to any option, it indicates that the option is inoperative


Summary In this session, we have covered:

Unix Overview Unix File System Unix Commands Vi editor

Thank You

Date post:	18-Jul-2016
Category:	Documents
Upload:	shaharukh-nadaf
View:	4 times
Download:	1 times

UNIX_DAY1_V1.3

Documents