Sed Command Tutorial

8/8/2019 Sed Command Tutorial

http://slidepdf.com/reader/full/sed-command-tutorial 1/42

Unix Sed Tutorial

Unix Sed Tutorial: Delete File Lines Using Address and Patterns ...................7

Unix Sed Tutorial: Find and Replace Text Inside a File Using RegEx ............ 11

Unix Sed Tutorial: How To Write to a File Using Sed .................................... 15

Unix Sed Tutorial: How To Execute Multiple Sed Commands ....................... 18

Unix Sed Tutorial: Advanced Sed Substitution Examples ............................. 20

Unix Sed Tutorial: Append, Insert, Replace, and Count File Lines ................ 23

Unix Sed Tutorial: Multi-Line File Operation with 6 Practical Examples ........29

Unix Sed Tutorial: 6 Examples for Sed Branching Operation ........................ 33

Unix Sed Tutorial : 7 Examples for Sed Hold and Pattern Buffer Operations 38

Página 1 de 42



Unix Sed Tutorial: Printing File Lines

using Address and Patterns

Let us review how to print file lines using address and patterns in this first part of sedtutorial.

We’ll be posting several awesome sed tutorials with examples in the upcoming weeks.

Unix Sed Introduction

• sed is a “non-interactive” stream-oriented editor. Since its an “non-interactive” it

can be used to automate editing if desired.• The name sed is an abbreviation for stream editor, and the utility derives many

of its commands from the ed line-editor (ed was the first UNIX text editor).• This allows you to edit multiple files, or to perform common editing operations

without ever having to open vi or emacs.• sed reads from a file or from its standard input, and outputs to its standard

output.• sed has two buffers which are called pattern buffer and hold buffer. Both are

initially empty.

Unix Sed Working methodology

This is called as one execution cycle. Cycle continues till end of file/input is reached.

1. Read an entire line from stdin/file.2. Removes any trailing newline.3. Places the line, in its pattern buffer.4. Modify the pattern buffer according to the supplied commands.5. Print the pattern buffer to stdout.

Página 2 de 42



Printing Operation in Sed

Linux Sed command allows you to print only specific lines based on the line number or

pattern matches. “p” is a command for printing the data from the pattern buffer.

To suppress automatic printing of pattern space use -n command with sed. sed -n optionwill not print anything, unless an explicit request to print is found.

Syntax:

# sed -n 'ADDRESS'p filename

# sed -n '/PATTERN/p' filename

Let us first create thegeekstuff.txt file that will be used in all the examples mentioned below.

# cat thegeekstuff.txt

1. Linux - Sysadmin, Scripting etc.

2. Databases - Oracle, mySQL etc.

3. Hardware

4. Security (Firewall, Network, Online Security etc)

5. Storage

6. Cool gadgets and websites

7. Productivity (Too many technologies to explore, not much time

available)

8. Website Design

9. Software Development

10.Windows- Sysadmin, reboot etc.

5 Sed ADDRESS Format Examples

Sed Address Format 1: NUMBER

This will match only Nth line in the input.

# sed -n ‘N’p filename

For example, 3p prints third line of input file thegeekstuff.txt as shown below.

# sed -n '3'p thegeekstuff.txt

3. Hardware

Sed Address Format 2: NUMBER1~NUMBER2

M~N with “p” command prints every Nth line starting from line M.

# sed -n ‘M~N’p filename

For example, 3~2p prints every 2nd line starting from 3rd line as shown below.

# sed -n '3~2'p thegeekstuff.txt

3. Hardware

Página 3 de 42



5. Storage


available)


Sed Address Format 3: START,END

M,N with “p” command prints Mth line to Nth line.

# sed -n ‘M,N’p filename

For example, 4,8p prints from 4th line to 8th line from input file thegeekstuff.txt

# sed -n '4,8'p thegeekstuff.txt


5. Storage



available)8. Website Design

Sed Address Format 4: ‘$’ Last Line

$ with “p” command matches only the last line from the input.

# sed -n ‘$’p filename

For example, $p prints only the last line as shown below.

# sed -n '$'p thegeekstuff.txt10.Windows- Sysadmin, reboot etc.

Sed Address Format 5: NUMBER,$

N,$ with “p” command prints from Nth line to end of file.

# sed -n ‘N,$p’ filename

For example 4,$p prints from 4th line to end of file.

# sed -n '4,$p' thegeekstuff.txt4. Security (Firewall, Network, Online Security etc)

5. Storage



available)

8. Website Design



6 Sed PATTERN Format Examples

Sed Pattern Format 1: PATTERN

Página 4 de 42



PATTERN could be unix regular expression. The below command prints only the linewhich matches the given pattern.

# sed -n /PATTERN/p filename

For example, following prints the line only which matches the pattern “Sysadmin”.

# sed -n /Sysadmin/p thegeekstuff.txt



Sed Pattern Format 2: /PATTERN/,ADDRESS

# sed -n ‘/PATTERN/,Np’ filename

For example, following prints lines which matches the pattern to Nth line, from input.

3rd line matches the pattern “Hardware”, so it prints from 3rd line to 6th line.

# sed -n '/Hardware/,6p' thegeekstuff.txt

3. Hardware


5. Storage


Sed Pattern Format 3: ADDRESS,/PATTERN/

It prints from the Nth line of the input, to the line which matches the pattern. If the

pattern doesnt match, it prints upto end of the input.

# sed -n ‘N,/PATTERN/p’ filename

For example, 4th line matches the pattern “Security”, so it prints from 3rd line to 4thline.

# sed -n '3,/Security/p' thegeekstuff.txt

3. Hardware


Sed Pattern Format 4: /PATTERN/,$

It prints from the line matches the given pattern to end of file.

# sed -n ‘/PATTERN/,$p’ filename

# sed -n '/Website/,$p' thegeekstuff.txt

8. Website Design



Sed Pattern Format 5: /PATTERN/,+N

It prints the lines which matches the pattern and next N lines following the matchedline.

Página 5 de 42



# sed -n ‘/PATTERN/,+Np’ filename

For example, following prints the 5th line which matches the pattern /Storage/ and nexttwo lines following /Storage/.

# sed -n '/Storage/,+2p' thegeekstuff.txt5. Storage



available)

Sed Pattern Format 6: /PATTERN/,/PATTERN/

Prints the section of file between two regular expression (including the matched line ).

# sed -n ‘/P1/,/P2/p’ filename

For example, 5th line matches “Storage” and 8th line matches “Design”, so it prints 5thto 8th.

# sed -n '/Storage/,/Design/p' thegeekstuff.txt

5. Storage



available)

8. Website Design

Página 6 de 42



Unix Sed Tutorial: Delete File Lines

Using Address and Patterns

In the previous sed tutorial we discussed about Unix sed command basics and

printing lines from a file using sed address and patterns.

In this article, let us review how to delete lines from a file using address and patternswith 8 awesome examples.

• “p” command prints the buffer (remember to use -n option with “p”)• “d” command is just opposite, its for deletion. ‘d’ will delete the pattern space

buffer and immediately starts the next cycle.

Syntax:

# sed 'ADDRESS'd filename

# sed /PATTERN/d filename

Syntax for ADDRESSES and PATTERNS given in the printing is applicable for deletion also, except -n option. (-n only to suppress printing pattern buffer, can be usedwith “p” command )

Let us first creates thegeekstuff.txt file that will be used in all the examples mentioned below.




3. Hardware


5. Storage



available)

8. Website Design9. Software Development


Página 7 de 42



1. Delete Nth Line

‘Nd’ deletes the Nth line and prints the other lines.

sed ‘Nd’ filename

As per sed methodology,

• It reads the first line and places in its pattern buffer.• Check whether supplied command is true for this line, if true, deletes pattern

space buffer and starts next cycle. i.e Read next line.• If supplied command doesnt true, as its normal behaviour it prints the content of

the pattern space buffer.

For example, 3d deletes 3rd line and prints other lines as shown below.

$ sed 3d thegeekstuff.txt




5. Storage



available)

8. Website Design



2. Delete Starting from 3rd line and every 2nd line from there.

$ sed '3~2d' thegeekstuff.txt





8. Website Design


3. Delete from 4th to 8th line from file.

$ sed '4,8d' thegeekstuff.txt

1. Linux - Sysadmin, Scripting etc.2. Databases - Oracle, mySQL etc.

3. Hardware



4. Delete the last line from input.

$ sed '$d' thegeekstuff.txt



3. Hardware

4. Security (Firewall, Network, Online Security etc)5. Storage


Página 8 de 42




available)

8. Website Design


5. Delete the line which matches the given pattern from input.

For example, the below command deletes the line which matches with “Sysadmin”.

$ sed /Sysadmin/d thegeekstuff.txt


3. Hardware


5. Storage



available)

8. Website Design


6. Deletes the line from which matches the given pattern to end of the file.

$ sed '/Website/,$d' thegeekstuff.txt



3. Hardware


5. Storage



available)

7. Deletes the line from which matches the given pattern and 2lines next

to that.

$ sed '/Storage/,+2d' thegeekstuff.txt



3. Hardware


8. Website Design



8. Delete blank Line from a file using sed

You can also remove blank lines with sed. The following sed example shows how touse sed and remove blank lines.

$ sed '/^$/d' thegeekstuff.txt



3. Hardware


5. Storage



available)

Página 9 de 42



8. Website Design



Página 10 de 42



Unix Sed Tutorial: Find and Replace

Text Inside a File Using RegEx

This article is part of on-going Unix Sed Tutorial series. In previous articles, wediscussed about sed print operation and sed delete operation.

In this article let us review how to use sed substitute command “s”.

The `s’ command is probably the most important in `sed’ and has a lot of differentoptions.

The `s’ command attempts to match the pattern space against the supplied REGEXP; if the match is successful, then that portion of the pattern space which was matched isreplaced with REPLACEMENT.

Syntax:

#sed 'ADDRESSs/REGEXP/REPLACEMENT/FLAGS' filename

#sed 'PATTERNs/REGEXP/REPLACEMENT/FLAGS' filename

• s is substitute command• / is a delimiter• REGEXP is regular expression to match• REPLACEMENT is a value to replace

FLAGS can be any of the following

• g Replace all the instance of REGEXP with REPLACEMENT• n Could be any number,replace nth instance of the REGEXP with

REPLACEMENT.• p If substitution was made, then prints the new pattern space.• i match REGEXP in a case-insensitive manner.• w file If substitution was made, write out the result to the given file.• We can use different delimiters ( one of @ % ; : ) instead of /


Página 11 de 42

http://www.thegeekstuff.com/2009/09/unix-sed-tutorial-printing-file-lines-using-address-and-patterns/

http://www.thegeekstuff.com/2009/09/unix-sed-tutorial-delete-file-lines-using-address-and-patterns/





$ cat thegeekstuff.txt

# Instruction Guides

1. Linux Sysadmin, Linux Scripting etc.



4. Storage in Linux


available)# Additional FAQS

6. Windows- Sysadmin, reboot etc.

Let us review some interesting examples for substitution now.

1. Substitute Word “Linux” to “Linux-Unix” Using sed s//

In the example below, in the output line “1. Linux-Unix Sysadmin, Linux Scripting etc”only first Linux is replaced by Linux-Unix. If no flags are specified the first match of line is replaced.

$ sed 's/Linux/Linux-Unix/' thegeekstuff.txt


1. Linux-Unix Sysadmin, Linux Scripting etc.



4. Storage in Linux-Unix


available)

# Additional FAQS


2. Substitute all Appearances of a Word Using sed s//g

The below sed command replaces all occurrences of Linux to Linux-Unix using globalsubstitution flag “g”.

$ sed 's/Linux/Linux-Unix/g' thegeekstuff.txt


1. Linux-Unix Sysadmin, Linux-Unix Scripting etc.





available)

# Additional FAQS


3. Substitute Only 2nd Occurrence of a Word Using sed s//2

In the example below, in the output line “1. Linux Sysadmin, Linux-Unix Scriptingetc.” only 2nd occurance of Linux is replaced by Linux-Unix.

$ sed 's/Linux/Linux-Unix/2' thegeekstuff.txt


1. Linux Sysadmin, Linux-Unix Scripting etc.

2. Databases - Oracle, mySQL etc.3. Security (Firewall, Network, Online Security etc)

4. Storage in Linux

Página 12 de 42




available)

# Additional FAQS


4. Write Changes to a File and Print the Changes Using sed s//gpw

The example below has substitution with three flags. It substitutes all the occurance of Linux to Linux-Unix and prints the substituted output as well as written the same to thegiven the file.

$ sed -n 's/Linux/Linux-Unix/gpw output' thegeekstuff.txt



$ cat output



5. Substitute Only When the Line Matches with the Pattern Using sed

In this example, if the line matches with the pattern “-”, then it replaces all thecharacters from “-” with the empty.

$ sed '/\-/s/\-.*//g' thegeekstuff.txt



2. Databases


4. Storage in Linux


available)# Additional FAQS

6. Windows

6. Delete Last X Number of Characters From Each Line Using sed

This sed example deletes last 3 characters from each line.

$ sed 's/...$//' thegeekstuff.txt

# Instruction Gui

1. Linux Sysadmin, Linux Scripting e

2. Databases - Oracle, mySQL e

3. Security (Firewall, Network, Online Security e4. Storage in Li


availab

# Additional F

6. Windows- Sysadmin, reboot e

7. Eliminate Comments Using sed

Delete all the comment lines from a file as shown below using sed command.

$ sed -e 's/#.*//' thegeekstuff.txt



Página 13 de 42




4. Storage in Linux


available)


8. Eliminate Comments and Empty Lines Using sed

In this example, there are two commands seperated by ‘;’

• First command replaces the lines starting with the # to the blanklines

• Second command deletes the empty lines.

$ sed -e 's/#.*//;/^$/d' thegeekstuff.txt



3. Security (Firewall, Network, Online Security etc)4. Storage in Linux


available)


9. Convert DOS newlines (CR/LF) to Unix format Using sed

Copy the DOS file to Unix, you could find \r\n in the end of each line.

This example converts the DOS file format to Unix file format using sed command.

$sed 's/.$//' filename

10. Eliminate HTML Tags from file Using sed

In this example, the regular expression given in the sed command matches the html tagsand replaces with the empty.

$ sed -e 's/<[^>]*>//g'

This <b> is </b> an <i>example</i>.

This is an example.

Página 14 de 42



Unix Sed Tutorial: How To Write to a

File Using Sed

This article is part of Unix Sed Tutorial series. In previous articles, we discussed aboutsed print operation , sed delete operation and sed find and replace.

In this article, let us review how to extract part of one file and write it to another fileusing sed.

Sed provides “w” command to write the pattern space data to a new file.

Sed creates or truncates the given filename before reads the first input line and it writesall the matches to a file without closing and re-opening the file.

Syntax:

#sed 'ADDERSSw outputfile' inputfilename

#sed '/PATTERN/w outputfile' inputfilename

Sed reads a line and place it in a pattern buffer and writes the pattern buffer to the givenoutput file according to the supplied commands.





3. Hardware


5. Storage



available)

8. Website Design9. Software Development


Página 15 de 42



http://www.thegeekstuff.com/2009/09/unix-sed-tutorial-replace-text-inside-a-file-using-substitute-command/







Let us review some examples of write command in sed.

1. Write 1st line of the file

In this example, 1 (address) refers the first line of the input and w writes the pattern buffer to the output file “output.txt”

$ sed -n '1w output.txt' thegeekstuff.txt

$ cat output.txt


2. Write first & last line of the file

In this example, 1 and $ refers first and last line respectively.

$ sed -n -e '1w output.txt' -e '$w output.txt' thegeekstuff.txt

$ cat output.txt



3. Write the lines matches with the pattern Storage or Sysadmin

In this example sed command writes the lines which matches the pattern “Storage” or “Sysadmin”.

$ sed -n -e '/Storage/w output.txt' -e '/Sysadmin/w output.txt'

thegeekstuff.txt

$ cat output.txt


5. Storage


4. Write the lines from which the pattern matches to till end of the file

In this example, /Storage/,$ represents line matches from Storage to end of the file.

$ sed -n '/Storage/,$w output.txt' thegeekstuff.txt

$ cat output.txt

5. Storage



available)

8. Website Design



5. Write the lines which matches pattern and next two lines from match

In this example, the send command writes the line matches for “Storage” and two linesnext to that.

$ sed -n '/Storage/,+2w output.txt' thegeekstuff.txt

Página 16 de 42



$ cat output.txt

5. Storage



available)

Página 17 de 42



Unix Sed Tutorial: How To Execute

Multiple Sed Commands

Question: Is it possible for me to combine multiple sed commands? Can I combine twosed commands and execute it as single sed command?

Answer: In our previous articles we learned sed with single commands — printing,deletion, substitute and file write.

In this article let us review how to combine multiple sed commands using option -e asshown below.

Syntax:

#sed -e 'command' -e 'command' filename

Note: -e option is optional for sed with single command. sed will execute the each set of command while processing input from the pattern buffer.





3. Hardware4. Security (Firewall, Network, Online Security etc)

5. Storage



available)

8. Website Design



1.Delete 4th and 2nd line from the input

This sed example deletes 4th and 2nd line from the file thegeekstuff.txt. Using “-e”option, you can give any number of commands with sed.

$ sed -e '4d' -e '2d' thegeekstuff.txt


3. Hardware

5. Storage



available)

8. Website Design



2. Print the lines which matches the pattern1 and lines matches pattern2

Página 18 de 42




http://www.thegeekstuff.com/2009/10/unix-sed-tutorial-how-to-write-to-a-file-using-sed/







This sed example prints all lines that matches either the pattern “Storage” or “Software”.

$ sed -n -e '/Software/p' -e '/Storage/p' thegeekstuff.txt

5. Storage


3. Delete the first,last and all the blank lines from input

This sed example deletes the first line, last line and all the blank lines from input file.

$ sed -e '1d' -e '$d' -e '/^$/d' thegeekstuff.txt


3. Hardware


5. Storage



available)8. Website Design


Página 19 de 42



Unix Sed Tutorial: Advanced Sed

Substitution Examples

This article is part of the on-going Unix Sed Tips and Tricks series.

In our previous sed articles we learned — sed printing, sed deletion, sed substitute , sedfile write, and sed multiple commands.

In this article, let us review some interesting workarounds with the “s” substitutecommand in sed with several practical examples.

I. Sed Substitution Delimiter

As we discussed in our previous post, we can use the different delimiters such as @ %| ; : in sed substitute command.

Let us first create path.txt file that will be used in all the examples mentioned below.

$ cat path.txt

/usr/kbos/bin:/usr/local/bin:/usr/jbin:/usr/bin:/usr/sas/bin

/usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin:/opt/omni/bin:

/opt/omni/lbin:/opt/omni/sbin:/root/bin

Example 1 – sed @ delimiter: Substitute /opt/omni/lbin to /opt/tools/bin

When you substitute a path name which has ‘/’, you can use @ as a delimiter instead of ‘/’. In the sed example below, in the last line of the input file, /opt/omni/lbin waschanged to /opt/tools/bin.

$ sed 's@/opt/omni/lbin@/opt/tools/bin@g' path.txt

/usr/kbos/bin:/usr/local/bin:/usr/jbin/:/usr/bin:/usr/sas/bin


/opt/tools/bin:/opt/omni/sbin:/root/bin

Página 20 de 42

http://www.thegeekstuff.com/tag/sed-tips-and-tricks/








http://www.thegeekstuff.com/2009/10/unix-sed-tutorial-how-to-execute-multiple-sed-commands/










Example 2 – sed / delimiter: Substitute /opt/omni/lbin to /opt/tools/bin

When you should use ‘/’ in path name related substitution, you have to escape ‘/’ in thesubstitution data as shown below. In this sed example, the delimiter ‘/’ was escaped inthe REGEXP and REPLACEMENT part.

$ sed 's/\/opt\/omni\/lbin/\/opt\/tools\/bin/g' path.txt

/usr/kbos/bin:/usr/local/bin:/usr/jbin/:/usr/bin:/usr/sas/bin


/opt/tools/bin:/opt/omni/sbin:/root/bin

II. Sed ‘&’ Get Matched String

The precise part of an input line on which the Regular Expression matches isrepresented by &, which can then be used in the replacement part.

Example 1 – sed & Usage: Substitute /usr/bin/ to /usr/bin/local$ sed 's@/usr/bin@&/local@g' path.txt

/usr/kbos/bin:/usr/local/bin:/usr/jbin/: /usr/bin/local:/usr/sas/bin/usr/local/sbin:/sbin:/bin/:/usr/sbin: /usr/bin/local:/opt/omni/bin:/opt/omni/lbin:/opt/omni/sbin:/root/bin

In the above example ‘&’ in the replacement part will replace with /usr/bin which ismatched pattern and add it with /local. So in the output all the occurrance of /usr/binwill be replaced with /usr/bin/local

Example 2 – sed & Usage: Match the whole line

& replaces whatever matches with the given REGEXP.

$ sed 's@^.*$@<<<&>>>@g' path.txt

<<</usr/kbos/bin:/usr/local/bin:/usr/jbin/:/usr/bin:/usr/sas/bin>>>

<<</usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin:/opt/omni/bin:>>>

<<</opt/omni/lbin:/opt/omni/sbin:/root/bin>>>

In the above example regexp has “^.*$” which matches the whole line. Replacement part <<<&>>> writes the whole line with <<< and >>> in the beginning and end of theline respectively.

III. Grouping and Back-references in Sed

Grouping can be used in sed like normal regular expression. A group is opened with “$” and closed with “$”.Grouping can be used in combination with back-referencing.

Back-reference is the re-use of a part of a Regular Expression selected by grouping.Back-references in sed can be used in both a Regular Expression and in the replacement

part of the substitute command.

Example 1: Get only the first path in each line

$ sed 's/$\/[^:]*$.*/\1/g' path.txt

/usr/kbos/bin

Página 21 de 42



/usr/local/sbin

/opt/omni/lbin

In the above example, $\/[^:]*$ matches the path available before first : comes. \1replaces the first matched group.

Example 2: Multigrouping

In the file path.txt change the order of field in the last line of the file.

$ sed '$s@$[^:]*$:$[^:]*$:$[^:]*$@\3:\2:\1@g' path.txt

/usr/kbos/bin:/usr/local/bin:/usr/jbin:/usr/bin:/usr/sas/bin

/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/omni/bin:

/root/bin:/opt/omni/sbin:/opt/omni/lbin

In the above command $ specifies substitution to happen only for the last line.Outputshows that the order of the path values in the last line has been reversed.

Example 3: Get the list of usernames in /etc/passwd file

This sed example displays only the first field from the /etc/passwd file.

$sed 's/$[^:]*$.*/\1/' /etc/passwd

root

bin

daemon

adm

lp

sync

shutdown

Example 4: Parenthesize first character of each word

This sed example prints the first character of every word in paranthesis.

$ echo "Welcome To The Geek Stuff" | sed 's/$\b[A-Z]$/$\1$/g'

(W)elcome (T)o (T)he (G)eek (S)tuff

Example 5: Commify the simple number.

Let us create file called numbers which has list of numbers. The below sed commandexample is used to commify the numbers till thousands.

$ cat numbers

1234

12121

3434

123

$sed 's/$^\|[^0-9.]$$[0-9]\+$$[0-9]\{3\}$/\1\2,\3/g' numbers

1,234

12,121

3,434

123

Página 22 de 42



Unix Sed Tutorial: Append, Insert,

Replace, and Count File Lines

This article is part of the on going Unix sed command tutorial series. In our previousarticles we learned sed with single commands — printing, deletion, substitute and filewrite.

Sed provides lot of commands to perform number of operations with the lines in a file.

In this article let us review how to append, insert, replace a line in a file and how to getline numbers of a file.

• Append Lines• Insert Lines• Replace Lines• Count Lines


$cat thegeekstuff.txt

Linux Sysadmin

Databases - Oracle, mySQL etc.

Security (Firewall, Network, Online Security etc)

Storage in Linux

Productivity (Too many technologies to explore, not much time

available)

Windows- Sysadmin, reboot etc.

Append Lines Using Sed Command

Sed provides the command “a” which appends a line after every line with the address or pattern.

Syntax:

#sed 'ADDRESS a\

Página 23 de 42







http://www.thegeekstuff.com/2009/11/unix-sed-tutorial-append-insert-replace-and-count-file-lines/#append_lines

http://www.thegeekstuff.com/2009/11/unix-sed-tutorial-append-insert-replace-and-count-file-lines/#insert_lines

http://www.thegeekstuff.com/2009/11/unix-sed-tutorial-append-insert-replace-and-count-file-lines/#replace_lines

http://www.thegeekstuff.com/2009/11/unix-sed-tutorial-append-insert-replace-and-count-file-lines/#count_lines






http://www.thegeekstuff.com/2009/11/unix-sed-tutorial-append-insert-replace-and-count-file-lines/#append_lines

http://www.thegeekstuff.com/2009/11/unix-sed-tutorial-append-insert-replace-and-count-file-lines/#insert_lines

http://www.thegeekstuff.com/2009/11/unix-sed-tutorial-append-insert-replace-and-count-file-lines/#replace_lines

http://www.thegeekstuff.com/2009/11/unix-sed-tutorial-append-insert-replace-and-count-file-lines/#count_lines



Line which you want to append' filename

#sed '/PATTERN/ a\

Line which you want to append' filename

Sed Append Example 1. Add a line after the 3rd line of the file.

Add the line “Cool gadgets and websites” after the 3rd line. sed “a” command insertsthe line after match.

$ sed '3 a\> Cool gadgets and websites' thegeekstuff.txt

Linux Sysadmin



Cool gadgets and websitesStorage in Linux


available)


Sed Append Example 2. Append a line after every line matching the

pattern

The below sed command will add the line “Linux Scripting” after every line thatmatches the pattern “Sysadmin”.

$ sed '/Sysadmin/a \> Linux Scripting' thegeekstuff.txt

Linux Sysadmin

Linux ScriptingDatabases - Oracle, mySQL etc.


Storage in Linux


available)


Linux Scripting

Sed Append Example 3. Append a line at the end of the file

The following example, appends the line “Website Design” at the end of the file.

$ sed '$ a\> Website Design' thegeekstuff.txt

Linux Sysadmin



Storage in Linux


available)


Website Design

Página 24 de 42



Insert Lines Using Sed Command

Sed command “i” is used to insert a line before every line with the range or pattern.

Syntax:

#sed 'ADDRESS i\

Line which you want to insert' filename

#sed '/PATTERN/ i\

Line which you want to insert' filename

Sed Insert Example 1. Add a line before the 4th line of the line.

Add a line “Cool gadgets and websites” before 4th line. “a” command inserts the lineafter match whereas “i” inserts before match.

$ sed '4 i\> Cool gadgets and websites' thegeekstuff.txt

Linux Sysadmin



Cool gadgets and websitesStorage in Linux


available)


Sed Insert Example 2. Insert a line before every line with the pattern

The below sed command will add a line “Linux Scripting” before every line thatmatches with the pattern called ‘Sysadmin”.

$ sed '/Sysadmin/i \> Linux Scripting' thegeekstuff.txt

Linux ScriptingLinux Sysadmin



Storage in Linux

Productivity (Too many technologies to explore, not much timeavailable)

Linux ScriptingWindows- Sysadmin, reboot etc.

Sed Insert Example 3. Insert a line before the last line of the file.

Append a line “Website Design” before the last line of the file.

$ sed '$ i\> Website Design' thegeekstuff.txt

Linux Sysadmin



Página 25 de 42



Storage in Linux


available)

Website DesignWindows- Sysadmin, reboot etc.

Replace Lines Using Sed Command

“c” command in sed used to replace every line matches with the pattern or ranges withthe new given line.

Syntax:

#sed 'ADDRESS c\

new line' filename

#sed '/PATTERN/ c\

new line' filename

Sed Replace Example 1. Replace a first line of the file

The below command replaces the first line of the file with the “The Geek Stuff”.

$ sed '1 c\> The Geek Stuff' thegeekstuff.txt

The Geek StuffDatabases - Oracle, mySQL etc.


Storage in Linux


available)


Sed Replace Example 2. Replace a line which matches the pattern

Replace everyline which has a pattern “Linux Sysadmin” to “Linux Sysadmin – Scripting”.

$ sed '/Linux Sysadmin/c \> Linux Sysadmin - Scripting' thegeekstuff.txt

Linux Sysadmin - ScriptingDatabases - Oracle, mySQL etc.


Storage in Linux


available)


Sed Replace Example 3. Replace the last line of the file

Sed command given below replaces the last line of the file with “Last Line of the file”.

$ sed '$ c\> Last line of the file' thegeekstuff.txt

Linux Sysadmin

Página 26 de 42





Storage in Linux


available)

Last line of the file

Print Line Numbers Using Sed Command

“=” is a command in sed to print the current line number to the standard output.

Syntax:

#sed '=' filename

The above send command syntax prints line number in the first line and the original linefrom the file in the next line .

sed ‘=’ command accepts only one address, so if you want to print line number for arange of lines, you must use the curly braces.

Syntax:

# sed -n '/PATTERN/,/PATTERN/ {

=

p

}' filename

Sed Line Number Example 1. Find the line number which contains the

pattern

The below sed command prints the line number for which matches with the pattern“Databases”

$ sed -n '/Databases/=' thegeekstuff.txt

2

Sed Line Number Example 2. Printing Range of line numbers

Print the line numbers for the lines matches from the pattern “Oracle” to “Productivity”.

$ sed -n '/Oracle/,/Productivity/{> => p> }' thegeekstuff.txt

2


3


4

Storage in Linux

5Productivity (Too many technologies to explore, not much time

available)

Página 27 de 42



Sed Line Number Example 3. Print the total number of lines in a file

Line number of the last line of the file will be the total lines in a file. Pattern $ specifiesthe last line of the file.

$ sed -n '$=' thegeekstuff.txt

6

Página 28 de 42



Unix Sed Tutorial: Multi-Line File

Operation with 6 Practical Examples

As part of our on going UNIX sed tutorial series earlier we covered the printing,deletion, substitution, file write, file manipulation commands etc., with the single line inthe pattern space.

In this article let us review how to do the multi-line operation in Sed.

Do you remember the Sed working methodology which we learned in our first sedtutorial ?. In that article we explained that Sed reads line by line, removes any trailing

new lines, places a line in a pattern space buffer, process as per the given commandsand prints the pattern space.

In case, if you want to delete all the newlines in a file, you cannot use the followingmethod. Because newline is already removed and placed in the pattern space.

$ sed 's/\n//' filename or $sed 's/\n/ENDOFLINE\n/' filename

For situations like this sed multi-line is appropriate. Sed provides the command “N” for Multi-Line Operations.

N command reads a next line from the input, Append next line to pattern space. Nextline is separated from the original pattern space by a newline character.



Linux Sysadmin




Storage in LinuxWebsite Design

Website Design

Página 29 de 42










$

Note: There are two consecutive blank lines in the above input. ( 5th and 6th line ).

Sed Example 1. Join Two Consecutive Lines

$ sed -e '{ Ns/\n/ @ /}' thegeekstuff.txt

Linux Sysadmin @ Databases - Oracle, mySQL etc.

Databases - Oracle, mySQL etc. @ Security (Firewall, Network, Online

Security etc)

@

Storage in Linux @ Website Design

Website Design @ Windows- Sysadmin, reboot etc.

In the above example,

• The curly braces “{” and “}” used to group the commands. The curlybraces and sed commands must be on the seperate lines.

• Sed reads the first line and place it in the pattern space, N commandreads the next line and appends with the pattern space i.e first lineseperated by newline. So now pattern space will havefirstline\nsecondline.

• Next substitution of \n to space@space and it prints the pattern spacecontent as its sed default behaviour. So consecutive lines are joinedand delimited by ” @ “

Sed Example 2. Number each non-blank line of a file

As mentioned in our previous article, = is a command to get a line number of a file.

$ sed '/./=' thegeekstuff.txt | sed 'N; s/\n/ /'1 Linux Sysadmin

2 Databases - Oracle, mySQL etc.

3 Databases - Oracle, mySQL etc.

4 Security (Firewall, Network, Online Security etc)

7 Storage in Linux

8 Website Design9 Website Design

10 Windows- Sysadmin, reboot etc.

• The first sed command prints the line number and prints the originalline in a next line if it is not a blank.( Execute it and see the output of the first sed command ).

• Next sed command is just appends pair of lines.

Sed Example 3. Delete Two Consecutive Blank Lines from Input

$ sed '/^$/{ N/^\n$/d }' thegeekstuff.txt

Página 30 de 42



Linux Sysadmin




Storage in Linux

Website Design

Website Design


If the line is blank, read and appends the next line, /^\n$/ represents, two lines areempty,\n is added by N command. Then just delete the pattern space and start the nextcycle using command ‘d’.

Sed Example 4. Delete Last 2 Lines of a file

Before viewing this example you must aware of two interesting sed command.

1. P – which prints the first line of a pattern space. (till first \n).

2. D – Delete first line from the pattern space. Control then passes to thetop of the script.

$ sed 'N;$!P;$!D;$d' thegeekstuff.txtLinux Sysadmin




Storage in Linux

Website Design

1. Reads the first line and place it in the pattern space.2. N command reads the next line and append to pattern space

seperated by newline. (Now firstline\nsecond line)3. If it not reaches the last line($), print the first line and delete the first

line alone from the pattern space. Then cycle starts again.4. Like this it joins 2nd\n3rd lines, 3rd\n4th lines and goes on.5. Atlast when it has 9th\n10th line in a pattern space, it reaches $ so it

just deletes the pattern space. ($!P and $!D wont print and delete if itis $).

Sed Example 5. Print Last 2 Lines of a file

$ sed '$!N;$!D' thegeekstuff.txtWebsite Design


The above joins and deletes each line until last two lines are there in the pattern space.When it reaches $, prints the pattern space which will have only last two lines.

Sed Example 6. Delete Duplicate, Consecutive Lines from a file

The below command checks each line joined with the next line, check if both are same

then it doesn’t the print pattern space(!P), just delete the first line from the patternspace. So only one line will be remaining in the pattern space.

$ sed '$!N; /^$.*$\n\1$/!P; D' thegeekstuff.txt

Página 31 de 42



Linux Sysadmin



Storage in Linux

Website Design


Página 32 de 42



Unix Sed Tutorial: 6 Examples for Sed

Branching Operation

Like any other programming language, sed also provides special branching commandsto control the flow of the program.

In this article, let us review following two types of Sed branching.

1. Sed Unconditional Branch2. Sed Conditional Branch

Sed Unconditional Branch Syntax:

$ sed ':label command(s) b label'

• :label - specification of label.• commands - Any sed command(s)• label - Any Name for the label• b label – jumps to the label with out checking any conditions. If label is not

specified, then jumps to the end of the script.

Sed Conditional Branch Syntax:

$ sed ':label command(s) t label'

• :label - specification of label.• commands - Any sed command(s)• label - Any Name for the label• t label – jumps to the label only if the last substitute command modified the

pattern space. If label is not specified, then jumps to the end of the script.

Create a sample test file

Let us first create thegeekstuff.txt file that will be used in the examples mentioned below.

Página 33 de 42




Linux

Administration

Scripting

Tips and Tricks

Windows

Administration

DatabaseAdministration of Oracle

Administration of Mysql

Security

Network

Online\

Security

Productivity

Google Search\

Tips

"Web Based Time Tracking,

Web Based Todo list and

Reduce Key Stores etc"

$

I. Sed Examples for Unconditional Branch

Sed Example 1. Replace the first occurrence of a pattern in a whole file

In the file thegeekstuff.txt replace the first occurrence of “Administration” to“Supervision”.

$ sed '/Administration/{

s/Administration/Supervision/

:loopn

b loop

}' thegeekstuff.txt

Linux

SupervisionScripting

Tips and Tricks

Windows

Administration

Database

Administration of Oracle


SecurityNetwork

Online\

Security

Productivity

Google Search\

Tips




1. In the above sed command, it just read line by line and prints the pattern space

till Administration occurs.2. Once Administration occurs, substitute Administration to Supervision (only

single occurrence, note that no ‘g’ flag in substitution).

Página 34 de 42



3. Once the first occurrence has been replaced, just read the remaining file contentand print.

o “n” is a sed command which prints the pattern space and overwrite it

with next line.o Used “loop” as a label. “n” prints the current line and overwrite pattern

space with the next line. b loop jumps to the :loop again. So this loop prints the remaining content of thegeekstuff.txt.

Sed Example 2. Remove the data between pattern ” ” in a whole file

In our example file there are three lines between “”.

sed -e ':loop

$!{

N

/\n$/!b loop

}

s/\"[^\"]*\"//g' thegeekstuff.txtLinux

Administration

Scripting

Tips and Tricks

Windows

Administration

Database



Security

Network

Online\

SecurityProductivity

Google Search\

Tips

$

1. Above command keep appends all the lines of a file till end of file occurs.o $! – If its not a end of file.

o N – Append the next line with the pattern space delimited by \n

o /\n$/!b loop – If this is not the last line of the file jump to the loop again.

2. Now all the lines will be available in pattern space delimited by newline.Substitute all the occurrence of data between ” with the empty.

Sed Example 3. Remove the HTML tags of a file

Let us say, I have a file with the following html content

$ cat index.html

<html><body>

<table

border=2><tr><td valign=top

align=right>1.</td>

<td>Line 1 Column 2</

td>

</table>

</body></html>

Página 35 de 42



The following sed command removes all the html tags from the given file

$ sed '/</{

:loop

s/<[^<]*>//g

/</{

N

b loop

}

}' index.html

1.

Line 1 Column 2

1. Each time find a line contains ‘<’, first remove all HTML tags of that line.2. If now the pattern space contains ‘<’, this implies a multi-line tag. Now repeat

the following loop:o Join next line

o Remove all HTML tags until no single ‘<’ exists3. When no ‘<’ exists in the pattern space, we print it out and start a new cycle.

II. Sed Examples for Conditional Branch

Sed Example 4. If a line ends with a backslash append the next line to it.

Our example file has two lines ends with backslash, now we have to append its next lineto it.

$ sed '

:loop

/\\$/N

s/\\\n */ /

t loop' thegeekstuff.txt

Linux

Administration

Scripting

Tips and Tricks

Windows

Administration

Database



SecurityNetwork

Online Security

Productivity

Google Search Tips




1. Check if the line ends with the backslash (/\\$/), if yes, read and append the nextline to pattern space, and substitute the \ at the end of the line and number of spaces followed by that, with the single space.

2. If the substitution is success repeat the above step. The branch will be executedonly if substitution is success.

3. Conditional branch mostly used for recursive patterns.

Página 36 de 42



Sed Example 5. Commify a numeric strings.

sed '

:loop

s/$.*[0-9]$$[0-9]\{3\}$/\1,\2/

t loop'

12342342342343434

12,342,342,342,343,434

1. Group the digits into two groups.2. The first group is all the digits up to last three digits. The last three digits gets

captures in the 2nd group.3. Then the two matching groups get separated by a comma. Then the same rules

get applied to the line again and again until all the numbers have been groupedin groups of three.

4. For example, in the first iteration it will be 12342342342343,4345. In the next iteration 12342342342,343,434 and goes on till there are less than

three digits.

Sed Example 6. Formatting : Replace every leading space of a line with

‘+’

$ sed '

s/^ */&\n/

:loop

s/^\n//;s/ \n/\n+/

t loop' test

Linux

++++++++Administration

++++++++Scripting++++++++++++++++Tips and Tricks

Windows

++++++++Administration

Database

++++++++Administration of Oracle

++++++++Administration of Mysql

Security

++++++++Network

+++++++++++++++++Online\

++++++++Security

Productivity

++++++++Google Search\

++++++++Tips++++++++"Web Based Time Tracking,

++++++++Web Based Todo list and

++++++++Reduce Key Stores etc"

1. Seperate all the leading spaces and other characters of a line with a newlinecharacter.

2. Now replace space and newline with newline and +. So from right to left spacewill be replaced by + and newline will be moved left for one character.

3. At last in the beginning of the line \n will be there, so remove that new line.

Página 37 de 42



Unix Sed Tutorial : 7 Examples for Sed

Hold and Pattern Buffer Operations

In our previous sed articles we learned — sed printing, sed deletion, sed substitute , sedfile write, sed multiple commands, sed multi-line operation, and sed manipulate filelines.

In our first part of sed tutorial we learned that sed has two buffers — sed pattern

buffer and sed hold buffer. So far we have talked only about pattern buffer.

In this article let us review how to use sed hold and pattern buffer using 7 practical sed

examples.

As its name implies, sed hold buffer is used to save all or part of the sed pattern space

for subsequent retrieval. The contents of the pattern space can be copied to the holdspace, then back again. No operations are performed directly on the hold space. sed

provides a set of hold and get functions to handle these movements.

Sed h function

The h (hold) function copies the contents of the pattern space into a holding area (alsocalled as sed hold space), destroying any previous contents of the holding area.

Sed H function

The H function appends the contents of the pattern space to the contents of the holdingarea. The former and new contents are separated by a newline.

Sed g function

The g function copies the contents of the holding area into the pattern space, destroyingthe previous contents of the pattern space.

Página 38 de 42








http://www.thegeekstuff.com/2009/11/unix-sed-tutorial-multi-line-file-operation-with-6-practical-examples/


http://www.thegeekstuff.com/2009/11/unix-sed-tutorial-append-insert-replace-and-count-file-lines/















Sed G function

The G function appends the contents of the holding area to the contents of the patternspace. The former and new contents are separated by a newline. The maximum number of addresses is two.

Sed x function

The exchange function interchanges the contents of the pattern space and the holdingarea. The maximum number of addresses is two.

Now let us see some examples to learn about the above commands.

Let us first create thegeekstuff.txt file that will be used in the examples mentioned below.


#Linux

Administration

Scripting

Tips and Tricks

#Windows

Administration

#Database

Mysql

Oracle

Queries

Procedures

1. Double Space a File Content Using Sed Command

$sed 'G' thegeekstuff.txt

#Linux

Administration

Scripting

Tips and Tricks

#Windows

Administration

#Database

Mysql

Oracle

Queries

Procedures

$

Página 39 de 42



In this example,

1. Sed reads a line and places it in the pattern buffer.2. G command appends the hold buffer to the pattern buffer separated

by \n. so one newline will be appended with the pattern spacecontent.

3. Similarly, If you want to triple space a file, append hold buffer contentto the pattern buffer twice. (G;G)

2. Print File Content in Reverse Order Using Sed Command

Print the lines of a file in reverse order (similar to tac command that we discussedearlier).

$sed -n '1!G;h;$p' thegeekstuff.txt

Procedures

Queries

Oracle

Mysql

#Database

Administration

#Windows

Tips and Tricks

Scripting

Administration

#Linux

In this example,

1. First line will be placed into the hold space as it is.2. From the 2nd line onwards, just append the hold space content with

the pattern space. (Remember 2nd line is in pattern space, and 1stline is in hold space).

3. Now 1st and 2nd line got reversed and move this to the hold space.4. Repeat the above steps till last line.5. Once the last line is reached, just append the hold space content with

the pattern space and print the pattern space.

3. Print a Paragraph (Only if it contains given pattern) Using Sed

Command

In thegeekstuff.txt print paragraph only if it contains the pattern “Administration”.

$ sed -e '/./{H;$!d;}' -e 'x;/Administration/!d' thegeekstuff.txt

Linux

Administration

Scripting

Tips and Tricks

Windows

Administration

In this example,

Página 40 de 42

http://www.thegeekstuff.com/2009/10/file-manipulation-examples-using-tac-rev-paste-and-join-unix-commands/





1. Till the empty line comes, keep appending the non empty lines intothe hold space

2. When empty line comes i.e paragraph ends, exchange the databetween pattern and hold space. So that whole paragraph will beavailable in pattern space.Check if pattern “Administration” is available, if yes don’t delete it i.e

print the pattern space

4. Print the line immediately before a pattern match using Sed Command

Print only the line immediately before,the pattern “Mysql”.

$ sed -n '/Mysql/{g;1!p;};h' thegeekstuff.txt

#Database

In this example,

1. For each cycle, place the line into hold buffer, if it doesn’t match withthe pattern “Mysql”.2. If the line matches with the pattern, get the data from the hold

space(previous line) using g command and print it.3. In case, if the first line matches with the pattern “Mysql”,anyway hold

space will be empty.(There is no previous line to the first line).So firstline should not get printed(1!p)

5. Delete the last line of each paragraph using Sed Command

$ sed -n -e '/^$/{x;d}' -e '/./x;p' thegeekstuff.txt

#Linux

Administration

Scripting

#Windows

#Database

Mysql

Oracle

Queries

In this example,

1. If the line is not empty,then exchange the line between pattern andhold space. So first line will be placed in the hold space.

2. When next non empty line comes, exchange the pattern space andhold space, and print the pattern space. i.e first non empty line will beprinted and 2nd line goes to hold. And in next cycle, 2nd non emptyline is printed when 3rd line goes to hold and goes on like this.

3. When empty line comes (previous line to the empty line will beavailable in hold buffer) just exchange pattern and hold space, anddelete the line (last line of the paragraph) and start the next cycle.

6. For each line, append the previous line to the end of it using SedCommand

$ sed 'H;x;s/^$.*$\n$.*$/\2\1/' thegeekstuff.txt

Página 41 de 42



#Linux

Administration#Linux

Scripting Administration

Tips and Tricks Scripting

Tips and Tricks

#Windows

Administration#Windows

Administration#Database

Mysql#Database

Oracle Mysql

Queries Oracle

Procedures Queries

In this example,

1. Place the first line in Hold buffer.2. When the second line comes, append to Hold space (first line)3. Then exchange pattern and hold buffer. So now pattern space will

have first and second line separated by \n, Hold space will have onlysecond line.

4. So interchange the lines in the pattern space.5. The above steps happens till the end of the file

7. Prepend tag of every block to every line of that block

$ sed '

/^#/{

h

d

}

Gs/^$.*$\n#$.*$/\2 \1/' thegeekstuff.txt

Linux Administration

Linux Scripting

Linux Tips and Tricks

Linux

Windows Administration

Windows

Database Mysql

Database Oracle

Database Queries

Database Procedures

In this example,

1. When the first line of a block is met (beginning with #)o keep that line to the Hold Space via command `h’o Then delete using ‘d’ to start another cycle.

2. For the rest lines of a block, Command `G’ appends the tag line fromthe Hold Space and substitute command interchanges tag and linesproperly.

Date post:	10-Apr-2018
Category:	Documents
Upload:	pubirz
View:	228 times
Download:	0 times

Sed Command Tutorial

Documents