While the word manipulation functionality of sed is great, it also allows us to manipulate whole lines. For example, we can delete certain lines, by number:
reader@ubuntu:~/scripts/chapter_10$ echo -e "Hi,\nthis is \nPatrick"
Hi,
this is
Patrick
reader@ubuntu:~/scripts/chapter_10$ echo -e "Hi,\nthis is \nPatrick" | sed 'd'
reader@ubuntu:~/scripts/chapter_10$ echo -e "Hi,\nthis is \nPatrick" | sed '1d'
this is
Patrick
By using echo -e combined with the newline character (\n), we can create multi-line statements. -e is explained on the man echo page as enable interpretation of backslash escapes. By piping this multi-line output into sed, we can use the delete functionality, which is a script that simply uses the character d.
If we prefix this with a line number, for example 1d, the first line is deleted. If we do not, all lines are deleted, which results in no output for us.
Another, often more interesting, possibility is deleting lines that contain a certain word:
reader@ubuntu:~/scripts/chapter_10$ echo -e "Hi,\nthis is \nPatrick" | sed '/Patrick/d'
Hi,
this is
reader@ubuntu:~/scripts/chapter_10$ echo -e "Hi,\nthis is \nPatrick" | sed '/patrick/d'
Hi,
this is
Patrick
In the same way as we used a script with word matching for the search-replace function of sed, so can we also delete a whole line if a word is present. As you can see from the preceding example, this is case-sensitive. Luckily, there's always a solution if we want to do this in a case-insensitive manner. In grep, this would be the -i flag, but for sed this -i is already reserved for --in-place functionality.
How do we do it then? By using our old friends regular expressions, of course! See the following example:
reader@ubuntu:~/scripts/chapter_10$ echo -e "Hi,\nthis is \nPatrick" | sed '/[Pp]atrick/d'
Hi,
this is
reader@ubuntu:~/scripts/chapter_10$ echo -e "Hi,\nthis is \nPatrick" | sed '/.atrick/d'
Hi,
this is
While it's not as graceful as the functionality provided by grep, it does get the job done in most situations. It should at least make you aware of the fact that using regular expressions with sed makes the whole thing much more flexible and much more powerful.
As with most things, with added flexibility and power comes added complexity. However, we hope that with this gentle introduction to regular expressions and sed, the combination of both does not feel unmanageably complex.
Instead of deleting lines from a file or stream, you might have a better use case for just showing a few files. There is a small issue with this, however: by default, sed prints all lines it processes. If you give sed the instruction to print a line (with the p script), it will print that line two times—once for the match on the script, and the other time for the default print.
This looks something like this:
reader@ubuntu:~/scripts/chapter_10$ cat error.txt
Process started.
Running normally.
ERROR: TCP socket broken.
ERROR: Cannot connect to database.
Exiting process.
reader@ubuntu:~/scripts/chapter_10$ sed '/ERROR/p' error.txt
Process started.
Running normally.
ERROR: TCP socket broken.
ERROR: TCP socket broken.
ERROR: Cannot connect to database.
ERROR: Cannot connect to database.
Exiting process.
The syntax for the print and delete scripts is similar: '/word/d' and '/word/p'. To suppress the default behavior of sed, which prints all lines, add a -n (also known as --quiet or --silent):
reader@ubuntu:~/scripts/chapter_10$ sed -n '/ERROR/p' error.txt
ERROR: TCP socket broken.
ERROR: Cannot connect to database.
There is one final use case for sed that we would like to highlight: you have a file or stream, and you need to delete not a whole line, but only some words in those lines. With grep, this cannot be (easily) achieved. sed has a very simple way of doing this, however.
What makes searching and replacing different to just plain deleting a word? Just the replacement pattern!
See the following example:
reader@ubuntu:~/scripts/chapter_10$ cat search.txt
How much stone would a stonechuck chuck
if a stonechuck could chuck stone?
reader@ubuntu:~/scripts/chapter_10$ sed 's/stone//g' search.txt
How much would a chuck chuck
if a chuck could chuck ?
By replacing the word stone with nothing (as that is exactly what is present between the second and third backslash in the sed script), we delete the word stone completely. In this example, however, you can see a common problem you will undoubtedly run across: extra whitespace after deleting a word.
This leads us to one more trick for sed, which helps you out in this regard:
reader@ubuntu:~/scripts/chapter_10$ sed -e 's/stone //g' -e 's/stone//g' search.txt
How much would a chuck chuck
if a chuck could chuck ?
By supplying -e, followed by a sed script, you can make sed run multiple scripts (in order!) over your stream. By default, sed expects at least one script, which is why you do not need to supply the -e if you're only processing a single script. For more scripts than this, you'll need to add a -e before each.