The here document and the << operator

It is a special type of block of text or code. It is also a special form of I/O redirection. It can be used to feed the command list to an interactive program.

The syntax of the usage of the here document or the << operator is as follows:

command << HERE
text1 …..
text 2….

HERE

This tells the shell that the command should receive the data from a current source, such as the here document, until the pattern is received. In this case, the pattern is HERE. We have used the delimiter as HERE. We can use any other word as the delimiter, such as quite or finish. All the text reads up to a pattern; or the HERE text is used as an input for command. The text or file received by the command is called as the Here document:

$ cat << QUIT
> first input line
> ...
> last input line
> QUIT

The block of text inserted after and before QUIT will be treated as a file. This content will be given as input to the command cat. We will also see more examples with various other commands, such as sort, wc, and similar.

Let's write the script here_01.sh:

#!/bin/bash
cat << quit
  Command is $0
  First Argument is $1
  Second Argument is $2
Quit
Save the file, give execute permission and run the script as follows:
$ chmod here_01.sh
$./here_01.sh Monday Tuesday

Output:

  Command is here_01.sh
  First Argument is Monday
  Second Argument is Tuesday

The text block created in the preceding script between the quit words is called as the here document. We can treat this here document as a separate document. It can also be treated as multiple line input redirected to a Shell script.

Let's learn a few more sample programs.

The here operator with the sort command

Let's write script for using the sort command along with the here document:

Write the script here_02.sh as follows:

#!/bin/bash
sort << EOF
> cherry
> mango
> apple
> banana
> EOF

Save the file, give the permission to execute, and run the script as follows:
```
$ chmod u+x here_02.sh
$ ./here_02.sh
```
The output is here:
```
apple
banana
cherry
mango
```

In this script, the here document is enclosed between the EOF pattern. We have used the here document to supply text to the sort command.

The here operator with the wc command

Let's write script for using the wc command along with the here document:

Create Shell script here_03.sh:

#!/bin/bash
wc -w << EOF
There was major earthquake
On April 25, 2015
in Nepal.
There was huge loss of human life in this tragic event.
EOF

Save the file, give the permission to execute, and run the script as follows:
```
$ chmod u+x here_03.sh
$ ./here_03.sh
```
The output is here:
```
21
```

In this script, we have used the here document as an input for the wc command to calculate the number of words:

Tape backup using << here operator

Let's write a script for taking the tape backup by using the tar command and the here document:

Let's write the script here_04.sh:

#!/bin/bash
# We have used tar utility for archiving home folder on tape
tar -cvf /dev/st0 /home/student 2>/dev/null

# store status of tar operation in variable status
[ $? -eq 0 ] && status="Success" || status="Failed"

# Send email to administrator
mail -s 'Backup status' ganesh@levanatech.com << End_Of_Message
The backup job finished.
End date: $(date)
Status : $status
End_Of_Message

Save the file, give the permission to execute, and run the script as follows:
```
$ chmod u+x here_04.sh
$ ./here_04.sh
```

This script uses the tar command to archive the home folder in the tape device, and then it sends mail to an administrator using the command mail. We have used the here document to feed data into the command mail.

The utility ed and here operator

The ed is a basic type of editor. We can edit text files using this editor:

Write the script here_05.sh:

#!/bin/bash
# flowers.txt contains the name of flowers
cat flowers.txt
ed flowers.txt << quit
,s/Rose/Lily/g
w
q
quit
cat flowers.txt

Save the file, give the permission to execute, and run the script as follows:
```
$ chmod u+x here_05.sh
$ ./here_05.sh
```

The output is here:

  Aster, Daffodil, Daisy, Jasmin, Lavender, Rose, Sunflower
  59
  59
  Aster, Daffodil, Daisy, Jasmin, Lavender, Lily, Sunflower

In this script, we have used passed the here document to utility for editing the file flowers.txt. We replaced the Rose word with Lily.

A script for sending messages to all logged-in users

All the users who are logged in will receive the message using the wall command:

Write the script here_06.sh:

#!/bin/bash
# wall utility is used for sending message to all logged in users
wall << End_Of_Message
Tomorrow, on Friday evening, we will be celebrating
Birthday of few of our colleagues.
All are requested to be present in cafeteria by 3.30 PM.
    John
End_Of_Message
echo "Message sent"

Save the file, give the permission to execute, and run the script as follows:
```
$ chmod u+x here_06.sh
$ ./here_06.sh
```

The command wall is used to send messages to the logged-in users. All the users that are logged in will receive the message.

Using the << here operator for FTP usage and data transfer

FTP is a commonly used protocol to transfer data on websites. FTP stands for File Transfer Protocol. The following steps show the usage of FTP and data transfer:

Write the script here_07.sh:

#!/bin/bash
# Checking number of arguments passed along with command
if [ $# -lt 2 ]
then
  echo "Error, usage is:"
  echo "ftpget hostname filename [directory]."
  exit -1
fi
hostname=$1
filename=$2
directory="." # Default value
if [ $# -ge 3 ]
then
  directory=$3
fi
ftp <<End_Of_Session
open $hostname
cd $directory
get $filename
quit
End_Of_Session
echo "FTP session ended."

Save the file, give the permission to execute, and run the script as follows:
```
$ chmod u+x here_07.sh
$ ./here_07.sh ftp.somehost.com index.html WWW
```

For a successful execution of the script, we need to set up an autologin for the ftp command. The here operator does not work well when the ftp command asks for a username and password.

Turning off variable substitution

Enter the following script to see how to avoid a variable substitution in these files:

Save the script under the name here_no.sh, shown as follows:

filename="test1"
cat <<'Quoted_End_Marker'
When we add quotes before and after here
Document marker, we can include variables
Such as $USER, $PATH, $name and similar
Quoted_End_Marker

When you run this script, you will see the output like the following:

$ bash here_no.sh
With quotes around the here document marker,
you can include variable references such
as $HOME, $filename, and $USER.

This script uses an ordinary here file, but it turns off the variable substitution. Otherwise, you would see the values of $HOME, $filename, and $USER in the output instead of the literal text. All of this is done by magically enclosing the end marker, Quoted_End_Marker, in quotes at the original reference. Do not enclose the marker in quotes at the end of the here file.

Previous Chapter

6. Neat Tricks with Shell Scripting

Next Chapter

The here string and the <<< operator

Table of Contents for Learning Linux Shell Scripting