Now let's look at the curl program as it is somewhat similar to wget. One of the main differences between wget and curl is how they handle the output.

The wget program by default displays some progress information on the screen and then downloads the index.html file. In contrast, curl normally displays the file itself on the screen.

Here is an example of curl running on my system using my favorite website (screenshot shortened to save space):

Another way to get the output into a file is by using redirection like this:

You will notice that when redirected to a file the transfer progress is displayed on the screen. Also note that any error output goes into the file if redirected and not the screen.

curl options

Here is a very brief list of the options available in curl:

Curl options	Explanation
-o	Output filename
-s	Silent mode. Shows nothing, not even errors
-S	Show errors if in silent mode
-v	Verbose, useful for debugging

There are many other options to curl, as well as several pages of return codes. For more information consult the curl man page.

And now here is a script showing how curl can be used to automatically get the current value of the Dow Jones Industrial Average:

Chapter 8 - Script 2

#!/bin/sh
# 6/6/2017
# Chapter 8 - Script 2

URL="https://www.google.com/finance?cid=983582"
FN=outfile1.txt              # output file
TF=temp1.txt                 # temp file for grep

loop=1
while [ $loop -eq 1 ]
do
 rm $FN 2> /dev/null         # remove old file
 curl -o $FN $URL            # output to file
 rc=$?
 if [ $rc -ne 0 ] ; then
  echo "curl returned code: $rc"
  echo "outfile:"
  cat $FN

  exit 200
 fi

 echo ""                     # carriage return
 date
 grep "ref_983582_l" $FN > $TF
 echo -n "DJIA: "
 cat $TF | cut -c 25-33

 sleep 1h
done

exit 0

Here's what it looks like on my system. Normally you would probably leave the progress information out of the output by using the -s option but I thought it looked cool and so left it in:

You can see that curl and wget work pretty much the same way. Remember when writing scripts such as these to keep in mind that the format of the page will almost certainly change from time to time so plan accordingly.