Table of Contents for

sed & awk, 2nd Edition

The syntax of a while loop is:

while (condition) 
	action

The newline is optional after the right parenthesis. The conditional expression is evaluated at the top of the loop and, if true, the action is performed. If the expression is never true, the action is not performed. Typically, the conditional expression evaluates to true and the action changes a value such that the conditional expression eventually returns false and the loop is exited. For instance, if you wanted to perform an action four times, you could write the following loop:

i = 1
while ( i <= 4 ) {
	print $i
	++i 
}

As in an if statement, an action consisting of more than one statement must be enclosed in braces. Note the role of each statement. The first statement assigns an initial value to i. The expression “i <= 4” compares i to 4 to determine if the action should be executed. The action consists of two statements, one that simply prints the value of a field referenced as “$i” and another that increments i. i is a counter variable and is used to keep track of how many times we go through the loop. If we did not increment the counter variable or if the comparison would never evaluate to false (e.g., i > 0), then the action would be repeated without end.

The do loop is a variation of the while loop. The syntax of a do loop is:

do
	action
while (condition)

The newline is optional after do. It is also optional after action providing the statement is terminated by a semicolon. The main feature of this loop is that the conditional expression appears after the action. Thus, the action is performed at least once. Look at the following do loop.

BEGIN {
	do {
		++x
		print x
	} while ( x <= 4 )
}

In this example, the value of x is set in the body of the loop using the auto-increment operator. The body of the loop is executed once and the expression is evaluated. In the previous example of a while loop, the initial value of i was set before the loop. The expression was evaluated first, then the body of the loop was executed once. Note the value of x when we run this example:

$ awk -f do.awk
1
2
3
4
5

Before the conditional expression is first evaluated, x is incremented to 1. (This relies on the fact that all awk variables are initialized to zero.) The body of the loop is executed five times, not four; when x equals 4, the conditional expression is true and the body of the loop is executed again, incrementing x to 5 and printing its value. Only then is the conditional expression evaluated to false and the loop exited. By changing the operator from “<=” to “<”, or less than, the body of the loop will be executed four times.

To keep in mind the difference between the do loop and the while loop, remember that the do loop always executes the body of the loop at least once. At the bottom of the procedure, you decide if you need to execute it again.

For an example, let’s look at a program that loops through the fields of a record, referencing as many fields as necessary until their cumulative value exceeds 100. The reason we use a do loop is that we will reference at least one of the fields. We add the value of the field to the total, and if the total exceeds 100 we don’t reference any other fields. We reference the second field only if the first field is less than 100. Its value is added to the total and if the total exceeds 100 then we exit the loop. If it is still less than 100, we execute the loop once again.

{
	total = i = 0
	do {
		++i
		total += $i
	} while ( total <= 100 )
	print i, ":", total
}

The first line of the script initializes the values of two variables: total and i. The loop increments the value of i and uses the field operator to reference a particular field. Each time through the loop, it refers to a different field. When the loop is executed for the first time, the field reference gets the value of field one and assigns it to the variable total. The conditional expression at the end of the loop evaluates whether total exceeds 100. If it does, the loop is exited. Then the value of i, the number of fields that we’ve referred to, and the total are printed. (This script assumes that each record totals at least 100; otherwise, we’d have to check that i does not exceed the number of fields for the record. We construct such a test in the example presented in next section to show the for loop.)

Here’s a test file containing a series of numbers:

$ cat test.do
45 25 60 20
10 105 50 40
33 5 9 67
108 3 5 4

Running the script on the test file produces the following:

$ awk -f do.awk test.do
3 : 130
2 : 115
4 : 114
1 : 108

For each record, only as many fields are referenced as needed for the total to exceed 100.

The for statement offers a more compact syntax that achieves the same result as a while loop. Although it looks more difficult, this syntax is much easier to use and makes sure that you provide all the required elements of a loop. The syntax of a for loop is:

for ( set_counter ; test_counter ; increment_counter ) 
    action

The newline after the right parenthesis is optional. The for loop consists of three expressions:

Look at this rather common for loop that prints each field on the input line.

for ( i = 1; i <= NF; i++ )
	print $i

As in the previous example, i is a variable that is used to reference a field using the field operator. The system variable NF contains the number of fields for the current input record, and we test it to determine if i has reached the last field on the line. The value of NF is the maximum number of times to go through the loop. Inside the loop, the print statement is executed, printing each field on its own line. A script using this construct can print each word on a line by itself, which can then be run through sort | uniq -c to get word distribution statistics for a file.

You can also write a loop to print from the last field to the first.

for ( i = NF; i >= 1; i-- )
	print $i

Each time through the loop the counter is decremented. You could use this to reverse the order of fields.

The grades.awk script that we showed earlier determined the average of five grades. We can make the script much more useful by averaging any number of grades. That is, if you were to run this script throughout the school year, the number of grades to average would increase. Rather than revising the script to accommodate the specific number of fields, we can write a generalized script that loops to read however many fields there are. Our earlier version of the program calculated the average of 5 grades using these two statements:

total = $2 + $3 + $4 + $5 + $6
avg = total / 5

We can revise that using a for loop to sum each field in the record.

total = 0
for (i = 2; i <= NF; ++i)
      total += $i
avg = total / (NF - 1)

We initialize the variable total each time because we don’t want its value to accumulate from one record to the next. At the beginning of the for loop, the counter i is initialized to 2 because the first numeric field is field 2. Each time through the loop the value of the current field is added to total. When the last field has been referenced (i is greater than NF), we break out of the loop and calculate the average. For instance, if the record consists of 4 fields, the first time through the loop, we assign the value of $2 to total. At the bottom of the loop, i is incremented by 1, then compared to NF, which is 4. The expression evaluates to true and total is incremented by the value of $3.

Notice how we divide the total by the number of fields minus 1 to remove the student name from the count. The parentheses are required around “NF - 1” because the precedence of operators would otherwise divide total by NF and then subtract 1, instead of subtracting 1 from NF first.

The factorial of a number is the product of successively multiplying that number by one less than that number. The factorial of 4 is 4 x 3 x 2 x 1, or 24. The factorial of 5 is 5 times the factorial of 4 or 5 x 24, or 120. Deriving a factorial for a given number can be expressed using a loop as follows:

fact = number
for (x = number - 1 ; x > 1; x--)
	fact *= x

where number is the number for which we will derive the factorial fact. Let’s say that number equals 5. The first time through the loop x is equal to 4. The action evaluates “5 * 4” and assigns the value to fact. The next time through the loop, x is 3 and 20 is multiplied by it. We go through the loop until x equals 1.

Here is the above fragment incorporated into a standalone script that prompts the user for a number and then prints the factorial of that number.

awk '# factorial: return factorial of user-supplied number
BEGIN {
	# prompt user; use printf, not print, to avoid the newline
	printf("Enter number: ")
}

# check that user enters a number
$1 ~ /^[0-9]+$/ { 
	# assign value of $1 to number & fact
	number = $1
	if (number == 0)
		fact = 1
	else
		fact = number
	# loop to multiply fact*x until x = 1
	for (x = number - 1; x > 1; x--)
		fact *= x
	printf("The factorial of %d is %g\n", number, fact)
	# exit -- saves user from typing CRTL-D.
	exit
}

# if not a number, prompt again.
{ printf("\nInvalid entry. Enter a number: ") 
}' -

This is an interesting example of a main input loop that prompts for input and reads the reply from standard input. The BEGIN rule is used to prompt the user to enter a number. Because we have specified that input is to come not from a file but from standard input, the program will halt after putting out the prompt and then wait for the user to type a number. The first rule checks that a number has been entered. If not, the second rule will be applied, prompting the user again to re-enter a number. We set up an input loop that will continue to read from standard input until a valid entry is found. See the lookup program in the next section for another example of constructing an input loop.

Here’s an example of how the factorial program works:

$ factorial
Enter number: 5
The factorial of 5 is 120

Note that the result uses “%g” as the conversion specification format in the printf statement. This permits floating point notation to be used to express very large numbers. Look at the following example:

$ factorial
Enter number: 33
The factorial of 33 is 8.68332e+36