Table of Contents for
sed & awk, 2nd Edition

Version ebook / Retour

Cover image for bash Cookbook, 2nd Edition sed & awk, 2nd Edition by Arnold Robbins Published by O'Reilly Media, Inc., 1997
  1. sed & awk, 2nd Edition
  2. Cover
  3. sed & awk, 2nd Edition
  4. A Note Regarding Supplemental Files
  5. Dedication
  6. Preface
  7. Scope of This Handbook
  8. Availability of sed and awk
  9. Obtaining Example Source Code
  10. Conventions Used in This Handbook
  11. About the Second Edition
  12. Acknowledgments from the First Edition
  13. Comments and Questions
  14. 1. Power Tools for Editing
  15. 1.1. May You Solve Interesting Problems
  16. 1.2. A Stream Editor
  17. 1.3. A Pattern-Matching Programming Language
  18. 1.4. Four Hurdles to Mastering sed and awk
  19. 2. Understanding Basic Operations
  20. 2.1. Awk, by Sed and Grep, out of Ed
  21. 2.2. Command-Line Syntax
  22. 2.3. Using sed
  23. 2.4. Using awk
  24. 2.5. Using sed and awk Together
  25. 3. Understanding Regular Expression Syntax
  26. 3.1. That’s an Expression
  27. 3.2. A Line-Up of Characters
  28. 3.3. I Never Metacharacter I Didn’t Like
  29. 4. Writing sed Scripts
  30. 4.1. Applying Commands in a Script
  31. 4.2. A Global Perspective on Addressing
  32. 4.3. Testing and Saving Output
  33. 4.4. Four Types of sed Scripts
  34. 4.5. Getting to the PromiSed Land
  35. 5. Basic sed Commands
  36. 5.1. About the Syntax of sed Commands
  37. 5.2. Comment
  38. 5.3. Substitution
  39. 5.4. Delete
  40. 5.5. Append, Insert, and Change
  41. 5.6. List
  42. 5.7. Transform
  43. 5.8. Print
  44. 5.9. Print Line Number
  45. 5.10. Next
  46. 5.11. Reading and Writing Files
  47. 5.12. Quit
  48. 6. Advanced sed Commands
  49. 6.1. Multiline Pattern Space
  50. 6.2. A Case for Study
  51. 6.3. Hold That Line
  52. 6.4. Advanced Flow Control Commands
  53. 6.5. To Join a Phrase
  54. 7. Writing Scripts for awk
  55. 7.1. Playing the Game
  56. 7.2. Hello, World
  57. 7.3. Awk’s Programming Model
  58. 7.4. Pattern Matching
  59. 7.5. Records and Fields
  60. 7.6. Expressions
  61. 7.7. System Variables
  62. 7.8. Relational and Boolean Operators
  63. 7.9. Formatted Printing
  64. 7.10. Passing Parameters Into a Script
  65. 7.11. Information Retrieval
  66. 8. Conditionals, Loops, and Arrays
  67. 8.1. Conditional Statements
  68. 8.2. Looping
  69. 8.3. Other Statements That Affect Flow Control
  70. 8.4. Arrays
  71. 8.5. An Acronym Processor
  72. 8.6. System Variables That Are Arrays
  73. 9. Functions
  74. 9.1. Arithmetic Functions
  75. 9.2. String Functions
  76. 9.3. Writing Your Own Functions
  77. 10. The Bottom Drawer
  78. 10.1. The getline Function
  79. 10.2. The close( ) Function
  80. 10.3. The system( ) Function
  81. 10.4. A Menu-Based Command Generator
  82. 10.5. Directing Output to Files and Pipes
  83. 10.6. Generating Columnar Reports
  84. 10.7. Debugging
  85. 10.8. Limitations
  86. 10.9. Invoking awk Using the #! Syntax
  87. 11. A Flock of awks
  88. 11.1. Original awk
  89. 11.2. Freely Available awks
  90. 11.3. Commercial awks
  91. 11.4. Epilogue
  92. 12. Full-Featured Applications
  93. 12.1. An Interactive Spelling Checker
  94. 12.2. Generating a Formatted Index
  95. 12.3. Spare Details of the masterindex Program
  96. 13. A Miscellany of Scripts
  97. 13.1. uutot.awk—Report UUCP Statistics
  98. 13.2. phonebill—Track Phone Usage
  99. 13.3. combine—Extract Multipart uuencoded Binaries
  100. 13.4. mailavg—Check Size of Mailboxes
  101. 13.5. adj—Adjust Lines for Text Files
  102. 13.6. readsource—Format Program Source Files for troff
  103. 13.7. gent—Get a termcap Entry
  104. 13.8. plpr—lpr Preprocessor
  105. 13.9. transpose—Perform a Matrix Transposition
  106. 13.10. m1—Simple Macro Processor
  107. A. Quick Reference for sed
  108. A.1. Command-Line Syntax
  109. A.2. Syntax of sed Commands
  110. A.3. Command Summary for sed
  111. B. Quick Reference for awk
  112. B.1. Command-Line Syntax
  113. B.2. Language Summary for awk
  114. B.3. Command Summary for awk
  115. C. Supplement for Chapter 12
  116. C.1. Full Listing of spellcheck.awk
  117. C.2. Listing of masterindex Shell Script
  118. C.3. Documentation for masterindex
  119. masterindex
  120. C.3.1. Background Details
  121. C.3.2. Coding Index Entries
  122. C.3.3. Output Format
  123. C.3.4. Compiling a Master Index
  124. Index
  125. About the Authors
  126. Colophon
  127. Copyright

A Menu-Based Command Generator

In this section, we look at a general use of the system( ) and getline functions to implement a menu-based command generator. The object of this program is to give unsophisticated users a simple way to execute long or complex UNIX commands. A menu is used to prompt the user with a description of the task to be performed, allowing the user to choose by number any selection of the menu to execute.

This program is designed as a kind of interpreter that reads from a file the descriptions that appear in the menu and the actual command lines that are executed. That way, multiple menu-command files can be used, and they can be easily modified by awk-less users without changing the program.

The format of a menu-command file contains the menu title as the first line in the file. Subsequent lines contain two fields: the first is the description of the action to be performed and the second is the command line that performs it. An example is shown below:

$ cat uucp_commands
UUCP Status Menu
Look at files in PUBDIR:find /var/spool/uucppublic -print
Look at recent status in LOGFILE:tail /var/spool/uucp/LOGFILE
Look for lock files:ls /var/spool/uucp/*.LCK

The first step in implementing the menu-based command generator is to read the menu-command file. We read the first line of this file and assign it to a variable named title. The rest of the lines contain two fields and are read into two arrays, one for the menu items and one for the commands to be executed. A while loop is used, along with getline, to read one line at a time from the file.

BEGIN { FS = ":"
if ((getline < CMDFILE) > 0)
	title = $1
else
	exit 1
while ((getline < CMDFILE) > 0) {
	# load array
	++sizeOfArray
	# array of menu items
	menu[sizeOfArray] = $1
	# array of commands associated with items
	command[sizeOfArray] = $2
	} 
...
}

Look carefully at the syntax of the expression tested by the if statement and the while loop.

(getline < CMDFILE) > 0

The variable CMDFILE is the name of the menu-command file, which is passed as a command-line parameter. The two angle-bracket symbols have completely different functions. The “<” symbol is interpreted by getline as the input redirection operator. Then the value returned by getline is tested to see if it is greater than (“>”) 0. It is parenthesized on purpose, in order to make this clear. In other words, “getline < CMDFILE" is evaluated first and then its return value is compared to 0.

This procedure is placed in the BEGIN pattern. However, there is one catch. Because we intended to pass the name of the menu file as a command-line parameter, the variable CMDFILE would not normally be defined and available in the BEGIN pattern. In other words, the following command will not work:

awk script CMDFILE="uucp_commands" -

because CMDFILE variable won’t be defined until the first line of input is read.

Fortunately, awk provides the -v option to handle just such a case. Using the -v option makes sure that the variable is set immediately and thus available in the BEGIN pattern.

awk -v CMDFILE="uucp_commands" script -

If your version of awk doesn’t have the -v option, you can pass the value of CMDFILE as a shell variable. Create a shell script to execute awk and in it define CMDFILE. Then change the line that reads CMDFILE in the invoke script (see below) as follows:

while ((getline < '"$CMDFILE"') > 0 ) {

Once the menu-command file is loaded, the program must display the menu and prompt the user. This is implemented as a function because we need to call it in two places: from the BEGIN pattern to prompt the user initially, and after we have processed the user’s response so that another choice can be made. Here’s the display_menu( ) function:

function display_menu( ) {
	# clear screen -- comment out if clear does not work
	system("clear")
	# print title, list of items, exit item, and prompt
	print "\t" title
	for (i = 1; i <= sizeOfArray; ++i)
		printf "\t%d. %s\n", i, menu[i]
	printf "\t%d. Exit\n", i
	printf("Choose one: ")
}

The first thing we do is use the system( ) function to call a command to clear the screen. (On my system, clear does this; on others it may be cls or some other command. Comment out the line if you cannot find such a command.) Then we print the title and each of the items in a numbered list. The last item is always “Exit.” Finally, we prompt the user for a choice.

The program will take standard input so that the user’s answer to the prompt will be the first line of input. Our reading of the menu-command file was done within the program and not as part of the input stream. Thus, the main procedure of the program is to respond to the user’s choice and execute a command. Here’s that part of the program:

# Applies the user response to prompt
{
   # test value of user response
   if ($1 > 0 && $1 <= sizeOfArray) {
	# print command that is executed
	printf("Executing ... %s\n", command[$1]) 
	# then execute it. 
	system(command[$1])
	printf("<Press RETURN to continue>")
 	# wait for input before displaying menu again
	getline
   }
   else 
	exit	
   # re-display menu 
   display_menu( )
}

First, we test the range of the user’s response. If the response falls outside the range, we simply exit the program. If it is a valid response, then we retrieve the command from the array command, display it, and then execute it using the system( ) function. The user sees the result of the command on the screen followed by the message “<Press RETURN to continue>.” The purpose of this message is to wait for the user to finish before clearing the screen and redisplaying the menu. The getline function causes the program to wait for a response. Note that we don’t do anything with the response. The display_menu( ) function is called at the end of this procedure to redisplay the menu and prompt for another line of input.

Here’s the invoke program in full:

awk -v CMDFILE="uucp_commands"  '# invoke -- menu-based
                                  # command generator
# first line in CMDFILE is the title of the menu
# subsequent lines contain: $1 - Description;
# $2 Command to execute
BEGIN { FS = ":" 
# process CMDFILE, reading items into menu array 
  if ((getline < CMDFILE) > 0)
	title = $1
  else
	exit 1
  while ((getline < CMDFILE) > 0) {
	# load array
	++sizeOfArray
	# array of menu items
	menu[sizeOfArray] = $1
	# array of commands associated with items
	command[sizeOfArray] = $2
  } 
  # call function to display menu items and prompt
  display_menu( )
}
# Applies the user response to prompt
{
   # test value of user response
   if ($1 > 0 && $1 <= sizeOfArray) {
	# print command that is executed
	printf("Executing ... %s\n", command[$1]) 
	# then execute it. 
	system(command[$1])
	printf("<Press RETURN to continue>")
 	# wait for input before displaying menu again
	getline
   }
   else 
	exit	
   # re-display menu 
   display_menu( )
}

function display_menu( ) {
	# clear screen -- if clear does not work, try "cls"
	system("clear")
	# print title, list of items, exit item, and prompt
	print "\t" title
	for (i = 1; i <= sizeOfArray; ++i)
		printf "\t%d. %s\n", i, menu[i]
	printf "\t%d. Exit\n", i
	printf("Choose one: ")
}' -

When a user runs the program, the following output is displayed:

UUCP Status Menu
        1. Look at files in PUBDIR
        2. Look at recent status in LOGFILE
        3. Look for lock files
        4. Exit
Choose one:

The user is prompted to enter the number of a menu selection. Anything other than a number between 1 and 3 exits the menu. For instance, if the user enters “1” to see a list of files in uucp’s public directory, then the following result is displayed on the screen:

Executing ...find /var/spool/uucppublic -print
/var/spool/uucppublic
/var/spool/uucppublic/dale
/var/spool/uucppublic/HyperBugs
<Press RETURN to continue>

When the user presses the RETURN key, the menu is redisplayed on the screen. The user can quit from the program by choosing “4”.

This program is really a shell for executing commands. Any sequence of commands (even other awk programs) can be executed by modifying the menu-command file. In other words, the part of the program that might change the most is extracted from the program itself and maintained in a separate file. This allows the menu list to be changed and extended very easily by a nontechnical user.