2.1.1 Working with Code Examples

Throughout this chapter, you will see numerous code examples. Compile them and if you have difficulty grasping their logic, try to execute them step by step using gdb. It is a great help in studying code. See Appendix A for a quick tutorial on gdb.

Appendix D provides more information about the system used for performance tests.

2.2.1 Basic Input and Output

Unix ideology postulates that “everything is a file.” A file, in a large sense, is anything that looks like a stream of bytes. Through files one can abstract such things as

• data access on a hard drive/SSD;

• data exchange between programs; and

• interaction with external devices.

We will follow the tradition of writing a simple “Hello, world!” program for a start. It displays a welcome message on screen and terminates. However, such a program must show characters on screen, which cannot be done directly if a program is not running on bare metal, without an operating system babysitting its activity. An operating system’s purpose is, among other things, to abstract and manage resources, and display is surely one of them. It provides a set of routines to handle communication with external devices, other programs, file systems, and so on. A program usually cannot bypass the operating system and interact directly with the resources it controls. It is limited to system calls, which are routines provided by an operating system to user applications.

Unix identifies a file with its descriptor as soon as it is opened by a program. A descriptor is nothing more than an integer value (like 42 or 999). A file is opened explicitly by invoking the open system call; however, three important files are opened as soon as a program starts and thus should not be managed manually. These are stdin, stdout, and stderr. Their descriptors are 0, 1, and 2, respectively. stdin is used to handle input, stdout to handle output, and stderr is used to output information about the program execution process but not its results (e.g., errors and diagnostics).

By default, keyboard input is linked to stdin and terminal output is linked to stdout . It means that “Hello, world!” should write into stdout .

Thus we need to invoke the write system call. It writes a given amount of bytes from memory starting at a given address to a file with a given descriptor (in our case, 1). The bytes will encode string characters using a predefined table (ASCII-table). Each entry is a character; an index in the table corresponds to its code in a range from 0 to 255.

See Listing 2-1 for our first complete example of an assembly program.

Listing 2-1. hello.asm

global _start

section .data
message: db 'hello, world!', 10

section .text
_start:
    mov     rax, 1           ;system call number should be stored in rax
    mov     rdi, 1           ; argument #1 in rdi: where to write (descriptor)?
    mov     rsi, message     ; argument #2 in rsi: where does the string start?
    mov     rdx, 14          ; argument #3 in rdx: how many bytes to write?
    syscall                  ; this instruction invokes a system call

This program invokes a write system call with correct arguments on lines 6-9. It is really the only thing it does. The next sections will explain this sample program in greater detail.

2.2.2 Program Structure

As we remember from the von Neumann machine description, there is only one memory, for both code and data; those are indistinguishable. However, a programmer wants to separate them. An assembly program is usually divided into sections. Each section has its use: for example, .text holds instructions, .data is for global variables (data available in every moment of the program execution). One can switch back and forth between sections; in the resulting program all data, corresponding to each section, will be gathered in one place.

To get rid of numeric address values programmers use labels. They are just readable names and addresses. They can precede any command and are usually separated from it by a colon. There is one label in this program at line 5. _start.

A notion of variable is typical for higher-level languages. In assembly language, in fact, notions of variables and procedures are quite subtle. It is more convenient to speak about labels (or addresses).

An assembly program can be divided into multiple files. One of them should contain the _start label. It is the entry point; it marks the first instruction to be executed.

This label should be declared global (see line 1). The meaning of it will be evident later.

Comments start with a semicolon and last until the end of the line.

Assembly language consists of commands, which are directly mapped into machine code. However, not all language constructs are commands. Others control the translation process and are usually called directives. ¹

In the “Hello, world!” example there are three directives: global, section, and db.

The db directive is used to create byte data. Usually data is defined using one of these directives, which differ by data format:

• db—bytes;

• dw—so-called words, equal to 2 bytes each;

• dd—double words, equal to 4 bytes; and

• dq—quad words, equal to 8 bytes.

Let’s see an example, in Listing 2-2.

Listing 2-2. data_decl.asm

section .data
   example1: db 5, 16, 8, 4, 2, 1
   example2: times 999 db 42
   example3: dw 999

times n cmd is a directive to repeat cmd n times in program code. As if you copy-pasted it n times. It also works with central processor unit (CPU) instructions.

Note that you can create data inside any section, including .text. As we told you earlier, for a CPU data and instructions are all alike and the CPU will try to interpret data as encoded instructions when asked to.

These directives allow you to define several data objects one by one, as in Listing 2-3, where a sequence of characters is followed by a single byte equal to 10.

Listing 2-3. hello.asm

message: db 'hello, world!', 10

Letters, digits, and other characters are encoded in ASCII. Programmers have agreed upon a table, where each character is assigned a unique number—its ASCII-code. We start at address corresponding to the label message. We store the ASCII codes for all letters of string "hello, world!", then we add a byte equal to 10. Why 10? By convention, to start a new line we output a special character with code 10.

2.2.3 Basic Instructions

The mov instruction is used to write a value into either register or memory. The value can be taken from other register or from memory, or it can be an immediate one. However,

1. mov cannot copy data from memory to memory;

2. the source and the destination operands must be of the same size.

The syscall instruction is used to perform system calls in *nix systems. The input/output operations depend on hardware (which can be also used by multiple programs at the same time), so programmers are not allowed to control them directly, bypassing the operating system.

Each system call has a unique number. To perform it

1. The rax register has to hold system call’s number;

2. The following registers should hold its arguments: rdi, rsi, rdx, r10, r8, and r9.

   System call cannot accept more than six arguments.

3. Execute syscall instruction.

It does not matter in which order the registers are initialized.

Note, that the syscall instruction changes rcx and r11! We will explain the cause later. When we wrote the “Hello, world!” program we used a simple write syscall. It accepts

1. File descriptor ;

2. The buffer address. We start taking consecutive bytes for writing from here;

3. The amount of bytes to write.

To compile our first program, save the code in hello.asm ² and then launch these commands in the shell:

> nasm -felf64 hello.asm -o hello.o
> ld -o hello hello.o
> chmod u+x hello

The details of compilation process along with compilation stages will be discussed in Chapter 5. Let’s launch “Hello, world!”

> ./hello
hello, world!
Segmentation fault

We have clearly output what we wanted. However, the program seems to have caused an error. What did we do wrong? After executing a system call, the program continues its work. We did not write any instructions after syscall, but the memory holds indeed some random values in the next cells.

A processor has no idea whether these values were intended to encode instructions or not. So, following its very nature, it tries to interpret them, because rip register points at them. It is highly unlikely these values encode correct instructions, so an interrupt with code 6 will occur (invalid instruction).³

So what do we do? We have to use the exit system call , which terminates the program in a correct way, as shown in Listing 2-4.

Listing 2-4. hello_proper_exit.asm

section .data
message: db 'hello, world!', 10

section .text
global _start

_start:
    mov     rax, 1           ; 'write' syscall number
    mov     rdi, 1           ; stdout descriptor
    mov     rsi, message     ; string address
    mov     rdx, 14          ; string length in bytes
    syscall

    mov     rax, 60          ; 'exit' syscall number
    xor     rdi, rdi
    syscall                                                                                                                                                                                

2.3.1 Local Labels

Notice the unusual label name . loop: it starts with a dot. This label is local. We can reuse the label names without causing name conflicts as long as they are local.

The last used dotless global label is a base one for all subsequent local labels (until the next global label occurs). The full name for .loop label is _start.loop. We can use this name to address it from anywhere in the program, even after other global labels occurs.

2.3.2 Relative Addressing

This demonstrates how to address memory in a more complex way than just by immediate address.

Listing 2-6. Relative Addressing:  print_rax.asm

lea rsi, [codes + rax]

Square brackets denote indirect addressing; the address is written inside them.

• mov rsi, rax—copies rax into rsi

• mov rsi, [rax]—copies memory contents (8 sequential bytes) starting at address, stored in rax, into rsi. How do we know that we have to copy exactly 8 bytes? As we know, mov operands are of the same size, and the size of rsi is 8 bytes. Knowing these facts, the assembler is able to deduce that exactly 8 bytes should be taken from memory.

The instructions lea and mov have a subtle difference between their meanings. lea means “load effective address.”

It allows you to calculate an address of a memory cell and store it somewhere. This is not always trivial, because there are tricky address modes (as we will see later): for example, the address can be a sum of several operands.

Listing 2-7 provides a quick demonstration of what lea and mov are doing.

Listing 2-7. lea_vs_mov.asm

; rsi <- address of label 'codes', a number
mov rsi, codes

; rsi <- memory contents starting at 'codes' address
; 8 consecutive bytes are taken because rsi is 8 bytes long
mov rsi, [codes]

; rsi <- address of 'codes'
; in this case it is equivalent of mov rsi, codes
; in general the address can contain several components
lea rsi, [codes]

; rsi <- memory contents starting at (codes+rax)
mov rsi, [codes + rax]

; rsi <- codes + rax
; equivalent of combination:
; -- mov rsi, codes
; -- add rsi, rax
; Can't do it with a single mov!
lea rsi, [codes + rax]

2.3.3 Order of Execution

All commands are executed consecutively except when special jump instructions occur. There is an unconditional jump instruction jmp addr. It can be viewed as a substitute of mov rip, addr.⁵

Conditional jumps rely on contents of rflags register. For example, jz address jumps to address only if zero flag is set.

Usually one uses either a test or a cmp instruction to set up necessary flags coupled with conditional jump instruction.

cmp subtracts the second operand from the first; it does not store the result anywhere, but it sets the appropriate flags based on it (e.g., if operands are equal, it will set zero flag). test does the same thing but uses logical AND instead of subtraction.

An example shown in Listing 2-8 incorporates writing 1 in rbx if rax < 42, and 0 otherwise.

Listing 2-8. jumps_example.asm

    cmp rax, 42
    jl yes
    mov rbx, 0
    jmp ex
yes:
    mov rbx, 1
ex:

It is a common (and fast) way to test register value for being zero with test reg,reg instruction.

At least two commands exist for each arithmetic flag F: jF and jnF. For example, sign flag: js and jns. Other useful commands include

1. ja (jump if above)/jb (jump if below) for a jump after a comparison of unsigned numbers with cmp.

2. jg (jump if greater)/jl (jump if less) for signed.

3. jae (jump if above or equal), jle (jump if less or equal) and similar. Some of common jump instructions are shown in Listing 2-9.

Listing 2-9. Jump Instructions:  jumps.asm

mov rax, -1
mov rdx, 2

cmp rax, rdx
jg location
ja location           ; different logic!

cmp rax, rdx
je  location          ; if rax equals rdx
jne location          ; if rax is not equal to rdx

2.5.1 Endianness

Let’s try to output a value stored in memory using the function we just wrote. We are going to do it in two different ways: first we will enumerate all its bytes separately and then we will type it as usual (see Listing 2-11).

Listing 2-11. endianness.asm

section .data
demo1: dq 0x1122334455667788
demo2: db 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88

section .text

_start:
    mov rdi, [demo1]
    call print_hex
    call print_newline

    mov rdi, [demo2]
    call print_hex
    call print_newline

    mov rax, 60
    xor rdi, rdi
    syscall

When we launch it, to our surprise, we get completely different results for demo1 and demo2.

> ./main
1122334455667788
8877665544332211

As we see, multi-byte numbers are stored in reverse order!

The bits in each byte are stored in a straightforward way, but the bytes are stored from the least significant to the most significant.

This applies only to memory operations: in registers, the bytes are stored in a natural way. Different processors have different conventions on how the bytes are stored.

• Big endian multibyte numbers are stored in memory starting with the most significant bytes.

• Little endian multibyte numbers are stored in memory starting with the least significant bytes.

As the example shows , Intel 64 is following the little endian convention. In general, choosing one convention over the other is a matter of choice, made by hardware engineers.

These conventions do not concern arrays and strings. However, if each character is encoded using 2 bytes rather than just 1, those bytes will be stored in reverse order.

The advantage of little endian is that we can discard the most significant bytes effectively converting the number from a wider format to a narrower one, like 8 bytes.

For example,  demo3: dq 0x1234. Then, to convert this number into dw we have to read a dword number starting at the same address demo3. See Table 2-1 for a complete memory layout.

Big endian is a native format often used inside network packets (e.g., TCP/IP). It is also an internal number format for Java Virtual Machine.

Middle endian is a not very well-known notion. Assume we want to create a set of routines to perform arithmetic with 128-bit numbers. Then the bytes can be stored as follows: first will be the 8 least significant bytes in reversed order and then the 8 most significant bytes also in reverse order:

• 7 6 5 4 3 2 1 0, 16 15 14 13 12 11 10 9 8

2.5.2 Strings

As we already know, the characters are encoded using the ASCII table . A code is assigned to each character. A string is obviously a sequence of character codes. However, it does not say anything about how to determine its length.

1. Strings start with their explicit length.

   db 27, 'Selling England by the Pound'

2. A special character denotes the string ending. Traditionally, the zero code is used. Such strings are called null-terminated.

   db 'Selling England by the Pound', 0

2.5.3 Constant Precomputation

It is not uncommon to see such code:

lab: db 0
...
   mov rax, lab + 1 + 2*3

NASM supports arithmetic expressions with parentheses and bit operations. Such expressions can only include constants known to the compiler. This way it can precompute all such expressions and insert the computation results (as constant numbers) in executable code. So, such expressions are NOT calculated at runtime.

A runtime analogue would need to use such instructions as add or mul.

2.5.4 Pointers and Different Addressing Types

Pointers are addresses of memory cells. They can be stored in memory or in registers.

The pointer size is 8 bytes. Data usually occupies several memory cells (i.e., several consecutive addresses). The pointers hold no information about the pointed data length. When trying to write somewhere a value whose size is not specified and can not be deduced (for example, mov [myvariable], 4), we can get compilation errors. In such cases we have to provide size explicitly as shown below:

section .data
test: dq -1

section .text

mov byte[test], 1 ;1
mov word[test], 1 ;2
mov dword[test], 1 ;4
mov qword[test], 1 ;8

Let’s see how one can encode operands in instructions .

1. Immediately:

   An instruction is itself contained in memory. The operands in some form are its parts; those parts have addresses of their own. Many instructions can contain the operand values themselves.

   This is the way to move a number 10 into rax.

   mov rax, 10

2. Through a register:

   This instruction transfers rbx value into rax.

   mov rax, rbx

3. By direct memory addressing:

   This instruction transfers 8 bytes starting at the tenth address into rax:

   mov rax, [10]

   We can also take the address from register:

   mov r9, 10
   mov  rax, [r9]

   We can use precomputations:

   buffer: dq 8841, 99, 00
   ...
   mov rax, [buffer+8]

   The address inside this instruction was precomputed, because both base and offset are constants in control of compiler. Now it is just a number.

4. Base-indexed with scale and displacement

   Most addressing modes are generalized by this mode. The address here is calculated based on the following components:

   Address = base + index ∗ scale + displacement

   • Base is either immediate or a register;

   • Scale can only be immediate equal to 1, 2, 4, or 8;

   • Index is immediate or a register; and

   • Displacement is always immediate.

Listing 2-12 shows examples of different addressing types .

Listing 2-12. addressing.asm

mov rax, [rbx + 4* rcx + 9]
mov rax, [4*r9]
mov rdx, [rax + rbx]
lea rax, [rbx + rbx * 4]     ; rax = rbx * 5
add r8, [9 + rbx*8 + 7]

A big picture You can think about byte, word, etc. as about type specifiers. For instance, you can either push 16-, 32-, or 64-bit numbers into the stack. Instruction push 1 is unclear about how many bits wide the operand is. In the same way mov word[test], 1 signifies, that [test] is a word; there is an information about number format encoded in push word 1.

2.7.1 Self-Evaluation

Before testing or when facing an unexpected result, check the following quick list:

1. Labels denoting functions should be global; others should be local.

2. You do not assume that registers hold zero “by default.”

3. You save and restore callee-saved registers if you are using them.

4. You save caller-saved registers you need before call and restore them after.

5. You do not use buffers in .data. Instead, you allocate them on the stack, which allows you to adapt multithreading if needed.

6. Your functions accept arguments in rdi, rsi, rdx, rcx, r8, and r9.

7. You do not print numbers digit after digit. Instead you transform them into strings of characters and use print_string.

8. parse_int and parse_uint are setting rdx correctly. It will be really important in the next assignment.

9. All parsing functions and read_word work when the input is terminated via Ctrl-D.

Done right, the code will not take more than 250 lines.