Introducing JITWatch

JITWatch is an open source JavaFX tool designed and built by one of this book’s authors (Chris Newland) as a personal project. The tool is now hosted under the AdoptOpenJDK initiative, as part of a program run by the London Java Community to improve the amount of community participation in the Java ecosystem.

JITWatch allows a development or devops team to better understand what HotSpot actually did to the bytecode during the execution of the application. It is possible to tune compilation switches to improve the performance of our applications, but it is essential that we have a mechanism of measuring any improvements we have made.

JITWatch provides objective measurements for comparison. Without these measurements there is a real risk of falling into the Missing the Bigger Picture antipattern discussed in the section “Performance Antipatterns Catalogue”.

To function, JITWatch parses and analyzes the detailed HotSpot compilation log output from the running Java application and displays it in the JavaFX GUI. This means that in order to work the tool requires the application to be run with a certain set of flags.

As well as any flags required for normal runs of the application, the following flags must be added for JITWatch if they are not already present:

-XX:+UnlockDiagnosticVMOptions -XX:+TraceClassLoading -XX:+LogCompilation

With these flags switched on, the JVM will produce a log that can be fed into JITWatch.

Basic JITWatch Views

After the application run has finished, you can start JITWatch, load the log, and produce a view like that shown in Figure 10-1, based on the actual run of the application.

Figure 10-1. JITWatch main window

As well as loading logs from executed programs, JITWatch also provides an environment for experimenting with JIT behaviors called the sandbox. This view allows you to quickly prototype small programs and see the JIT decisions made by the JVM. A sample view is shown in Figure 10-2.

Figure 10-2. JITWatch sandbox

The sandbox workflow allows you to create or load a program written in Java or one of the other supported JVM languages, as an alternative to loading an existing logfile.

When you click the Run button, JITWatch will:

1. Compile your program into bytecode.

2. Execute the program on the JVM with JIT logging enabled.

3. Load the JIT logfile into JITWatch for analysis.

The sandbox is designed to allow fast feedback on the way small changes can affect the optimization choices made by the JVM. As well as Java, you can use the sandbox with Scala, Kotlin, Groovy, and JavaScript (Nashorn) by configuring their paths.

Warning

The sandbox can be incredibly useful, so you should pay attention to the warning presented in the edit window. Always remember that code run inside the sandbox may behave very differently from that in a real application.

The sandbox also allows you to experiment with the VM switches that control the JIT subsystem (see Figure 10-3). For example, using the sandbox configurations setting, you can change JVM JIT behaviors including:

• Outputting disassembled native methods (requires a disassembly binary such as hsdis to be installed into your JRE) and choosing the assembly syntax.

• Overriding the JVM default setting for tiered compilation (using both C1 and C2 JIT compilers)

• Overriding compressed oops usage (disabling this can make the assembly easier to read by removing the address-shifting instructions)

• Disabling on-stack replacement

• Overriding the inlining default limits

Figure 10-3. JITWatch sandbox configuration

Warning

Changing these settings can have significant effects on JVM performance. Altering them is not recommended for a production system apart from in extreme edge cases. Even then they should not be changed without extensive testing.

Another way in which the sandbox differs from a regular run of a JVM application is that in a full-size application the JVM will have the ability to combine optimizations across a larger view of code, rather than just a sandbox snippet. Aggressive JIT compilers (such as C2) may even apply one set of optimizations as a first pass to broaden the scope of program code visible to the optimizer.

For example, the inlining technique collapses methods into their callers. This means that the optimizer can now consider further optimizations (such as more inlining, or other forms) that would not have been obvious before the initial inlining. In turn, this means that a toy application in the sandbox that has only trivial inlining behavior may be treated differently by the JIT compiler than a method from a real application with real inlining.

As a result, most practitioners prefer a more sophisticated view of application compilation than the sandbox. Fortunately, the primary view that JITWatch provides is a very capable view, known as the TriView. This view shows how the source code was compiled into both bytecode and assembly. An example is shown in Figure 10-4; it shows evidence of the JIT compiler removing some unnecessary object allocations—an important optimization in modern JVMs that we will meet later in this chapter.

Figure 10-4. JITWatch TriView

JITWatch allows you to visualize where each compiled method is stored in the code cache. This is a relatively new feature and is still under active development, but the current view is shown in Figure 10-5.

Figure 10-5. JITWatch code cache layout

In Java 8 and earlier, the code cache stores profiled compiled methods, nonprofiled compiled methods, and the VM’s own native code in a single region.

Java 9 introduces a segmented code cache where these different types of native code are stored in different regions to reduce fragmentation and sweeper times and improve the locality of fully compiled code. We’ll explore this feature fully in Chapter 15 when we discuss Java 9 features in depth.

-XX:+PrintAssembly

Introducing JIT Compilation

To complement the view of compiled code from the tool, it is necessary for the performance engineer to have a working knowledge of how the VM collects data and what optimizations it will carry out on an executing program.

We’ve already seen that HotSpot uses profile-guided optimization (PGO) to guide JIT compilation decisions. Under the hood, HotSpot stores profile data about the running program in structures called method data objects, or MDOs for short.

MDOs are used by the bytecode interpreter and the C1 compiler to record the information the JIT compilers use when determining what optimizations to make. MDOs store information such as methods invoked, branches taken, and types observed at call sites.

Counters that record the “hotness” of a profiled property are maintained, and the values in them are decayed during profiling. This ensures that methods are compiled only if they are still hot when they reach the head of the compilation queue.

Once this profiling data has been gathered and the decision has been made to compile, the specific details of the individual compiler take over. The compiler builds an internal representation of the code to be compiled; the exact nature of the representation depends on which compiler (C1 or C2) is in use.

From this internal representation, the compiler will heavily optimize the code. HotSpot’s JIT compilers are able to perform a wide range of modern compiler optimization techniques, including:

• Inlining

• Loop unrolling

• Escape analysis

• Lock elision and lock coarsening

• Monomorphic dispatch

• Intrinsics

• On-stack replacement

In the following sections we will describe each in turn.

As you meet each technique, it is important to recall that the majority of these optimizations are partially or completely dependent on runtime information and support.

The two JIT compilers within HotSpot also use different subsets of the preceding techniques and have different philosophies about how compilation should be approached. In particular, C1 does not engage in speculative optimization. This is the use of optimizations that rely on an unproven assumption about the nature of the execution. Aggressive optimizers (e.g., C2) will make an assumption based on the observed runtime execution and will make an optimization based on that assumption. That simplifying assumption may allow large (sometimes very large) performance speedups to be made.

To protect against the case that the assumption was not actually correct and is later invalidated, a speculative optimization is always protected with a “sanity check” known as a guard. The guard ensures that the assumption still holds, and is checked just before each time the optimized code is run.

If the guard is ever failed, the compiled code is no longer safe and must be removed. HotSpot immediately deoptimizes and demotes the method to interpreted mode to prevent any incorrect code from ever being executed.

Inlining

Inlining is the process of taking a called method (the callee) and copying its contents to the place where it was called from (the call site).

This eliminates the overhead associated with calling a method, which is not large but can include:

• Setting up parameters to be passed

• Looking up the exact method to be called

• Creating new runtime data structures for the new call frame (such as local variables and an evaluation stack)

• Transferring control to the new method

• Possibly returning a result to the caller

Inlining is one of the first optimizations to be applied by the JIT compilers and is known as a gateway optimization, as it brings related code closer together by eliminating method boundaries.

Given the following code:

int result = add(a, b);

private int add(int x, int y) {
    return x + y;
}

The inlining optimization copies the body of the add() method to its call site, effectively giving:

int result = a + b;

HotSpot’s inlining compiler optimization allows the developer to write well-organized and reusable code. It represents the view that a developer should not need to micro-optimize by hand. Instead, HotSpot uses automatic statistical analysis to determine when related code should be brought together. Inlining therefore broadens the horizon for other optimizations, including:

• Escape analysis

• Dead code elimination (DCE)

• Loop unrolling

• Lock elision

Inlining Limits

It is sometimes necessary for the VM to place limits on the inlining subsystem. For example, the VM may need to control:

• The amount of time the JIT compiler spends optimizing a method

• The size of native code produced (and hence its use of the code cache memory)

Without these constraints the compiler could become tied up inlining very deep call chains or filling up the code cache with huge native methods. The general principle—that JIT compilation resources are precious—comes into play again here.

HotSpot considers several factors when determining whether to inline a method, including:

• The bytecode size of the method to be inlined

• The depth of the method to be inlined in the current call chain

• The amount of space in the code cache already taken up by compiled versions of this method

In Figure 10-6 we can see JITWatch visualizing that the JIT compiler has inlined a chain of method calls into their ultimate caller but has rejected inlining a method created deliberately above the maximum default limit.

Figure 10-6. Chain of method calls inlined into the parent method

This depth of inlining is entirely typical and helps broaden the scope of the optimization.

Loop Unrolling

Once any method calls within the loop have been inlined (where possible), the compiler has a better idea of the size and cost of each loop iteration. Based on this, it can consider unrolling the loop to reduce the number of times the execution has to jump back to the beginning.

Each back branch can have undesirable processor effects, as the CPU dumps its pipeline of incoming instructions due to the back branch. In general, the shorter the loop body, the higher the relative cost of the back branch. Therefore, HotSpot makes decisions on whether to unroll loops based on criteria including:

• The type of the loop counter variable (usually int or long rather than an object type)

• The loop stride (how the loop counter changes on each iteration)

• The number of exit points (return or break) within the loop

Consider some example methods that perform summation of data fetched sequentially from an array. This access pattern can be spotted in the assembly as [base, index, offset] addressing:

• base register contains the start address of data in the array.

• index register contains the loop counter (which gets multiplied by the datatype size).

• offset is used for offsetting each unrolled access.

  add rbx, QWORD PTR [base register + index register * size + offset]

Given a loop over a long[] array, we can look at the conditions under which the loop will be unrolled. When a loop accesses an array, HotSpot can eliminate array bounds checks by splitting the loop into three sections, as shown in Table 10-2.

Here is some example setup code to create an array over which the loop will iterate:

    private static final int MAX = 1_000_000;

    private long[] data = new long[MAX];

    private void createData()
    {
        java.util.Random random = new java.util.Random();

        for (int i = 0; i < MAX; i++)
        {
            data[i] = random.nextLong();
        }
    }

We can use a JMH benchmark to compare the performance of iterating over this same array using either an int counter or a long counter:

package optjava.jmh;

import org.openjdk.jmh.annotations.*;
import java.util.concurrent.TimeUnit;

@BenchmarkMode(Mode.Throughput)
@OutputTimeUnit(TimeUnit.SECONDS)
@State(Scope.Thread)
public class LoopUnrollingCounter
{
	private static final int MAX = 1_000_000;

	private long[] data = new long[MAX];

	@Setup
	public void createData()
	{
		java.util.Random random = new java.util.Random();

		for (int i = 0; i < MAX; i++)
		{
			data[i] = random.nextLong();
		}
	}

	@Benchmark
	public long intStride1()
	{
		long sum = 0;
		for (int i = 0; i < MAX; i++)
		{
			sum += data[i];
		}
		return sum;
	}

	@Benchmark
	public long longStride1()
	{
		long sum = 0;
		for (long l = 0; l < MAX; l++)
		{
			sum += data[(int) l];
		}
		return sum;
	}
}

The results are as follows:

Benchmark                          Mode  Cnt     Score   Error  Units
LoopUnrollingCounter.intStride1   thrpt  200  2423.818 ± 2.547  ops/s
LoopUnrollingCounter.longStride1  thrpt  200  1469.833 ± 0.721  ops/s

The loop with the int counter performs nearly 64% more operations per second.

If we were to dive into the assembly, we would see that the body of a loop with a long counter will not be unrolled, and the loop will also contain a safepoint poll. These safepoint checks have been inserted by the JIT compiler to mitigate the possibility that compiled code runs for long periods without checking the safepoint flag (see “JVM Safepoints”).

Other microbenchmarks might include using a variable stride, where the increment is stored in a variable and is not a compile-time constant. With a variable stride, we see that the loop is not unrolled and a safepoint is inserted before the back branch.

Escape Analysis

HotSpot can perform a scope-based analysis for determining if work done within a method is visible or has side effects outside the boundary of that method. This escape analysis technique can be used to determine whether an object allocated within the method is visible outside of the method scope.

During the escape analysis phase HotSpot categorizes potential escapee objects into three types. This piece of code from hotspot/src/share/vm/opto/escape.hpp details the different possible escape scenarios:

  typedef enum {

    NoEscape      = 1, // An object does not escape method or thread and it is
                       // not passed to call. It could be replaced with scalar.

    ArgEscape     = 2, // An object does not escape method or thread but it is
                       // passed as argument to call or referenced by argument
                       // and it does not escape during call.

    GlobalEscape  = 3  // An object escapes the method or thread.
  }

public long noEscape() {
    long sum = 0;

    for (int i = 0; i < 1_000_000; i++) {
        MyObj foo = new MyObj(i); // foo does not escape the method (NoEscape)
        sum += foo.bar();
    }

    return sum;
}

public long argEscape() {
    long sum = 0;

    for (int i = 0; i < 1_000_000; i++) {
        MyObj foo = new MyObj(i);
        sum += extBar(foo); // foo is passed as an argument to extBar (ArgEscape)
    }

    return sum;
}

Locks and Escape Analysis

HotSpot is able to use escape analysis and some related techniques to optimize the performance of locks.

Note

This applies only to intrinsic locks (those that use synchronized). The locks from java.util.concurrent are not eligible for these optimizations.

The key lock optimizations that are available are:

• Removing locks on nonescaping objects (lock elision)

• Merging consecutive locked regions that share the same lock (lock coarsening)

• Detecting blocks where the same lock is acquired repeatedly without an unlock (nested locks)

When consecutive locks on the same object are encountered, HotSpot will check if it is possible to enlarge the locked region. To achieve this, when HotSpot encounters a lock it will search backward to try to find an unlock on the same object. If a match is found, then it will consider whether the two lock regions can be joined to produce a single larger region.

You can read more about this in the JVM specification. In Figure 10-7 we can see this effect directly in JITWatch.

Figure 10-7. Lock coarsening

The lock coarsening optimization is enabled by default, but you can disable it using the VM switch -XX:-EliminateLocks to see its impact.

HotSpot is also able to detect nested locks that lock on the same object and remove the inner locks since the lock will already have been acquired by the thread.

Tip

At the time of writing, the nested lock elimination in Java 8 appears to work with locks that are declared as static final and with locks on this.

The nested lock optimization is enabled by default but you can disable it using the VM switch -XX:-EliminateNestedLocks. Nested lock detection in JITWatch can be seen in Figure 10-8.

Figure 10-8. A nested lock is eliminated

HotSpot automatically calculates when it is safe to coarsen or eliminate locks. Tools such as JITWatch can visualize where locks have been optimized, and if you use a debug JVM you can output extra information on locks.

-XX:EliminateAllocationArraySizeLimit=<n>

package optjava.jmh;

import org.openjdk.jmh.annotations.*;
import java.util.concurrent.TimeUnit;

@State(Scope.Thread)
@BenchmarkMode(Mode.Throughput)
@OutputTimeUnit(TimeUnit.SECONDS)
public class EscapeTestArraySize {

    private java.util.Random random = new java.util.Random();

    @Benchmark
    public long arraySize63() {
        int[] a = new int[63];

        a[0] = random.nextInt();
        a[1] = random.nextInt();

        return a[0] + a[1];
    }

    @Benchmark
    public long arraySize64() {
        int[] a = new int[64];

        a[0] = random.nextInt();
        a[1] = random.nextInt();

        return a[0] + a[1];
    }

    @Benchmark
    public long arraySize65() {
        int[] a = new int[65];

        a[0] = random.nextInt();
        a[1] = random.nextInt();

        return a[0] + a[1];
    }
}

Benchmark                         Mode  Cnt         Score       Error  Units
EscapeTestArraySize.arraySize63  thrpt  200  49824186.696 ±  9366.780  ops/s
EscapeTestArraySize.arraySize64  thrpt  200  49815447.849 ±  2328.928  ops/s
EscapeTestArraySize.arraySize65  thrpt  200  21115003.388 ± 34005.817  ops/s

$ java -XX:EliminateAllocationArraySizeLimit=65 -jar target/benchmarks.jar

Benchmark                         Mode  Cnt         Score      Error  Units
EscapeTestArraySize.arraySize63  thrpt  200  49814492.787 ± 2283.941  ops/s
EscapeTestArraySize.arraySize64  thrpt  200  49815595.566 ± 5833.359  ops/s
EscapeTestArraySize.arraySize65  thrpt  200  49818143.279 ± 2347.695  ops/s

for (int i = 0; i < 100_000_000; i++) {
    Object mightEscape = new Object(i);

    if (condition) {
        result += inlineableMethod(mightEscape);
    } else {
        result += tooBigToInline(mightEscape);
    }
}

for (int i = 0; i < 100_000_000; i++) {

    if (condition) {
        Object mightEscape = new Object(i);
        result += inlineableMethod(mightEscape);
    } else {
        Object mightEscape = new Object(i);
        result += tooBigToInline(mightEscape);
    }
}

Monomorphic Dispatch

Many of the speculative optimizations that HotSpot’s C2 compiler undertakes are based on empirical research. One example is the technique called monomorphic dispatch (monomorphic is derived from the Greek for “single form”).

It relies on a strange, but powerful, observed fact: in human-authored code, very often only one runtime type is ever observed to be the type of the receiver object at any individual call site.

That is, when we’re calling a method on an object, if we examine the runtime type of that object the first time we call that method, it is quite likely to always be the same type for every subsequent call.

If that speculative assumption is true, then the call to the method being invoked at that site can be optimized. In particular, the indirection of looking up methods in vtables can be eliminated. If it’s always the same type, then we can compute the call target once and replace the invokevirtual instruction with a quick type test (the guard) and then a branch to the compiled method body.

Put another way, the virtual lookup and associated indirection through the klass pointer and vtable only needs to be done once and can be cached for future invocations at that call site.

Consider the following piece of code:

java.util.Date date = getDate();
System.out.println(date.toInstant());

If the getDate() method always returns an instance of java.util.Date, then the call to toInstant() can be assumed to be monomorphic. However, if after many iterations of this code, getDate() suddenly returns an instance of java.sql.Date, then the monomorphic assumption is no longer valid, as an entirely different implementation of toInstant() must now be called.

HotSpot’s solution is to back out of the optimization and revert the call site to using full virtual dispatch. The guard that is used to protect the monomorphic call is very straightforward: it is a simple equality of klass words, and it is checked before every call instruction to ensure that no incorrect code is executed.

A very large number of calls in a typical application will be monomorphic. HotSpot also has another optimization, which is more rarely useful, called bimorphic dispatch. This allows two different types to be handled in a similar way to the monomorphic case, by caching two different klass words per call site.

Call sites that are not either monomorphic or bimorphic are known as megamorphic (Greek for “many forms”). If you find that you have a megamorphic call site with only a small number of observed types, you can use a trick to regain some performance. It works by “peeling” off types from the original call site using instanceof checks so that you leave only a bimorphic call site that observes two concrete types.

An example of this can be seen here:

package optjava.jmh;

import org.openjdk.jmh.annotations.*;
import java.util.concurrent.TimeUnit;

interface Shape {
	int getSides();
}

class Triangle implements Shape {
	public int getSides() {
		return 3;
	}
}

class Square implements Shape {
	public int getSides() {
		return 4;
	}
}

class Octagon implements Shape {
	public int getSides() {
		return 8;
	}
}

@State(Scope.Thread)
@BenchmarkMode(Mode.Throughput)
@OutputTimeUnit(TimeUnit.SECONDS)
public class PeelMegamorphicCallsite {

	private java.util.Random random = new java.util.Random();

	private Shape triangle = new Triangle();
	private Shape square = new Square();
	private Shape octagon = new Octagon();

	@Benchmark
	public int runBimorphic() {
		Shape currentShape = null;

		switch (random.nextInt(2))
		{
		case 0:
			currentShape = triangle;
			break;
		case 1:
			currentShape = square;
			break;
		}

		return currentShape.getSides();
	}

	@Benchmark
	public int runMegamorphic() {
		Shape currentShape = null;

		switch (random.nextInt(3))
		{
		case 0:
			currentShape = triangle;
			break;
		case 1:
			currentShape = square;
			break;
		case 2:
			currentShape = octagon;
			break;
		}

		return currentShape.getSides();
	}

	@Benchmark
	public int runPeeledMegamorphic() {
		Shape currentShape = null;

		switch (random.nextInt(3))
		{
		case 0:
			currentShape = triangle;
			break;
		case 1:
			currentShape = square;
			break;
		case 2:
			currentShape = octagon;
			break;
		}

		// peel one observed type from the original call site
		if (currentShape instanceof Triangle) {
			return ((Triangle) currentShape).getSides();
		}
		else {
			return currentShape.getSides(); // now only bimorphic
		}
	}
}

Running this benchmark produces the following output:

Benchmark                                  Mode  Cnt     Score    Error  Units
PeelMega...Callsite.runBimorphic          thrpt  200  75844310 ±  43557  ops/s
PeelMega...Callsite.runMegamorphic        thrpt  200  54650385 ±  91283  ops/s
PeelMega...Callsite.runPeeledMegamorphic  thrpt  200  62021478 ± 150092  ops/s

When two implementations are observed at the call site, bimorphic inlining occurs. This performs around 38% more operations per second than the megamorphic call site, which observes three implementations and where method dispatch remains a virtual call. Note that this is not a fair comparison, as the code is behaviorally different.

When one of the observed types is peeled off into a different call site, the program performs around 13% more operations per second than the megamorphic code.

The subject of method dispatch, and the resulting performance implications, is a deep one. Aleksey Shipilëv gives a master class in his blog post “The Black Magic of (Java) Method Dispatch”.

Intrinsics

An intrinsic is the name given to a highly tuned native implementation of a method that is preknown to the JVM rather than generated dynamically by the JIT subsystem. They are used for performance-critical core methods where the functionality is supported by specific features of the operating system or CPU architecture. This makes them platform-specific, and some intrinsics may not be supported on every platform.

When the JVM starts up, the CPU is probed at runtime and a list of the available processor features is built. This means that the decision about which optimizations to use can be deferred until runtime, and does not have to be made at code compilation time.

Examples of some common intrinsics are shown in Table 10-3.

The intrinsic templates are available to view in the HotSpot source code of OpenJDK. They are contained within .ad files (the suffix stands for “architecture dependent”).

For the x86_64 architecture, they can be found in the file hotspot/src/cpu/x86/vm/x86_64.ad.

For example, to calculate the base 10 logarithm of a number, we can use the method from java.lang.Math:

public static double log10(double a)

On the x86_64 architecture this calculation can be performed with two instructions of the floating-point unit (FPU):

1. Calculate the base 10 logarithm of the constant 2.

2. Multiply this by the base 2 logarithm of your argument.

The intrinsic code for this is:

instruct log10D_reg(regD dst) %{
  // The source and result Double operands in XMM registers
  // match(Set dst (Log10D dst));
  // fldlg2       ; push log_10(2) on the FPU stack; full 80-bit number
  // fyl2x        ; compute log_10(2) * log_2(x)
  format %{ "fldlg2\t\t\t#Log10\n\t"
            "fyl2x\t\t\t# Q=Log10*Log_2(x)\n\t"
         %}
   ins_encode(Opcode(0xD9), Opcode(0xEC),
              Push_SrcXD(dst),

              Opcode(0xD9), Opcode(0xF1),
              Push_ResultXD(dst));
  ins_pipe( pipe_slow );
%}

When considering adding a new intrinsic, you will need to evaluate the tradeoff of the additional complexity versus the likelihood of the intrinsic being useful.

For example, we could conceive of intrinsics that perform basic arithmetic identities, such as the sum of the first n numbers. Traditional Java code would require O(n) operations to calculate this value, but it has a trivial formula that will calculate it in O(1) time.

Should we implement an intrinsic to calculate the sum in constant time?

The answer depends on how many classes observed in the wild have a need to calculate such a sum—in this case, not many. Such an intrinsic would therefore clearly be of limited utility and almost certainly not worth the extra complexity in the JVM.

This underlines the point that intrinsics can have a large performance impact only for those operations that are genuinely seen frequently in real code.

On-Stack Replacement

Sometimes you will encounter code that contains a hot loop within a method that is not called enough times to trigger a compilation—for example, in a Java program’s main() method.

HotSpot can still optimize this code using a technique called on-stack replacement (OSR). This trick counts the loop back branches in the interpreter; when these cross a threshold, the interpreted loop will be compiled and execution will switch to this compiled version.

The compiler is responsible for making sure any state changes, such as local variables and locks that were accessed within the interpreted loop, are made available to the compiled version. Once the compiled loop exits, all state changes must be visible at the point where execution continues.

For example, if we have this hot loop inside the main() method:

package optjava;

public class OnStackReplacement {
	// method called once
	public static void main(String[] args) {
		java.util.Random r = new java.util.Random();

		long sum = 0;

		// first long-running loop
		for (int i = 0; i < 1_000_000; i++) {
			sum += r.nextInt(100);
		}

		// second long-running loop
		for (int i = 0; i < 1_000_000; i++) {
			sum += r.nextInt(100);
		}

		System.out.println(sum);
	}
}

Then the method bytecode will look like this:

 public static void main(java.lang.String[]);
  descriptor: ([Ljava/lang/String;)V
  flags: ACC_PUBLIC, ACC_STATIC
  Code:
    stack=4, locals=5, args_size=1
     0: new           #2      // class java/util/Random
     3: dup
     4: invokespecial #3      // Method java/util/Random."<init>":()V
     7: astore_1
     8: lconst_0
     9: lstore_2
    10: iconst_0
    11: istore        4
    13: iload         4
    15: ldc           #4      // int 1000000
    17: if_icmpge     36
    20: lload_2
    21: aload_1
    22: bipush        100
    24: invokevirtual #5      // Method java/util/Random.nextInt:(I)I
    27: i2l
    28: ladd
    29: lstore_2
    30: iinc          4, 1
    33: goto          13
    36: getstatic     #6      // Field java/lang/System.out:Ljava/io/PrintStream;
    39: lload_2
    40: invokevirtual #7      // Method java/io/PrintStream.println:(J)V
    43: return

The goto bytecode at index 33 returns the flow to the loop check at index 13.

HotSpot can perform OSR compilation using both the C1 and C2 JIT compilers.

JITWatch can highlight which loop was OSR-compiled in the bytecode and source code, as can be seen in Figure 10-9.

Figure 10-9. On-stack replacement

Safepoints Revisited

Before we leave the topic of JIT compilation, it makes sense to bring together a list of all the conditions in the JVM that require the VM to be at a safepoint. As well as GC STW events, the following activities require all threads to be at a safepoint:

• Deoptimizing a method

• Creating a heap dump

• Revoking a biased lock

• Redefining a class (e.g., for instrumentation)

In compiled code, the JIT compiler is responsible for emitting safepoint checks (as seen previously with loop unrolling) and in HotSpot it will generate them:

• At Loop back branches

• On Method return

This means that sometimes threads may take a certain amount of time to safepoint (e.g., if they are executing a loop containing a lot of arithmetic code without any method calls). If the loop is unrolled, then there may be an appreciable time before a safepoint is encountered.

The compiler will try to balance the cost of safepoint polling checks with avoiding long Time to SafePoint (TTSP) durations where threads already at their safepoint have to wait for other threads to reach theirs.

We will discuss safepoints again in Chapter 13, as they are a cross-cutting concern that has an impact on many of the JVM subsystems.

Core Library Methods

To conclude this chapter, let’s take a quick look at some of the impacts of the size of JDK core library methods on JIT compilation.

$ ./jarScan.sh --mode=maxMethodSize \
               --limit=325 \
               --packages=java.* \
               /path/to/java/jre/lib/rt.jar

public String toUpperCase(Locale locale) {
      if (locale == null) {
          throw new NullPointerException();
      }

      int firstLower;
      final int len = value.length;

      /* Now check if there are any characters that need to be changed. */
      scan: {
          for (firstLower = 0 ; firstLower < len; ) {
              int c = (int)value[firstLower];
              int srcCount;
              if ((c >= Character.MIN_HIGH_SURROGATE)
                      && (c <= Character.MAX_HIGH_SURROGATE)) {
                  c = codePointAt(firstLower);
                  srcCount = Character.charCount(c);
              } else {
                  srcCount = 1;
              }
              int upperCaseChar = Character.toUpperCaseEx(c);
              if ((upperCaseChar == Character.ERROR)
                      || (c != upperCaseChar)) {
                  break scan;
              }
              firstLower += srcCount;
          }
          return this;
      }

      /* result may grow, so i+resultOffset is the write location in result */
      int resultOffset = 0;
      char[] result = new char[len]; /* may grow */

      /* Just copy the first few upperCase characters. */
      System.arraycopy(value, 0, result, 0, firstLower);

      String lang = locale.getLanguage();
      boolean localeDependent =
              (lang == "tr" || lang == "az" || lang == "lt");
      char[] upperCharArray;
      int upperChar;
      int srcChar;
      int srcCount;
      for (int i = firstLower; i < len; i += srcCount) {
          srcChar = (int)value[i];
          if ((char)srcChar >= Character.MIN_HIGH_SURROGATE &&
              (char)srcChar <= Character.MAX_HIGH_SURROGATE) {
              srcChar = codePointAt(i);
              srcCount = Character.charCount(srcChar);
          } else {
              srcCount = 1;
          }
          if (localeDependent) {
              upperChar = ConditionalSpecialCasing.
                                     toUpperCaseEx(this, i, locale);
          } else {
              upperChar = Character.toUpperCaseEx(srcChar);
          }
          if ((upperChar == Character.ERROR)
                  || (upperChar >= Character.MIN_SUPPLEMENTARY_CODE_POINT)) {
              if (upperChar == Character.ERROR) {
                  if (localeDependent) {
                      upperCharArray =
                              ConditionalSpecialCasing.
                                         toUpperCaseCharArray(this, i, locale);
                  } else {
                      upperCharArray = Character.toUpperCaseCharArray(srcChar);
                  }
              } else if (srcCount == 2) {
                  resultOffset += Character.toChars(upperChar,
                                                    result, i + resultOffset)
                                                    - srcCount;
                  continue;
              } else {
                  upperCharArray = Character.toChars(upperChar);
              }

              /* Grow result if needed */
              int mapLen = upperCharArray.length;
              if (mapLen > srcCount) {
                  char[] result2 = new char[result.length + mapLen - srcCount];
                  System.arraycopy(result, 0, result2, 0, i + resultOffset);
                  result = result2;
              }
              for (int x = 0; x < mapLen; ++x) {
                  result[i + resultOffset + x] = upperCharArray[x];
              }
              resultOffset += (mapLen - srcCount);
          } else {
              result[i + resultOffset] = (char)upperChar;
          }
      }
      return new String(result, 0, len + resultOffset);
  }

package optjava.jmh;

import org.openjdk.jmh.annotations.*;
import java.util.concurrent.TimeUnit;

@State(Scope.Thread)
@BenchmarkMode(Mode.Throughput)
@OutputTimeUnit(TimeUnit.SECONDS)
public class DomainSpecificUpperCase {

    private static final String SOURCE =
          "The quick brown fox jumps over the lazy dog";

    public String toUpperCaseASCII(String source) {
        int len = source.length();
        char[] result = new char[len];
        for (int i = 0; i < len; i++) {
            char c = source.charAt(i);
            if (c >= 'a' && c <= 'z') {
                c -= 32;
            }
            result[i] = c;
        }
        return new String(result);
    }

    @Benchmark
    public String testStringToUpperCase() {
        return SOURCE.toUpperCase();
    }

    @Benchmark
    public String testCustomToUpperCase() {
        return toUpperCaseASCII(SOURCE);
    }
}

Benchmark                                   Mode  Cnt     Score   Error  Units
DomainS...UpperCase.testCustomToUpperCase  thrpt  200  20138368 ± 17807  ops/s
DomainS...UpperCase.testStringToUpperCase  thrpt  200   8350400 ±  7199  ops/s

./jarScan.sh --mode=maxMethodSize --limit=8000 /path/to/java/jre/lib/rt.jar

private java.util.Random r = new java.util.Random();

@Benchmark
public long lessThan8000() {
    return r.nextInt() +
           r.nextInt() +
    ... // for a total method size of just under 8000 bytes of bytecode
}

@Benchmark
public long moreThan8000() {
    return r.nextInt() +
           r.nextInt() +
    ... // for a total method size of just over 8000 bytes of bytecode
}

Benchmark                 Mode  Cnt      Score     Error  Units
HugeMethod.lessThan8000  thrpt  100  89550.631 ±  77.703  ops/s
HugeMethod.moreThan8000  thrpt  100  44429.392 ± 102.076  ops/s

Summary

In this chapter, we have covered the basics of HotSpot’s JIT compilation subsystem and examined how some of its optimizations are performed. We have looked at the parameters that can be used to tune the JIT compilers and inspected some of the assembly produced.

In terms of actionable, practical techniques, we can use the -XX:+PrintCompilation flag and the technique introduced in the last chapter to confirm the optimization of individual methods. The general principle of “write good code first, optimize only when required” definitely applies here. A knowledge of the limits for inlining and other compilation thresholds will allow developers to refactor and stay within those limits (or, in rare cases, alter the thresholds). Understanding the existence of monomorphic dispatch and type sharpening means that applications can be written according to a classic principle: design for interfaces, even if there’s just a single implementation. This gives the best of two approaches—the class can still be dummied up or mocked for testing, but the single implementation preserves monomorphic dispatch.