JIT Compilation

Overview

This lesson explores JIT Compilation (Just-In-Time) within the Android Runtime (ART). Introduced in Android 7.0 (Nougat) to complement AOT compilation, the JIT compiler significantly reduces installation times and storage footprint while maintaining high runtime performance.

JIT Compiler Architecture in ART

ART uses a hybrid compilation strategy. When an application is installed, it is typically not fully AOT compiled. Instead, it starts by running in the interpreter. As the user interacts with the app, the JIT compiler monitors execution and dynamically compiles the most frequently used ("hot") methods into native code in the background.

The JIT architecture consists of several key components:

1. Interpreter: Executes DEX bytecode directly, maintaining profiling counters.
2. JIT Thread Pool: Background worker threads that compile DEX bytecode to native code without blocking the main execution thread.
3. JIT Code Cache: A dedicated memory region where the compiled native code is stored.
4. Garbage Collector Integration: JIT code caches must be managed and cleared when methods are no longer needed or classes are unloaded.

Hot Method Detection and Compilation

The transition from interpreted bytecode to JIT-compiled native code is driven by execution counters.

Profiling and Thresholds

When a method is executed in the interpreter, or when a loop branch is taken (backward branch), ART increments a counter. Once this counter reaches a specific threshold, the method is considered "hot."

// AOSP reference: art/runtime/jit/jit.cc
// Simplified logic for hotness threshold
uint16_t hot_method_threshold_ = 10000; 

void Jit::AddSamples(Thread* self, ArtMethod* method, uint16_t count) {
  uint16_t new_count = method->IncrementCounter(count);
  if (new_count >= hot_method_threshold_) {
    CompileMethod(method);
  }
}

The Compilation Task

When a method crosses the hotness threshold:

1. A compilation task is created and added to the JIT thread pool queue.
2. A background thread picks up the task and uses the Optimizing Compiler to generate native machine code.
3. The compiled code is placed in the JIT Code Cache.
4. The method's entry point is updated. The next time the method is invoked, execution jumps directly to the newly compiled native code instead of the interpreter.

The JIT Code Cache

The JIT Code Cache is a specialized, memory-mapped region used to store dynamically generated native code and associated data (like inline caches and profiling info).

Because this code is generated at runtime, the code cache must be managed carefully to avoid exhausting memory.

• Capacity: The code cache has a maximum capacity (e.g., 64MB or 128MB, depending on the device).
• Code Cache Garbage Collection: When the cache is nearing its limit, ART triggers a specialized garbage collection phase to evict compiled methods that haven't been executed recently. If a method is evicted, subsequent calls fall back to the interpreter until it becomes hot again.

JIT Profiles

A critical feature of ART's JIT compiler is that it doesn't just compile code; it records what it compiled.

As the JIT compiler identifies hot methods and resolves virtual calls (inline caches), it writes this information to a profile file (.prof) stored in the app's private data directory.

# Example location of a JIT profile for an app
adb shell ls -l /data/misc/profiles/cur/0/com.example.app/primary.prof

This profile is persistent. When the device is idle and charging, the background dex2oat daemon reads this profile and uses it to perform AOT compilation only on the methods the user actually uses. This mechanism is known as Profile-Guided Optimization (PGO), which bridges the gap between JIT and AOT paradigms.