VDEX Format

Overview

This lesson dives into the VDEX Format, a crucial component introduced in Android 8.0 (Oreo) to optimize application updates and speed up the verification process. VDEX stands for Verified DEX.

VDEX: Verified DEX

Before VDEX, whenever an application was updated or the system underwent an OTA (Over-The-Air) update, the ART compiler (dex2oat) had to completely re-verify the DEX bytecode. Bytecode verification is a computationally expensive process that ensures the code does not violate memory safety or typing rules.

The VDEX format was introduced to solve this bottleneck. A .vdex file contains the uncompressed DEX files alongside the results of the verification process.

When a system update occurs, if the underlying DEX files have not changed, ART can completely skip the verification phase by reading the previously computed verification metadata from the VDEX file. This drastically reduces the time required for system-wide app optimization (the dreaded "Android is upgrading" screen).

Structure of a VDEX File

The VDEX file consists of several distinct sections:

1. Header: Contains magic numbers (vdex), versioning, and section offsets.
2. DEX Section: The raw, uncompressed .dex files extracted from the APK.
3. Verifier Dependencies: Metadata recording the classes and methods that the verified code depends on. If a framework class changes, ART can use this section to determine if a specific DEX file needs to be re-verified.
4. Type Lookup Table: An optimized data structure to speed up class resolution.

Inline Caches Stored in VDEX

Starting in Android P (9.0), VDEX files took on an additional responsibility: storing Inline Caches.

Inline caching is a runtime optimization technique used to speed up virtual method dispatch. When the JIT compiler or the interpreter executes a method, it observes the actual runtime types of the objects being used. It records this information in a profile.

This profile data, specifically the resolved targets of virtual method calls, is serialized and stored in the VDEX file as an inline cache. When the AOT compiler (dex2oat) runs in the background, it reads these inline caches from the VDEX file to generate highly optimized native code, bypassing the need for slow virtual method lookups at runtime.

Relationship Between DEX, VDEX, and OAT

Understanding the relationship between these three formats is essential for mastering the ART compilation pipeline.

1. DEX (classes.dex): The raw bytecode packaged inside the APK by the developer. It is architecture-independent.
2. VDEX (base.vdex): Generated by dex2oat. Contains the original DEX files, the verification results, and potentially inline caches. It avoids redundant verification during updates.
3. OAT (base.odex or base.art): Generated by dex2oat. Contains the AOT-compiled native machine code (ELF format) specific to the device's architecture (e.g., ARM64).

The Loading Sequence

When an application starts, ART determines how to execute the code based on the presence and status of these files:

• ART maps the OAT file to execute any AOT-compiled native code.
• If a method is not compiled in the OAT file (e.g., it was compiled with the quicken filter or is running interpreted), ART falls back to reading the bytecode from the VDEX file.
• The raw DEX file inside the APK is only extracted and processed if the VDEX and OAT files are missing, invalid, or out of date.

This multi-layered approach allows Android to balance installation speed, storage space, and runtime performance.