Codec2 Framework

Overview

The Codec2 Framework is Android's modern, low-overhead multimedia plugin architecture. Introduced in Android 10, it replaces the legacy OMX (OpenMAX) layer within the Stagefright media engine. Codec2 was designed from the ground up to support zero-copy buffer passing, hardware-accelerated video pipelines, and treble-compliant IPC (Inter-Process Communication).

Codec2 Architecture Replaces OMX

The legacy OMX framework had several limitations, most notably its complex state machine and high overhead when passing buffers across processes. Codec2 simplifies this by aligning closely with how modern hardware accelerators actually operate.

Key Architectural Improvements:

• Zero-Copy: Codec2 heavily utilizes GraphicBuffer and Ashmem to ensure that raw video frames and audio PCM data are never needlessly copied between the CPU, GPU, and Video Processing Unit (VPU).
• Treble Compliance: Codec2 is designed natively as a HIDL/AIDL service, cleanly separating the Android media framework from vendor-specific decoder implementations.
• Stateless Operation: Unlike OMX, which maintained strict internal states, Codec2 components are largely stateless. They accept a C2Work item and produce a C2Work item, making pipeline management much simpler.

C2Component and C2Interface

A vendor implementing a hardware decoder must implement the Codec2 interfaces.

1. C2Component: The actual workhorse. It processes input buffers and generates output buffers. A component could be an H.264 decoder or an AAC encoder.
2. C2Interface: The configuration layer. It exposes the component's capabilities (supported profiles, levels, color formats) and allows the framework to set parameters (target bitrate, framerate) independently of data processing.

Codec2 Work Items

The core unit of execution in Codec2 is the C2Work structure.

// A simplified view of C2Work processing
struct C2Work {
    std::list<std::unique_ptr<C2Worklet>> worklets;
    // ... metadata and tuning parameters
};

struct C2Worklet {
    C2FrameData input;  // The compressed NAL unit
    C2FrameData output; // The resulting YUV frame (populated by the component)
};

When MediaCodec wants to decode a frame, it encapsulates the compressed data into a C2Work object and queues it to the C2Component. When the hardware finishes, it populates the output field with a reference to a GraphicBuffer and returns the C2Work to the framework.

Software and Hardware Codecs

Codec2 unifies how both software (Google-provided) and hardware (Vendor-provided) codecs are handled.

• Software Codecs: Google provides highly optimized software codecs (e.g., c2.android.avc.decoder) running in the mediaswcodec process.
• Hardware Codecs: Vendors provide hardware-accelerated codecs (e.g., c2.qti.avc.decoder) running in the media.codec process.

The framework determines which codec to instantiate based on the media_codecs.xml configuration file, which defines the priority and capabilities of each component.

Buffer Management

Codec2 relies on C2BlockPool for memory allocation. For video decoding, the framework usually creates a block pool backed by a BufferQueue tied to a Surface. When the hardware decoder finishes a frame, it directly writes the YUV data into the GraphicBuffer allocated by SurfaceFlinger, meaning the CPU never touches the raw video pixels.

You can inspect active Codec2 instances and their buffer queues using:

adb shell dumpsys media.player

This will reveal the underlying C2Component being used by any active MediaCodec instances.