> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/zwc456baby/ScrcpyForAndroid/llms.txt
> Use this file to discover all available pages before exploring further.

# Video Decoder

> Hardware-accelerated video decoder using Android MediaCodec for H.264/AVC streams

## Overview

The `VideoDecoder` class provides hardware-accelerated H.264/AVC video decoding using Android's MediaCodec API. It runs in a separate thread and renders decoded frames directly to a Surface.

**Package:** `org.client.scrcpy.decoder`

**Video Codec:** `video/avc` (H.264/AVC)

## Initialization

### start()

Starts the decoder worker thread.

```java theme={null}
public void start()
```

**Example:**

```java theme={null}
VideoDecoder videoDecoder = new VideoDecoder();
videoDecoder.start();
```

### stop()

Stops the decoder and releases resources.

```java theme={null}
public void stop()
```

**Example:**

```java theme={null}
@Override
protected void onDestroy() {
    if (videoDecoder != null) {
        videoDecoder.stop();
    }
    super.onDestroy();
}
```

## Configuration

### configure()

Configures the MediaCodec decoder with stream parameters.

```java theme={null}
public void configure(Surface surface, int width, int height, 
                      ByteBuffer csd0, ByteBuffer csd1)
```

<ParamField path="surface" type="Surface" required>
  Target surface for rendering decoded frames
</ParamField>

<ParamField path="width" type="int" required>
  Video width in pixels
</ParamField>

<ParamField path="height" type="int" required>
  Video height in pixels
</ParamField>

<ParamField path="csd0" type="ByteBuffer" required>
  Codec-specific data 0 (SPS - Sequence Parameter Set)
</ParamField>

<ParamField path="csd1" type="ByteBuffer" required>
  Codec-specific data 1 (PPS - Picture Parameter Set)
</ParamField>

**Example:**

```java theme={null}
VideoPacket.StreamSettings settings = VideoPacket.getStreamSettings(configData);
videoDecoder.configure(surface, 1920, 1080, settings.sps, settings.pps);
```

**Internal Implementation:**

```java theme={null}
// From source (VideoDecoder.java:68-78)
MediaFormat format = MediaFormat.createVideoFormat("video/avc", width, height);
format.setByteBuffer("csd-0", csd0);
format.setByteBuffer("csd-1", csd1);

mCodec = MediaCodec.createDecoderByType("video/avc");
mCodec.configure(format, surface, null, 0);
mCodec.start();
```

<Note>
  Calling `configure()` while already configured will stop the current decoder and create a new one. This allows dynamic reconfiguration for resolution changes.
</Note>

## Decoding

### decodeSample()

Queues an encoded video sample for decoding.

```java theme={null}
public void decodeSample(byte[] data, int offset, int size, 
                         long presentationTimeUs, int flags)
```

<ParamField path="data" type="byte[]" required>
  Byte array containing the encoded video frame
</ParamField>

<ParamField path="offset" type="int" required>
  Starting position in the data array
</ParamField>

<ParamField path="size" type="int" required>
  Number of bytes to decode
</ParamField>

<ParamField path="presentationTimeUs" type="long" required>
  Presentation timestamp in microseconds
</ParamField>

<ParamField path="flags" type="int" required>
  MediaCodec flags (e.g., `MediaCodec.BUFFER_FLAG_KEY_FRAME`)
</ParamField>

**Example:**

```java theme={null}
// Decode a key frame
videoDecoder.decodeSample(
    packet,                              // data
    VideoPacket.getHeadLen(),           // offset
    packet.length - VideoPacket.getHeadLen(), // size
    0,                                   // presentationTimeUs
    VideoPacket.Flag.KEY_FRAME.getFlag() // flags
);
```

**Frame Processing:**

```java theme={null}
// From source (VideoDecoder.java:86-98)
int index = mCodec.dequeueInputBuffer(-1);
if (index >= 0) {
    ByteBuffer buffer;
    
    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP) {
        buffer = mCodec.getInputBuffers()[index];
        buffer.clear();
    } else {
        buffer = mCodec.getInputBuffer(index);
    }
    
    if (buffer != null) {
        buffer.put(data, offset, size);
        mCodec.queueInputBuffer(index, 0, size, presentationTimeUs, flags);
    }
}
```

## VideoPacket Structure

Video data is transmitted using the `VideoPacket` structure.

### Packet Format

**Header (10 bytes):**

```
Byte 0:    Type (0 = VIDEO)
Byte 1:    Flag (frame type)
Bytes 2-9: Presentation timestamp (long)
```

**Total packet structure:**

```
[Type][Flag][Timestamp][Data...]
  1B    1B      8B       Variable
```

### Flag Types

<ResponseField name="FRAME" type="byte">
  Value: `0` - Regular P-frame or B-frame
</ResponseField>

<ResponseField name="KEY_FRAME" type="byte">
  Value: `1` - I-frame (keyframe)
</ResponseField>

<ResponseField name="CONFIG" type="byte">
  Value: `2` - Configuration data (SPS/PPS)
</ResponseField>

<ResponseField name="END" type="byte">
  Value: `4` - End of stream
</ResponseField>

### StreamSettings

Extracted from CONFIG packets containing SPS/PPS data.

```java theme={null}
public static class StreamSettings {
    public ByteBuffer pps;  // Picture Parameter Set
    public ByteBuffer sps;  // Sequence Parameter Set
}
```

**Example:**

```java theme={null}
byte[] configData = new byte[dataLength];
System.arraycopy(packet, VideoPacket.getHeadLen(), configData, 0, dataLength);
VideoPacket.StreamSettings settings = VideoPacket.getStreamSettings(configData);

// Settings contain:
// settings.sps - Sequence Parameter Set
// settings.pps - Picture Parameter Set
```

## Usage Example

Complete example of setting up and using the video decoder:

```java theme={null}
public class VideoPlayer {
    private VideoDecoder videoDecoder;
    private Surface surface;
    
    public void startVideoPlayback() {
        // 1. Initialize decoder
        videoDecoder = new VideoDecoder();
        videoDecoder.start();
        
        // 2. Wait for configuration packet
        VideoPacket configPacket = receivePacket();
        if (configPacket.flag == VideoPacket.Flag.CONFIG) {
            // Extract SPS/PPS
            byte[] configData = extractData(configPacket);
            VideoPacket.StreamSettings settings = 
                VideoPacket.getStreamSettings(configData);
            
            // Configure decoder
            videoDecoder.configure(surface, 1920, 1080, 
                                  settings.sps, settings.pps);
        }
        
        // 3. Decode frames
        while (isPlaying) {
            VideoPacket packet = receivePacket();
            
            if (packet.flag == VideoPacket.Flag.KEY_FRAME || 
                packet.flag == VideoPacket.Flag.FRAME) {
                
                videoDecoder.decodeSample(
                    packet.data,
                    VideoPacket.getHeadLen(),
                    packet.data.length - VideoPacket.getHeadLen(),
                    packet.presentationTimeStamp,
                    packet.flag.getFlag()
                );
            }
            
            if (packet.flag == VideoPacket.Flag.END) {
                break;
            }
        }
        
        // 4. Cleanup
        videoDecoder.stop();
    }
}
```

## Threading Model

The VideoDecoder uses an internal Worker thread:

```java theme={null}
// From source (VideoDecoder.java:104-125)
@Override
public void run() {
    MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
    while (mIsRunning.get()) {
        if (mIsConfigured.get()) {
            int index = mCodec.dequeueOutputBuffer(info, 0);
            if (index >= 0) {
                // Render frame onto Surface
                mCodec.releaseOutputBuffer(index, true);
                
                if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) 
                    == MediaCodec.BUFFER_FLAG_END_OF_STREAM) {
                    break;
                }
            }
        } else {
            // Wait for configuration
            Thread.sleep(5);
        }
    }
}
```

## Performance Considerations

<Warning>
  The decoder uses hardware acceleration. Ensure that:

  * The device supports H.264/AVC hardware decoding
  * The Surface is properly initialized before calling `configure()`
  * Frame rates don't exceed device capabilities
</Warning>

**Key Points:**

* Decoded frames render directly to the Surface (zero-copy)
* Input buffer dequeue timeout is set to `-1` (wait indefinitely)
* Output buffer dequeue timeout is `0` (non-blocking)
* Setting `releaseOutputBuffer(index, true)` renders the frame

## See Also

* [Audio Decoder](/api/audio-decoder) - Audio stream decoding
* [Scrcpy Service](/api/scrcpy-service) - Main service integration
* [Event Controller](/api/event-controller) - Input event handling