> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/zwc456baby/ScrcpyForAndroid/llms.txt
> Use this file to discover all available pages before exploring further.

# Audio Decoder

> Hardware-accelerated audio decoder using Android MediaCodec for AAC audio streams

## Overview

The `AudioDecoder` class provides hardware-accelerated AAC audio decoding using Android's MediaCodec API and plays the decoded PCM audio through AudioTrack. It runs in a separate thread for continuous audio playback.

**Package:** `org.client.scrcpy.decoder`

**Audio Codec:** `audio/mp4a-latm` (AAC-LC)

**Sample Rate:** 48000 Hz

**Channels:** 2 (Stereo)

**Encoding:** PCM 16-bit

## Initialization

### start()

Starts the decoder worker thread.

```java theme={null}
public void start()
```

**Example:**

```java theme={null}
AudioDecoder audioDecoder = new AudioDecoder();
audioDecoder.start();
```

### stop()

Stops the decoder, AudioTrack, and releases resources.

```java theme={null}
public void stop()
```

**Example:**

```java theme={null}
@Override
protected void onDestroy() {
    if (audioDecoder != null) {
        audioDecoder.stop();
    }
    super.onDestroy();
}
```

## Configuration

### configure()

Configures the MediaCodec decoder with AAC codec-specific data.

```java theme={null}
public void configure(byte[] data)
```

<ParamField path="data" type="byte[]" required>
  Codec-specific data (csd-0) containing AAC configuration
</ParamField>

**Example:**

```java theme={null}
// Receive CONFIG packet
AudioPacket configPacket = AudioPacket.readHead(packet);
if (configPacket.flag == AudioPacket.Flag.CONFIG) {
    byte[] configData = new byte[dataLength];
    System.arraycopy(packet, AudioPacket.getHeadLen(), 
                    configData, 0, dataLength);
    audioDecoder.configure(configData);
}
```

**Internal Implementation:**

```java theme={null}
// From source (AudioDecoder.java:90-109)
MediaFormat format = MediaFormat.createAudioFormat(
    MIMETYPE_AUDIO_AAC, // "audio/mp4a-latm"
    SAMPLE_RATE,        // 48000
    2                   // stereo channels
);

// Set bit rate
format.setInteger(MediaFormat.KEY_BIT_RATE, 128000);

// Set codec-specific data
format.setByteBuffer("csd-0", ByteBuffer.wrap(data));

mCodec = MediaCodec.createDecoderByType(MIMETYPE_AUDIO_AAC);
mCodec.configure(format, null, null, 0);
mCodec.start();

// Initialize and start AudioTrack
initAudioTrack();
audioTrack.play();
```

<Note>
  Calling `configure()` while already configured will stop the current decoder and AudioTrack, then create new instances. This allows dynamic reconfiguration.
</Note>

## Decoding

### decodeSample()

Queues an encoded audio sample for decoding and playback.

```java theme={null}
public void decodeSample(byte[] data, int offset, int size, 
                         long presentationTimeUs, int flags)
```

<ParamField path="data" type="byte[]" required>
  Byte array containing the encoded audio frame
</ParamField>

<ParamField path="offset" type="int" required>
  Starting position in the data array
</ParamField>

<ParamField path="size" type="int" required>
  Number of bytes to decode
</ParamField>

<ParamField path="presentationTimeUs" type="long" required>
  Presentation timestamp in microseconds
</ParamField>

<ParamField path="flags" type="int" required>
  MediaCodec flags
</ParamField>

**Example:**

```java theme={null}
AudioPacket audioPacket = AudioPacket.readHead(packet);
if (audioPacket.flag == AudioPacket.Flag.FRAME || 
    audioPacket.flag == AudioPacket.Flag.KEY_FRAME) {
    
    audioDecoder.decodeSample(
        packet,
        AudioPacket.getHeadLen(),
        packet.length - AudioPacket.getHeadLen(),
        audioPacket.presentationTimeStamp,
        audioPacket.flag.getFlag()
    );
}
```

**Frame Processing:**

```java theme={null}
// From source (AudioDecoder.java:116-129)
int index = mCodec.dequeueInputBuffer(-1);
if (index >= 0) {
    ByteBuffer buffer;
    
    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP) {
        buffer = mCodec.getInputBuffers()[index];
        buffer.clear();
    } else {
        buffer = mCodec.getInputBuffer(index);
    }
    
    if (buffer != null) {
        buffer.put(data, offset, size);
        mCodec.queueInputBuffer(index, 0, size, presentationTimeUs, flags);
    }
}
```

## AudioTrack Integration

The decoder automatically manages an AudioTrack instance for audio playback.

### AudioTrack Configuration

```java theme={null}
// From source (AudioDecoder.java:28-32)
private void initAudioTrack() {
    int bufferSizeInBytes = AudioTrack.getMinBufferSize(
        SAMPLE_RATE,                      // 48000 Hz
        AudioFormat.CHANNEL_OUT_STEREO,   // 2 channels
        AudioFormat.ENCODING_PCM_16BIT    // 16-bit PCM
    );
    
    audioTrack = new AudioTrack(
        AudioManager.STREAM_MUSIC,
        SAMPLE_RATE,
        AudioFormat.CHANNEL_OUT_STEREO,
        AudioFormat.ENCODING_PCM_16BIT,
        bufferSizeInBytes,
        AudioTrack.MODE_STREAM
    );
}
```

### Audio Playback Loop

```java theme={null}
// From source (AudioDecoder.java:140-157)
int index = mCodec.dequeueOutputBuffer(info, 0);
if (index >= 0) {
    if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) 
        == MediaCodec.BUFFER_FLAG_END_OF_STREAM) {
        break;
    }
    
    // Read decoded PCM data
    ByteBuffer outputBuffer = mCodec.getOutputBuffer(index);
    if (outputBuffer != null) {
        byte[] data = new byte[info.size];
        outputBuffer.get(data);
        outputBuffer.clear();
        
        // Write to AudioTrack for playback
        audioTrack.write(data, 0, info.size);
    }
    
    // Release output buffer
    mCodec.releaseOutputBuffer(index, true);
}
```

## AudioPacket Structure

Audio data is transmitted using the `AudioPacket` structure.

### Packet Format

**Header (10 bytes):**

```
Byte 0:    Type (1 = AUDIO)
Byte 1:    Flag (frame type)
Bytes 2-9: Presentation timestamp (long)
```

**Total packet structure:**

```
[Type][Flag][Timestamp][Data...]
  1B    1B      8B       Variable
```

### Flag Types

<ResponseField name="FRAME" type="byte">
  Value: `0` - Regular audio frame
</ResponseField>

<ResponseField name="KEY_FRAME" type="byte">
  Value: `1` - Key audio frame
</ResponseField>

<ResponseField name="CONFIG" type="byte">
  Value: `2` - Configuration data (codec-specific)
</ResponseField>

<ResponseField name="END" type="byte">
  Value: `4` - End of audio stream
</ResponseField>

### Reading Packets

```java theme={null}
// Read packet header only
AudioPacket audioPacket = AudioPacket.readHead(packet);

// Access packet properties
MediaPacket.Type type = audioPacket.type;              // AUDIO
AudioPacket.Flag flag = audioPacket.flag;             // FRAME, CONFIG, etc.
long timestamp = audioPacket.presentationTimeStamp;    // microseconds

// Extract data from full packet
int dataLength = packet.length - AudioPacket.getHeadLen();
byte[] audioData = new byte[dataLength];
System.arraycopy(packet, AudioPacket.getHeadLen(), audioData, 0, dataLength);
```

## Usage Example

Complete example of setting up and using the audio decoder:

```java theme={null}
public class AudioPlayer {
    private AudioDecoder audioDecoder;
    
    public void startAudioPlayback() {
        // 1. Initialize decoder
        audioDecoder = new AudioDecoder();
        audioDecoder.start();
        
        // 2. Wait for configuration packet
        byte[] packet = receiveAudioPacket();
        AudioPacket configPacket = AudioPacket.readHead(packet);
        
        if (configPacket.flag == AudioPacket.Flag.CONFIG) {
            // Extract configuration data
            int dataLength = packet.length - AudioPacket.getHeadLen();
            byte[] configData = new byte[dataLength];
            System.arraycopy(packet, AudioPacket.getHeadLen(), 
                           configData, 0, dataLength);
            
            // Configure decoder
            audioDecoder.configure(configData);
        }
        
        // 3. Decode and play audio frames
        while (isPlaying) {
            packet = receiveAudioPacket();
            AudioPacket audioPacket = AudioPacket.readHead(packet);
            
            if (audioPacket.flag == AudioPacket.Flag.FRAME || 
                audioPacket.flag == AudioPacket.Flag.KEY_FRAME) {
                
                audioDecoder.decodeSample(
                    packet,
                    AudioPacket.getHeadLen(),
                    packet.length - AudioPacket.getHeadLen(),
                    audioPacket.presentationTimeStamp,
                    audioPacket.flag.getFlag()
                );
            }
            
            if (audioPacket.flag == AudioPacket.Flag.END) {
                break;
            }
        }
        
        // 4. Cleanup
        audioDecoder.stop();
    }
}
```

## Threading Model

The AudioDecoder uses an internal Worker thread:

```java theme={null}
// From source (AudioDecoder.java:134-167)
@Override
public void run() {
    try {
        MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
        while (mIsRunning.get()) {
            if (mIsConfigured.get()) {
                int index = mCodec.dequeueOutputBuffer(info, 0);
                if (index >= 0) {
                    // Check for end of stream
                    if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) 
                        == MediaCodec.BUFFER_FLAG_END_OF_STREAM) {
                        break;
                    }
                    
                    // Decode and play PCM audio
                    ByteBuffer outputBuffer = mCodec.getOutputBuffer(index);
                    if (outputBuffer != null) {
                        byte[] data = new byte[info.size];
                        outputBuffer.get(data);
                        outputBuffer.clear();
                        audioTrack.write(data, 0, info.size);
                    }
                    
                    mCodec.releaseOutputBuffer(index, true);
                }
            } else {
                // Wait for configuration
                Thread.sleep(5);
            }
        }
    } catch (IllegalStateException e) {
        // Handle codec errors
    }
}
```

## Audio Specifications

<ResponseField name="MIMETYPE_AUDIO_AAC" type="String">
  Value: `"audio/mp4a-latm"` - AAC Low Complexity profile
</ResponseField>

<ResponseField name="SAMPLE_RATE" type="int">
  Value: `48000` - 48 kHz sampling rate
</ResponseField>

<ResponseField name="CHANNELS" type="int">
  Value: `2` - Stereo output
</ResponseField>

<ResponseField name="BIT_RATE" type="int">
  Value: `128000` - 128 kbps encoding bitrate
</ResponseField>

<ResponseField name="PCM_ENCODING" type="int">
  Value: `AudioFormat.ENCODING_PCM_16BIT` - 16-bit PCM output
</ResponseField>

## Performance Considerations

<Warning>
  The decoder uses hardware acceleration and real-time audio playback. Ensure that:

  * The device supports AAC hardware decoding
  * Audio focus is properly managed
  * The AudioTrack stream type is appropriate for your use case
  * Frame processing keeps up with the sample rate to avoid buffer underruns
</Warning>

**Key Points:**

* Decoded PCM data is written directly to AudioTrack
* Input buffer dequeue timeout is set to `-1` (wait indefinitely)
* Output buffer dequeue timeout is `0` (non-blocking)
* AudioTrack uses `MODE_STREAM` for continuous playback
* Minimum buffer size is calculated automatically

## Common Issues

### Buffer Underruns

If you experience audio stuttering:

```java theme={null}
// Increase AudioTrack buffer size
int minBufferSize = AudioTrack.getMinBufferSize(...);
int bufferSize = minBufferSize * 2; // Double the minimum size

audioTrack = new AudioTrack(
    AudioManager.STREAM_MUSIC,
    SAMPLE_RATE,
    AudioFormat.CHANNEL_OUT_STEREO,
    AudioFormat.ENCODING_PCM_16BIT,
    bufferSize, // Larger buffer
    AudioTrack.MODE_STREAM
);
```

### Audio Focus

Manage audio focus properly:

```java theme={null}
AudioManager audioManager = (AudioManager) getSystemService(Context.AUDIO_SERVICE);
int result = audioManager.requestAudioFocus(
    focusChangeListener,
    AudioManager.STREAM_MUSIC,
    AudioManager.AUDIOFOCUS_GAIN
);

if (result == AudioManager.AUDIOFOCUS_REQUEST_GRANTED) {
    audioDecoder.start();
}
```

## See Also

* [Video Decoder](/api/video-decoder) - Video stream decoding
* [Scrcpy Service](/api/scrcpy-service) - Main service integration
* [Event Controller](/api/event-controller) - Input event handling