Architecture Overview

Deep dive into Scrcpy for Android’s client-server architecture and internal components

Architecture Overview

Scrcpy for Android uses a client-server architecture to enable Android-to-Android screen mirroring and control. The application is split into two distinct modules that communicate over TCP sockets.

Two-Module Structure

The project consists of two independent modules:

Client (App Module)

Located in app/src/main/java/org/client/scrcpy/

Runs on the controlling Android device and handles:

• Video/audio decoding
• Display rendering
• Touch input capture
• Network communication

Server Module

Located in server/src/main/java/org/server/scrcpy/

Runs on the target Android device and handles:

• Screen capture
• Video/audio encoding
• Input event injection
• ADB socket management

Client-Server Communication

The architecture follows a bidirectional communication pattern:

Video Streaming Pipeline

The video streaming process involves multiple stages from capture to display:

Server Side (Encoding)

1. Screen Capture (ScreenCapture.java)

   • Uses Android’s MediaCodec API with VirtualDisplay
   • Captures screen content as a Surface
   • Supports rotation changes dynamically

2. Video Encoding (ScreenEncoder.java)

   • Encodes to H.264 (AVC) format
   • Configuration: bitRate, frameRate (default 60fps), iFrameInterval (default 10s)
   • Packets are tagged with flags: CONFIG, KEY_FRAME, FRAME, or END
   • Each packet includes a presentation timestamp

3. Packet Structure (VideoPacket.java)

   [Type (1 byte)][Flag (1 byte)][PTS (8 bytes)][Size (4 bytes)][Data (N bytes)]
   

Client Side (Decoding)

1. Network Reception (Scrcpy.java:342-470)

   • Reads packets from DataInputStream
   • Packet size is transmitted first (4 bytes)
   • Validates packet size (max 4MB per packet)

2. Video Decoding (VideoDecoder.java)

   • Configures MediaCodec with SPS/PPS from CONFIG packets
   • Decodes H.264 stream to Surface
   • Handles surface updates on rotation

3. Rendering

   • Direct rendering to SurfaceView via MediaCodec
   • Aspect ratio preservation with padding calculation

Audio Streaming Pipeline

Audio follows a parallel pipeline to video:

Server Side

• Audio Capture (AudioCapture.java, AudioDirectCapture.java)

  • Captures system audio using AudioRecord
  • Default bitrate: 128kbps
  • Runs asynchronously in separate thread (ScreenEncoder.java:113-129)

• Audio Encoding (AudioEncoder.java)

  • Encodes to AAC format
  • Packets tagged with CONFIG, FRAME, or END flags

Client Side

• Audio Decoding (AudioDecoder.java)
  • Decodes AAC packets
  • Configures from CONFIG packet data
  • Synchronized with video using presentation timestamps

Touch Event Handling Flow

Touch events flow from the client device to inject input on the target:

Touch Event Protocol

Each touch event is serialized as 20 bytes:

[action (4)][buttonState (4)][x (4)][y (4)][pointerId (4)]

Coordinate Transformation

The client performs coordinate scaling (Scrcpy.java:136-182):

1. Calculate remote device aspect ratio
2. Scale local coordinates to remote resolution
3. Handle landscape/portrait orientation
4. Support multi-touch (up to MAX_POINTERS)

Server-Side Injection

The server receives events and injects them (EventController.java:66-121):

1. Read 20-byte buffer from input stream
2. Deserialize action, coordinates, and pointer ID
3. Convert to physical screen coordinates
4. Create MotionEvent with proper tool type (TOOL_TYPE_FINGER)
5. Handle multi-pointer actions (ACTION_POINTER_DOWN/UP)
6. Inject via InputManager.injectInputEvent()

ADB Port Forwarding Mechanism

The connection setup relies on ADB’s port forwarding feature:

Connection Establishment

1. ADB Connection (SendCommands.java:92-114)

   adb connect <ip>:<port>  // Default port 5555
   adb push scrcpy-server.jar /data/local/tmp/
   adb forward tcp:7008 tcp:7007
   

2. Server Socket (DroidConnection.java:27-36)

   • Server opens ServerSocket on port 7007
   • Waits for incoming connection
   • Validates client IP address

3. Client Socket (Scrcpy.java:227-290)

   • Client connects to 127.0.0.1:7008 (forwarded port)
   • Connection timeout: 5000ms
   • Retry mechanism: up to 50 attempts

Initial Handshake

1. Server sends device resolution as first packet (16 bytes):

   [width (4 bytes)][height (4 bytes)][0 (4 bytes)][0 (4 bytes)]
   

2. Client reads resolution and configures decoders

3. Bidirectional streaming begins

Key Components Reference

Component	Location	Responsibility
Server.java	server/src/main/java/org/server/scrcpy/	Entry point, orchestrates server components
Scrcpy.java	app/src/main/java/org/client/scrcpy/	Client service, manages connection and decoding
DroidConnection.java	server/src/main/java/org/server/scrcpy/	Socket management on server
ScreenEncoder.java	server/src/main/java/org/server/scrcpy/	Video encoding and streaming
AudioEncoder.java	server/src/main/java/org/server/scrcpy/	Audio encoding and streaming
EventController.java	server/src/main/java/org/server/scrcpy/	Input event handling and injection
VideoDecoder.java	app/src/main/java/org/client/scrcpy/decoder/	Video decoding on client
AudioDecoder.java	app/src/main/java/org/client/scrcpy/decoder/	Audio decoding on client

Performance Considerations

Video Encoding

• Default: 60fps, adjustable via Options.java
• Resolution must be multiple of 8 (H.264 requirement)
• Key frame interval: 10 seconds
• Bitrate configurable per connection

Network Buffering

• Client polls input stream with available() check
• Events queued in LinkedList for async sending
• Frame dropping based on presentation timestamp delay

Threading Model

• Server: Main thread for video encoding, separate thread for event controller
• Client: Decoder threads managed by MediaCodec, main loop in service thread

The architecture is designed for low-latency local network operation. For public network use, additional latency management may be required.