Control Features

Master touch controls, multi-touch gestures, and hardware button interactions

Overview

Scrcpy for Android provides comprehensive control features to interact with the remote device as if you were using it directly. All control features respect the aspect ratio and resolution of both devices.

Touch Control

Touch events are automatically translated from your local device screen coordinates to the remote device coordinates.

Single Touch

Basic touch interactions work exactly as you’d expect:

• Tap: Touch and release quickly
• Long press: Touch and hold
• Drag: Touch, hold, and move
• Swipe: Quick touch and slide gesture

All touch coordinates are automatically scaled based on:

• Remote device resolution
• Local display dimensions
• Current orientation (portrait/landscape)
• Navigation bar visibility

Coordinate Mapping

// From MainActivity.java:136-183
// Touch coordinates are mapped from display to remote resolution
int mappedX = (int) (touchX * remoteWidth / displayWidth);
int mappedY = (int) (touchY * remoteHeight / displayHeight);

Multi-Touch Support

The app supports full multi-touch gestures with up to 10 simultaneous touch points:

Pinch to Zoom

Use two fingers to zoom in/out in supported apps

• Each touch point is tracked independently
• Pointer IDs are preserved across events
• Works in maps, photos, browsers, etc.

Two-Finger Swipe

Scroll with two fingers for precise control

• Common in web browsers
• Supported in many photo galleries

Multi-Finger Gestures

Advanced gestures like:

• Three-finger swipe for app switching
• Four-finger gestures (if supported by remote device)

Multi-touch events are handled through MotionEvent.ACTION_POINTER_DOWN and MotionEvent.ACTION_POINTER_UP for middle fingers, while ACTION_DOWN and ACTION_UP handle the first and last fingers.

Touch Event Types

The following touch events are supported:

Event Type	Description	Code Reference
ACTION_DOWN	First finger touches screen	MotionEvent.ACTION_DOWN
ACTION_POINTER_DOWN	Additional finger touches	MotionEvent.ACTION_POINTER_DOWN
ACTION_MOVE	Any finger moves	MotionEvent.ACTION_MOVE
ACTION_POINTER_UP	Middle finger lifts	MotionEvent.ACTION_POINTER_UP
ACTION_UP	Last finger lifts	MotionEvent.ACTION_UP

Hardware Navigation Buttons

When the “Bottom control button” option is enabled, on-screen navigation buttons provide quick access to Android system functions.

Available Buttons

Back

Symbol: <

Sends: KEYCODE_BACK

Returns to previous screen

Home

Symbol: o

Sends: KEYCODE_HOME

Returns to home screen

App Switch

Symbol: =

Sends: KEYCODE_APP_SWITCH

Opens recent apps

Navigation Bar Layout

The navigation bar adapts to device orientation:

Portrait Mode:

• Positioned at bottom of screen
• Reduces available height by 96 pixels
• Buttons arranged horizontally

Landscape Mode:

• Positioned at right side of screen
• Reduces available width by 96 pixels
• Buttons arranged vertically

The 96-pixel dimension is used because it’s a multiple of 8, which ensures proper video encoding alignment.

Wake/Sleep with Proximity Sensor

Scrcpy for Android includes proximity sensor integration for power management.

This feature is currently disabled in the code but can be enabled by uncommenting the sensor event handlers in MainActivity.java:716-730.

How It Works

1. Cover proximity sensor (usually near the front camera)
2. Tap the screen while sensor is covered
3. Remote device locks (screen turns off)
4. Uncover sensor to wake the device

Sensor Implementation

// From MainActivity.java:716-731
@Override
public void onSensorChanged(SensorEvent sensorEvent) {
    if (sensorEvent.sensor.getType() == Sensor.TYPE_PROXIMITY) {
        if (sensorEvent.values[0] == 0) {
            // Proximity sensor covered - can trigger sleep
        } else {
            // Proximity sensor uncovered - can trigger wake
        }
    }
}

Accessing Local Navbar

Even in full-screen mode, you can access your local device’s navigation:

Swipe Up Gesture

1. Swipe up from the very bottom edge of your screen
2. Your local navigation bar appears temporarily
3. Use your local navigation to switch apps, go home, etc.
4. The gesture doesn’t interfere with the remote device

This works because the app uses SYSTEM_UI_FLAG_IMMERSIVE_STICKY mode, which allows the system UI to temporarily reappear on swipe-up.

Full-Screen Mode

The app uses immersive full-screen mode with these flags:

• SYSTEM_UI_FLAG_LAYOUT_STABLE
• SYSTEM_UI_FLAG_LAYOUT_HIDE_NAVIGATION
• SYSTEM_UI_FLAG_LAYOUT_FULLSCREEN
• SYSTEM_UI_FLAG_HIDE_NAVIGATION
• SYSTEM_UI_FLAG_FULLSCREEN
• SYSTEM_UI_FLAG_IMMERSIVE_STICKY

Screen Rotation Handling

The app intelligently handles rotation changes:

Automatic Rotation

When the remote device rotates:

1. Detection: Server sends new resolution configuration
2. Unbind: Current service connection is released
3. Swap: Screen dimensions are swapped (height ↔ width)
4. Reorient: Local orientation changes to match
5. Rebind: Service reconnects with new parameters

Portrait to Landscape

• App switches to SCREEN_ORIENTATION_SENSOR_LANDSCAPE
• Dimensions swapped automatically
• Touch coordinates recalculated
• No reconnection required

Landscape to Portrait

• App switches to SCREEN_ORIENTATION_SENSOR_PORTRAIT
• Reverse dimension swap
• Touch mapping updated
• Seamless transition

Aspect Ratio Adjustment

The app automatically adjusts padding to maintain correct aspect ratio:

When remote device is narrower:

• Adds horizontal padding (letterboxing)
• Centers the video stream

When remote device is wider:

• Adds vertical padding (pillarboxing)
• Maintains proportions

Aspect Ratio Logic

// From MainActivity.java:386-410
float remoteAspectRatio = remoteHeight / remoteWidth;
float localAspectRatio = localHeight / localWidth;

if (remoteAspectRatio > localAspectRatio) {
    // Add horizontal padding
    float wantWidth = localHeight / remoteAspectRatio;
    int padding = (int) (localWidth - wantWidth) / 2;
    linearLayout.setPadding(padding, 0, padding, 0);
} else if (remoteAspectRatio < localAspectRatio) {
    // Add vertical padding
    linearLayout.setPadding(0, verticalPadding, 0, 0);
}

Control Precision

For applications requiring precise touch input:

High-Precision Mode

• Touch coordinates use floating-point calculations
• Conversion to integer only at final step
• Minimizes rounding errors
• Accurate to pixel level on remote device

Timing Considerations

• Touch events are queued for reliable delivery
• Events sent as fast as network allows
• No artificial delays added
• Queue cleared on disconnect to prevent stale events

Disabling Controls

For view-only mode (presentations, monitoring):

1. Enable “Viewing mode (no control)”
2. All touch listeners are removed
3. Navigation buttons are disabled
4. Only viewing is possible

This is useful for demonstrations where you want to prevent accidental input.

Next Steps

Configuration

Optimize settings for your use case

Troubleshooting

Solve control responsiveness issues