Document UHID

Rework the documentation to present the keyboard and mouse input modes.

PR #4473 <https://github.com/Genymobile/scrcpy/pull/4473>

FAQ.md

@@ -133,9 +133,9 @@ Try with another USB cable or plug it into another USB port. See [#281] and
 [#283]: https://github.com/Genymobile/scrcpy/issues/283
 
 
-## HID/OTG issues on Windows
+## OTG issues on Windows
 
-On Windows, if `scrcpy --otg` (or `--hid-keyboard`/`--hid-mouse`) results in:
+On Windows, if `scrcpy --otg` (or `--keyboard=aoa`/`--mouse=aoa`) results in:
 
 >     ERROR: Could not find any USB device
 
@@ -170,12 +170,13 @@ The default text injection method is [limited to ASCII characters][text-input].
 A trick allows to also inject some [accented characters][accented-characters],
 but that's all. See [#37].
 
-It is also possible to simulate a [physical keyboard][hid] (HID).
+To avoid the problem, [change the keyboard mode to simulate a physical
+keyboard][hid].
 
 [text-input]: https://github.com/Genymobile/scrcpy/issues?q=is%3Aopen+is%3Aissue+label%3Aunicode
 [accented-characters]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-accented-characters
 [#37]: https://github.com/Genymobile/scrcpy/issues/37
-[hid]: doc/hid-otg.md
+[hid]: doc/keyboard.md#physical-keyboard-simulation
 
 
 ## Client issues

README.md

@@ -32,10 +32,13 @@ Its features include:
  - [configurable quality](doc/video.md)
  - [camera mirroring](doc/camera.md) (Android 12+)
  - [mirroring as a webcam (V4L2)](doc/v4l2.md) (Linux-only)
- - [physical keyboard/mouse simulation (HID)](doc/hid-otg.md)
+ - physical [keyboard][hid-keyboard] and [mouse][hid-mouse] simulation (HID)
- - [OTG mode](doc/hid-otg.md#otg)
+ - [OTG mode](doc/otg.md)
  - and more…
 
+[hid-keyboard]: doc/keyboard.md#physical-keyboard-simulation
+[hid-mouse]: doc/mouse.md#physical-mouse-simulation
+
 ## Prerequisites
 
 The Android device requires at least API 21 (Android 5.0).
@@ -53,8 +56,7 @@ this option is set.
 
 [control]: https://github.com/Genymobile/scrcpy/issues/70#issuecomment-373286323
 
-Note that USB debugging is not required to run scrcpy in [OTG
+Note that USB debugging is not required to run scrcpy in [OTG mode](doc/otg.md).
-mode](doc/hid-otg.md#otg).
 
 
 ## Get the app
@@ -73,11 +75,13 @@ documented in the following pages:
  - [Video](doc/video.md)
  - [Audio](doc/audio.md)
  - [Control](doc/control.md)
+ - [Keyboard](doc/keyboard.md)
+ - [Mouse](doc/mouse.md)
  - [Device](doc/device.md)
  - [Window](doc/window.md)
  - [Recording](doc/recording.md)
  - [Tunnels](doc/tunnels.md)
- - [HID/OTG](doc/hid-otg.md)
+ - [OTG](doc/otg.md)
  - [Camera](doc/camera.md)
  - [Video4Linux](doc/v4l2.md)
  - [Shortcuts](doc/shortcuts.md)

doc/control.md

@@ -10,36 +10,9 @@ scrcpy --no-control
 scrcpy -n   # short version
 ```
 
+## Keyboard and mouse
 
-## Text injection preference
+Read [keyboard](keyboard.md) and [mouse](mouse.md).
-
-Two kinds of [events][textevents] are generated when typing text:
- - _key events_, signaling that a key is pressed or released;
- - _text events_, signaling that a text has been entered.
-
-By default, letters are injected using key events, so that the keyboard behaves
-as expected in games (typically for WASD keys).
-
-But this may [cause issues][prefertext]. If you encounter such a problem, you
-can avoid it by:
-
-```bash
-scrcpy --prefer-text
-```
-
-(but this will break keyboard behavior in games)
-
-On the contrary, you could force to always inject raw key events:
-
-```bash
-scrcpy --raw-key-events
-```
-
-These options have no effect on HID keyboard (all key events are sent as
-scancodes in this mode).
-
-[textevents]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-text-input
-[prefertext]: https://github.com/Genymobile/scrcpy/issues/650#issuecomment-512945343
 
 
 ## Copy-paste
@@ -85,6 +58,7 @@ way as <kbd>MOD</kbd>+<kbd>Shift</kbd>+<kbd>v</kbd>).
 To disable automatic clipboard synchronization, use
 `--no-clipboard-autosync`.
 
+
 ## Pinch-to-zoom, rotate and tilt simulation
 
 To simulate "pinch-to-zoom": <kbd>Ctrl</kbd>+_click-and-move_.
@@ -100,20 +74,7 @@ at a location inverted through the center of the screen. When pressing
 <kbd>Ctrl</kbd> the x and y coordinates are inverted. Using <kbd>Shift</kbd>
 only inverts x.
 
-
+This only works for the default mouse mode (`--mouse=sdk`).
-## Key repeat
-
-By default, holding a key down generates repeated key events. This can cause
-performance problems in some games, where these events are useless anyway.
-
-To avoid forwarding repeated key events:
-
-```bash
-scrcpy --no-key-repeat
-```
-
-This option has no effect on HID keyboard (key repeat is handled by Android
-directly in this mode).
 
 
 ## Right-click and middle-click
@@ -147,7 +108,3 @@ The target directory can be changed on start:
 ```bash
 scrcpy --push-target=/sdcard/Movies/
 ```
-
-## Physical keyboard and mouse simulation
-
-See the dedicated [HID/OTG](hid-otg.md) page.

doc/develop.md

@@ -234,7 +234,7 @@ The video and audio streams are decoded by [FFmpeg].
 The client parses the command line arguments, then [runs one of two code
 paths][run]:
  - scrcpy in "normal" mode ([`scrcpy.c`])
- - scrcpy in [OTG mode](hid-otg.md) ([`scrcpy_otg.c`])
+ - scrcpy in [OTG mode](otg.md) ([`scrcpy_otg.c`])
 
 [run]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/app/src/main.c#L81-L82
 [`scrcpy.c`]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/app/src/scrcpy.c#L292-L293

doc/hid-otg.md

@@ -1,112 +0,0 @@
-# HID/OTG
-
-By default, _scrcpy_ injects input events at the Android API level. As an
-alternative, when connected over USB, it is possible to send HID events, so that
-scrcpy behaves as if it was a physical keyboard and/or mouse connected to the
-Android device.
-
-A special [OTG](#otg) mode allows to control the device without mirroring (and
-without USB debugging).
-
-
-## Physical keyboard simulation
-
-By default, _scrcpy_ uses Android key or text injection. It works everywhere,
-but is limited to ASCII.
-
-Instead, it can simulate a physical USB keyboard on Android to provide a better
-input experience (using [USB HID over AOAv2][hid-aoav2]): the virtual keyboard
-is disabled and it works for all characters and IME.
-
-[hid-aoav2]: https://source.android.com/devices/accessories/aoa2#hid-support
-
-However, it only works if the device is connected via USB.
-
-Note: On Windows, it may only work in [OTG mode](#otg), not while mirroring (it
-is not possible to open a USB device if it is already open by another process
-like the _adb daemon_).
-
-To enable this mode:
-
-```bash
-scrcpy --hid-keyboard
-scrcpy -K   # short version
-```
-
-If it fails for some reason (for example because the device is not connected via
-USB), it automatically fallbacks to the default mode (with a log in the
-console). This allows using the same command line options when connected over
-USB and TCP/IP.
-
-In this mode, raw key events (scancodes) are sent to the device, independently
-of the host key mapping. Therefore, if your keyboard layout does not match, it
-must be configured on the Android device, in Settings → System → Languages and
-input → [Physical keyboard].
-
-This settings page can be started directly:
-
-```bash
-adb shell am start -a android.settings.HARD_KEYBOARD_SETTINGS
-```
-
-However, the option is only available when the HID keyboard is enabled (or when
-a physical keyboard is connected).
-
-[Physical keyboard]: https://github.com/Genymobile/scrcpy/pull/2632#issuecomment-923756915
-
-
-## Physical mouse simulation
-
-By default, _scrcpy_ uses Android mouse events injection with absolute
-coordinates. By simulating a physical mouse, a mouse pointer appears on the
-Android device, and relative mouse motion, clicks and scrolls are injected.
-
-To enable this mode:
-
-```bash
-scrcpy --hid-mouse
-scrcpy -M   # short version
-```
-
-When this mode is enabled, the computer mouse is "captured" (the mouse pointer
-disappears from the computer and appears on the Android device instead).
-
-Special capture keys, either <kbd>Alt</kbd> or <kbd>Super</kbd>, toggle
-(disable or enable) the mouse capture. Use one of them to give the control of
-the mouse back to the computer.
-
-
-## OTG
-
-It is possible to run _scrcpy_ with only physical keyboard and mouse simulation
-(HID), as if the computer keyboard and mouse were plugged directly to the device
-via an OTG cable.
-
-In this mode, `adb` (USB debugging) is not necessary, and mirroring is disabled.
-
-This is similar to `--hid-keyboard --hid-mouse`, but without mirroring.
-
-To enable OTG mode:
-
-```bash
-scrcpy --otg
-# Pass the serial if several USB devices are available
-scrcpy --otg -s 0123456789abcdef
-```
-
-It is possible to enable only HID keyboard or HID mouse:
-
-```bash
-scrcpy --otg --hid-keyboard              # keyboard only
-scrcpy --otg --hid-mouse                 # mouse only
-scrcpy --otg --hid-keyboard --hid-mouse  # keyboard and mouse
-# for convenience, enable both by default
-scrcpy --otg                             # keyboard and mouse
-```
-
-Like `--hid-keyboard` and `--hid-mouse`, it only works if the device is
-connected over USB.
-
-## HID/OTG issues on Windows
-
-See [FAQ](/FAQ.md#hidotg-issues-on-windows).

doc/keyboard.md

@@ -0,0 +1,136 @@
+# Keyboard
+
+Several keyboard input modes are available:
+
+ - `--keyboard=sdk` (default)
+ - `--keyboard=uhid` (or `-K`): simulates a physical HID keyboard using the UHID
+   kernel module on the device
+ - `--keyboard=aoa`: simulates a physical HID keyboard using the AOAv2 protocol
+ - `--keyboard=disabled`
+
+By default, `sdk` is used, but if you use scrcpy regularly, it is recommended to
+use [`uhid`](#uhid) and configure the keyboard layout once and for all.
+
+
+## SDK keyboard
+
+In this mode (`--keyboard=sdk`, or if the parameter is omitted), keyboard input
+events are injected at the Android API level. It works everywhere, but it is
+limited to ASCII and some other characters.
+
+Note that on some devices, an additional option must be enabled in developer
+options for this keyboard mode to work. See
+[prerequisites](/README.md#prerequisites).
+
+Additional parameters (specific to `--keyboard=sdk`) described below allow to
+customize the behavior.
+
+
+### Text injection preference
+
+Two kinds of [events][textevents] are generated when typing text:
+ - _key events_, signaling that a key is pressed or released;
+ - _text events_, signaling that a text has been entered.
+
+By default, numbers and "special characters" are inserted using text events, but
+letters are injected using key events, so that the keyboard behaves as expected
+in games (typically for WASD keys).
+
+But this may [cause issues][prefertext]. If you encounter such a problem, you
+can inject letters as text (or just switch to [UHID](#uhid)):
+
+```bash
+scrcpy --prefer-text
+```
+
+(but this will break keyboard behavior in games)
+
+On the contrary, you could force to always inject raw key events:
+
+```bash
+scrcpy --raw-key-events
+```
+
+[textevents]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-text-input
+[prefertext]: https://github.com/Genymobile/scrcpy/issues/650#issuecomment-512945343
+
+
+### Key repeat
+
+By default, holding a key down generates repeated key events. Ths can cause
+performance problems in some games, where these events are useless anyway.
+
+To avoid forwarding repeated key events:
+
+```bash
+scrcpy --no-key-repeat
+```
+
+
+## Physical keyboard simulation
+
+Two modes allow to simulate a physical HID keyboard on the device.
+
+To work properly, it is necessary to configure (once and for all) the keyboard
+layout on the device to match that of the computer.
+
+The configuration page can be opened in one of the following ways:
+ - from the scrcpy window (when `uhid` or `aoa` is used), by pressing
+   <kbd>MOD</kbd>+<kbd>k</kbd> (see [shortcuts](shortcuts.md))
+ - from the device, in Settings → System → Languages and input → Physical
+   devices
+ - from a terminal on the computer, by executing `adb shell am start -a
+   android.settings.HARD_KEYBOARD_SETTINGS`
+
+From this configuration page, it is also possible to enable or disable on-screen
+keyboard.
+
+
+### UHID
+
+This mode simulates a physical HID keyboard using the [UHID] kernel module on the
+device.
+
+[UHID]: https://kernel.org/doc/Documentation/hid/uhid.txt
+
+To enable UHID keyboard, use:
+
+```bash
+scrcpy --keyboard=uhid
+scrcpy -K  # short version
+```
+
+Once the keyboard layout is configured (see above), it is the best mode for
+using the keyboard while mirroring:
+
+ - it works for all characters and IME (contrary to `--keyboard=sdk`)
+ - the on-screen keyboard can be disabled (contrary to `--keyboard=sdk`)
+ - it works over TCP/IP (wirelessly) (contrary to `--keyboard=aoa`)
+ - there are no issues on Windows (contrary to `--keyboard=aoa`)
+
+One drawback is that it may not work on old Android versions due to permission
+errors.
+
+
+### AOA
+
+This mode simulates a physical HID keyboard using the [AOAv2] protocol.
+
+[AOAv2]: https://source.android.com/devices/accessories/aoa2#hid-support
+
+To enable AOA keyboard, use:
+
+```bash
+scrcpy --keyboard=aoa
+```
+
+Contrary to the other modes, it works at the USB level directly (so it only
+works over USB).
+
+It does not use the scrcpy server, and does not require `adb` (USB debugging).
+Therefore, it is possible to control the device (but not mirror) even with USB
+debugging disabled (see [OTG](otg.md)).
+
+Note: On Windows, it may only work in [OTG mode](otg.md), not while mirroring
+(it is not possible to open a USB device if it is already open by another
+process like the _adb daemon_).

doc/mouse.md

@@ -0,0 +1,70 @@
+# Mouse
+
+Several mouse input modes are available:
+
+ - `--mouse=sdk` (default)
+ - `--mouse=uhid` (or `-M`): simulates a physical HID mouse using the UHID
+   kernel module on the device
+ - `--mouse=aoa`: simulates a physical HID mouse using the AOAv2 protocol
+ - `--mouse=disabled`
+
+
+## SDK mouse
+
+In this mode (`--mouse=sdk`, or if the parameter is omitted), mouse input events
+are injected at the Android API level with absolute coordinates.
+
+Note that on some devices, an additional option must be enabled in developer
+options for this mouse mode to work. See
+[prerequisites](/README.md#prerequisites).
+
+
+## Physical mouse simulation
+
+Two modes allow to simulate a physical HID mouse on the device.
+
+In these modes, the computer mouse is "captured": the mouse pointer disappears
+from the computer and appears on the Android device instead.
+
+Special capture keys, either <kbd>Alt</kbd> or <kbd>Super</kbd>, toggle
+(disable or enable) the mouse capture. Use one of them to give the control of
+the mouse back to the computer.
+
+
+### UHID
+
+This mode simulates a physical HID mouse using the [UHID] kernel module on the
+device.
+
+[UHID]: https://kernel.org/doc/Documentation/hid/uhid.txt
+
+To enable UHID mouse, use:
+
+```bash
+scrcpy --mouse=uhid
+scrcpy -M  # short version
+```
+
+
+### AOA
+
+This mode simulates a physical HID mouse using the [AOAv2] protocol.
+
+[AOAv2]: https://source.android.com/devices/accessories/aoa2#hid-support
+
+To enable AOA mouse, use:
+
+```bash
+scrcpy --mouse=aoa
+```
+
+Contrary to the other modes, it works at the USB level directly (so it only
+works over USB).
+
+It does not use the scrcpy server, and does not require `adb` (USB debugging).
+Therefore, it is possible to control the device (but not mirror) even with USB
+debugging disabled (see [OTG](otg.md)).
+
+Note: On Windows, it may only work in [OTG mode](otg.md), not while mirroring
+(it is not possible to open a USB device if it is already open by another
+process like the _adb daemon_).

doc/otg.md

@@ -0,0 +1,37 @@
+# OTG
+
+By default, _scrcpy_ injects input events at the Android API level. As an
+alternative, when connected over USB, it is possible to send HID events, so that
+scrcpy behaves as if it was a physical keyboard and/or mouse connected to the
+Android device.
+
+A special mode allows to control the device without mirroring, using AOA
+[keyboard](keyboard.md#aoa) and [mouse](mouse.md#aoa). Therefore, it is possible
+to run _scrcpy_ with only physical keyboard and mouse simulation (HID), as if
+the computer keyboard and mouse were plugged directly to the device via an OTG
+cable.
+
+In this mode, `adb` (USB debugging) is not necessary, and mirroring is disabled.
+
+This is similar to `--keyboard=aoa --mouse=aoa`, but without mirroring.
+
+To enable OTG mode:
+
+```bash
+scrcpy --otg
+# Pass the serial if several USB devices are available
+scrcpy --otg -s 0123456789abcdef
+```
+
+It is possible to disable HID keyboard or HID mouse:
+
+```bash
+scrcpy --otg --keyboard=disabled
+scrcpy --otg --mouse=disabled
+```
+
+It only works if the device is connected over USB.
+
+## OTG issues on Windows
+
+See [FAQ](/FAQ.md#otg-issues-on-windows).