mirror of https://github.com/Genymobile/scrcpy
Document UHID
Rework the documentation to present the keyboard and mouse input modes. PR #4473 <https://github.com/Genymobile/scrcpy/pull/4473>
This commit is contained in:
Romain Vimont
parent
dd479ed176
commit
b9d244b4c9
9
FAQ.md
9
FAQ.md
|
|
@ -133,9 +133,9 @@ Try with another USB cable or plug it into another USB port. See [#281] and
|
|||
[#283]: https://github.com/Genymobile/scrcpy/issues/283
|
||||
|
||||
|
||||
## HID/OTG issues on Windows
|
||||
## OTG issues on Windows
|
||||
|
||||
On Windows, if `scrcpy --otg` (or `--hid-keyboard`/`--hid-mouse`) results in:
|
||||
On Windows, if `scrcpy --otg` (or `--keyboard=aoa`/`--mouse=aoa`) results in:
|
||||
|
||||
> ERROR: Could not find any USB device
|
||||
|
||||
|
|
@ -170,12 +170,13 @@ The default text injection method is [limited to ASCII characters][text-input].
|
|||
A trick allows to also inject some [accented characters][accented-characters],
|
||||
but that's all. See [#37].
|
||||
|
||||
It is also possible to simulate a [physical keyboard][hid] (HID).
|
||||
To avoid the problem, [change the keyboard mode to simulate a physical
|
||||
keyboard][hid].
|
||||
|
||||
[text-input]: https://github.com/Genymobile/scrcpy/issues?q=is%3Aopen+is%3Aissue+label%3Aunicode
|
||||
[accented-characters]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-accented-characters
|
||||
[#37]: https://github.com/Genymobile/scrcpy/issues/37
|
||||
[hid]: doc/hid-otg.md
|
||||
[hid]: doc/keyboard.md#physical-keyboard-simulation
|
||||
|
||||
|
||||
## Client issues
|
||||
|
|
|
|||
14
README.md
14
README.md
|
|
@ -32,10 +32,13 @@ Its features include:
|
|||
- [configurable quality](doc/video.md)
|
||||
- [camera mirroring](doc/camera.md) (Android 12+)
|
||||
- [mirroring as a webcam (V4L2)](doc/v4l2.md) (Linux-only)
|
||||
- [physical keyboard/mouse simulation (HID)](doc/hid-otg.md)
|
||||
- [OTG mode](doc/hid-otg.md#otg)
|
||||
- physical [keyboard][hid-keyboard] and [mouse][hid-mouse] simulation (HID)
|
||||
- [OTG mode](doc/otg.md)
|
||||
- and more…
|
||||
|
||||
[hid-keyboard]: doc/keyboard.md#physical-keyboard-simulation
|
||||
[hid-mouse]: doc/mouse.md#physical-mouse-simulation
|
||||
|
||||
## Prerequisites
|
||||
|
||||
The Android device requires at least API 21 (Android 5.0).
|
||||
|
|
@ -53,8 +56,7 @@ this option is set.
|
|||
|
||||
[control]: https://github.com/Genymobile/scrcpy/issues/70#issuecomment-373286323
|
||||
|
||||
Note that USB debugging is not required to run scrcpy in [OTG
|
||||
mode](doc/hid-otg.md#otg).
|
||||
Note that USB debugging is not required to run scrcpy in [OTG mode](doc/otg.md).
|
||||
|
||||
|
||||
## Get the app
|
||||
|
|
@ -73,11 +75,13 @@ documented in the following pages:
|
|||
- [Video](doc/video.md)
|
||||
- [Audio](doc/audio.md)
|
||||
- [Control](doc/control.md)
|
||||
- [Keyboard](doc/keyboard.md)
|
||||
- [Mouse](doc/mouse.md)
|
||||
- [Device](doc/device.md)
|
||||
- [Window](doc/window.md)
|
||||
- [Recording](doc/recording.md)
|
||||
- [Tunnels](doc/tunnels.md)
|
||||
- [HID/OTG](doc/hid-otg.md)
|
||||
- [OTG](doc/otg.md)
|
||||
- [Camera](doc/camera.md)
|
||||
- [Video4Linux](doc/v4l2.md)
|
||||
- [Shortcuts](doc/shortcuts.md)
|
||||
|
|
|
|||
|
|
@ -10,36 +10,9 @@ scrcpy --no-control
|
|||
scrcpy -n # short version
|
||||
```
|
||||
|
||||
## Keyboard and mouse
|
||||
|
||||
## Text injection preference
|
||||
|
||||
Two kinds of [events][textevents] are generated when typing text:
|
||||
- _key events_, signaling that a key is pressed or released;
|
||||
- _text events_, signaling that a text has been entered.
|
||||
|
||||
By default, letters are injected using key events, so that the keyboard behaves
|
||||
as expected in games (typically for WASD keys).
|
||||
|
||||
But this may [cause issues][prefertext]. If you encounter such a problem, you
|
||||
can avoid it by:
|
||||
|
||||
```bash
|
||||
scrcpy --prefer-text
|
||||
```
|
||||
|
||||
(but this will break keyboard behavior in games)
|
||||
|
||||
On the contrary, you could force to always inject raw key events:
|
||||
|
||||
```bash
|
||||
scrcpy --raw-key-events
|
||||
```
|
||||
|
||||
These options have no effect on HID keyboard (all key events are sent as
|
||||
scancodes in this mode).
|
||||
|
||||
[textevents]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-text-input
|
||||
[prefertext]: https://github.com/Genymobile/scrcpy/issues/650#issuecomment-512945343
|
||||
Read [keyboard](keyboard.md) and [mouse](mouse.md).
|
||||
|
||||
|
||||
## Copy-paste
|
||||
|
|
@ -85,6 +58,7 @@ way as <kbd>MOD</kbd>+<kbd>Shift</kbd>+<kbd>v</kbd>).
|
|||
To disable automatic clipboard synchronization, use
|
||||
`--no-clipboard-autosync`.
|
||||
|
||||
|
||||
## Pinch-to-zoom, rotate and tilt simulation
|
||||
|
||||
To simulate "pinch-to-zoom": <kbd>Ctrl</kbd>+_click-and-move_.
|
||||
|
|
@ -100,20 +74,7 @@ at a location inverted through the center of the screen. When pressing
|
|||
<kbd>Ctrl</kbd> the x and y coordinates are inverted. Using <kbd>Shift</kbd>
|
||||
only inverts x.
|
||||
|
||||
|
||||
## Key repeat
|
||||
|
||||
By default, holding a key down generates repeated key events. This can cause
|
||||
performance problems in some games, where these events are useless anyway.
|
||||
|
||||
To avoid forwarding repeated key events:
|
||||
|
||||
```bash
|
||||
scrcpy --no-key-repeat
|
||||
```
|
||||
|
||||
This option has no effect on HID keyboard (key repeat is handled by Android
|
||||
directly in this mode).
|
||||
This only works for the default mouse mode (`--mouse=sdk`).
|
||||
|
||||
|
||||
## Right-click and middle-click
|
||||
|
|
@ -147,7 +108,3 @@ The target directory can be changed on start:
|
|||
```bash
|
||||
scrcpy --push-target=/sdcard/Movies/
|
||||
```
|
||||
|
||||
## Physical keyboard and mouse simulation
|
||||
|
||||
See the dedicated [HID/OTG](hid-otg.md) page.
|
||||
|
|
|
|||
|
|
@ -234,7 +234,7 @@ The video and audio streams are decoded by [FFmpeg].
|
|||
The client parses the command line arguments, then [runs one of two code
|
||||
paths][run]:
|
||||
- scrcpy in "normal" mode ([`scrcpy.c`])
|
||||
- scrcpy in [OTG mode](hid-otg.md) ([`scrcpy_otg.c`])
|
||||
- scrcpy in [OTG mode](otg.md) ([`scrcpy_otg.c`])
|
||||
|
||||
[run]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/app/src/main.c#L81-L82
|
||||
[`scrcpy.c`]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/app/src/scrcpy.c#L292-L293
|
||||
|
|
|
|||
112
doc/hid-otg.md
112
doc/hid-otg.md
|
|
@ -1,112 +0,0 @@
|
|||
# HID/OTG
|
||||
|
||||
By default, _scrcpy_ injects input events at the Android API level. As an
|
||||
alternative, when connected over USB, it is possible to send HID events, so that
|
||||
scrcpy behaves as if it was a physical keyboard and/or mouse connected to the
|
||||
Android device.
|
||||
|
||||
A special [OTG](#otg) mode allows to control the device without mirroring (and
|
||||
without USB debugging).
|
||||
|
||||
|
||||
## Physical keyboard simulation
|
||||
|
||||
By default, _scrcpy_ uses Android key or text injection. It works everywhere,
|
||||
but is limited to ASCII.
|
||||
|
||||
Instead, it can simulate a physical USB keyboard on Android to provide a better
|
||||
input experience (using [USB HID over AOAv2][hid-aoav2]): the virtual keyboard
|
||||
is disabled and it works for all characters and IME.
|
||||
|
||||
[hid-aoav2]: https://source.android.com/devices/accessories/aoa2#hid-support
|
||||
|
||||
However, it only works if the device is connected via USB.
|
||||
|
||||
Note: On Windows, it may only work in [OTG mode](#otg), not while mirroring (it
|
||||
is not possible to open a USB device if it is already open by another process
|
||||
like the _adb daemon_).
|
||||
|
||||
To enable this mode:
|
||||
|
||||
```bash
|
||||
scrcpy --hid-keyboard
|
||||
scrcpy -K # short version
|
||||
```
|
||||
|
||||
If it fails for some reason (for example because the device is not connected via
|
||||
USB), it automatically fallbacks to the default mode (with a log in the
|
||||
console). This allows using the same command line options when connected over
|
||||
USB and TCP/IP.
|
||||
|
||||
In this mode, raw key events (scancodes) are sent to the device, independently
|
||||
of the host key mapping. Therefore, if your keyboard layout does not match, it
|
||||
must be configured on the Android device, in Settings → System → Languages and
|
||||
input → [Physical keyboard].
|
||||
|
||||
This settings page can be started directly:
|
||||
|
||||
```bash
|
||||
adb shell am start -a android.settings.HARD_KEYBOARD_SETTINGS
|
||||
```
|
||||
|
||||
However, the option is only available when the HID keyboard is enabled (or when
|
||||
a physical keyboard is connected).
|
||||
|
||||
[Physical keyboard]: https://github.com/Genymobile/scrcpy/pull/2632#issuecomment-923756915
|
||||
|
||||
|
||||
## Physical mouse simulation
|
||||
|
||||
By default, _scrcpy_ uses Android mouse events injection with absolute
|
||||
coordinates. By simulating a physical mouse, a mouse pointer appears on the
|
||||
Android device, and relative mouse motion, clicks and scrolls are injected.
|
||||
|
||||
To enable this mode:
|
||||
|
||||
```bash
|
||||
scrcpy --hid-mouse
|
||||
scrcpy -M # short version
|
||||
```
|
||||
|
||||
When this mode is enabled, the computer mouse is "captured" (the mouse pointer
|
||||
disappears from the computer and appears on the Android device instead).
|
||||
|
||||
Special capture keys, either <kbd>Alt</kbd> or <kbd>Super</kbd>, toggle
|
||||
(disable or enable) the mouse capture. Use one of them to give the control of
|
||||
the mouse back to the computer.
|
||||
|
||||
|
||||
## OTG
|
||||
|
||||
It is possible to run _scrcpy_ with only physical keyboard and mouse simulation
|
||||
(HID), as if the computer keyboard and mouse were plugged directly to the device
|
||||
via an OTG cable.
|
||||
|
||||
In this mode, `adb` (USB debugging) is not necessary, and mirroring is disabled.
|
||||
|
||||
This is similar to `--hid-keyboard --hid-mouse`, but without mirroring.
|
||||
|
||||
To enable OTG mode:
|
||||
|
||||
```bash
|
||||
scrcpy --otg
|
||||
# Pass the serial if several USB devices are available
|
||||
scrcpy --otg -s 0123456789abcdef
|
||||
```
|
||||
|
||||
It is possible to enable only HID keyboard or HID mouse:
|
||||
|
||||
```bash
|
||||
scrcpy --otg --hid-keyboard # keyboard only
|
||||
scrcpy --otg --hid-mouse # mouse only
|
||||
scrcpy --otg --hid-keyboard --hid-mouse # keyboard and mouse
|
||||
# for convenience, enable both by default
|
||||
scrcpy --otg # keyboard and mouse
|
||||
```
|
||||
|
||||
Like `--hid-keyboard` and `--hid-mouse`, it only works if the device is
|
||||
connected over USB.
|
||||
|
||||
## HID/OTG issues on Windows
|
||||
|
||||
See [FAQ](/FAQ.md#hidotg-issues-on-windows).
|
||||
|
|
@ -0,0 +1,136 @@
|
|||
# Keyboard
|
||||
|
||||
Several keyboard input modes are available:
|
||||
|
||||
- `--keyboard=sdk` (default)
|
||||
- `--keyboard=uhid` (or `-K`): simulates a physical HID keyboard using the UHID
|
||||
kernel module on the device
|
||||
- `--keyboard=aoa`: simulates a physical HID keyboard using the AOAv2 protocol
|
||||
- `--keyboard=disabled`
|
||||
|
||||
By default, `sdk` is used, but if you use scrcpy regularly, it is recommended to
|
||||
use [`uhid`](#uhid) and configure the keyboard layout once and for all.
|
||||
|
||||
|
||||
## SDK keyboard
|
||||
|
||||
In this mode (`--keyboard=sdk`, or if the parameter is omitted), keyboard input
|
||||
events are injected at the Android API level. It works everywhere, but it is
|
||||
limited to ASCII and some other characters.
|
||||
|
||||
Note that on some devices, an additional option must be enabled in developer
|
||||
options for this keyboard mode to work. See
|
||||
[prerequisites](/README.md#prerequisites).
|
||||
|
||||
Additional parameters (specific to `--keyboard=sdk`) described below allow to
|
||||
customize the behavior.
|
||||
|
||||
|
||||
### Text injection preference
|
||||
|
||||
Two kinds of [events][textevents] are generated when typing text:
|
||||
- _key events_, signaling that a key is pressed or released;
|
||||
- _text events_, signaling that a text has been entered.
|
||||
|
||||
By default, numbers and "special characters" are inserted using text events, but
|
||||
letters are injected using key events, so that the keyboard behaves as expected
|
||||
in games (typically for WASD keys).
|
||||
|
||||
But this may [cause issues][prefertext]. If you encounter such a problem, you
|
||||
can inject letters as text (or just switch to [UHID](#uhid)):
|
||||
|
||||
```bash
|
||||
scrcpy --prefer-text
|
||||
```
|
||||
|
||||
(but this will break keyboard behavior in games)
|
||||
|
||||
On the contrary, you could force to always inject raw key events:
|
||||
|
||||
```bash
|
||||
scrcpy --raw-key-events
|
||||
```
|
||||
|
||||
[textevents]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-text-input
|
||||
[prefertext]: https://github.com/Genymobile/scrcpy/issues/650#issuecomment-512945343
|
||||
|
||||
|
||||
### Key repeat
|
||||
|
||||
By default, holding a key down generates repeated key events. Ths can cause
|
||||
performance problems in some games, where these events are useless anyway.
|
||||
|
||||
To avoid forwarding repeated key events:
|
||||
|
||||
```bash
|
||||
scrcpy --no-key-repeat
|
||||
```
|
||||
|
||||
|
||||
## Physical keyboard simulation
|
||||
|
||||
Two modes allow to simulate a physical HID keyboard on the device.
|
||||
|
||||
To work properly, it is necessary to configure (once and for all) the keyboard
|
||||
layout on the device to match that of the computer.
|
||||
|
||||
The configuration page can be opened in one of the following ways:
|
||||
- from the scrcpy window (when `uhid` or `aoa` is used), by pressing
|
||||
<kbd>MOD</kbd>+<kbd>k</kbd> (see [shortcuts](shortcuts.md))
|
||||
- from the device, in Settings → System → Languages and input → Physical
|
||||
devices
|
||||
- from a terminal on the computer, by executing `adb shell am start -a
|
||||
android.settings.HARD_KEYBOARD_SETTINGS`
|
||||
|
||||
From this configuration page, it is also possible to enable or disable on-screen
|
||||
keyboard.
|
||||
|
||||
|
||||
### UHID
|
||||
|
||||
This mode simulates a physical HID keyboard using the [UHID] kernel module on the
|
||||
device.
|
||||
|
||||
[UHID]: https://kernel.org/doc/Documentation/hid/uhid.txt
|
||||
|
||||
To enable UHID keyboard, use:
|
||||
|
||||
```bash
|
||||
scrcpy --keyboard=uhid
|
||||
scrcpy -K # short version
|
||||
```
|
||||
|
||||
Once the keyboard layout is configured (see above), it is the best mode for
|
||||
using the keyboard while mirroring:
|
||||
|
||||
- it works for all characters and IME (contrary to `--keyboard=sdk`)
|
||||
- the on-screen keyboard can be disabled (contrary to `--keyboard=sdk`)
|
||||
- it works over TCP/IP (wirelessly) (contrary to `--keyboard=aoa`)
|
||||
- there are no issues on Windows (contrary to `--keyboard=aoa`)
|
||||
|
||||
One drawback is that it may not work on old Android versions due to permission
|
||||
errors.
|
||||
|
||||
|
||||
### AOA
|
||||
|
||||
This mode simulates a physical HID keyboard using the [AOAv2] protocol.
|
||||
|
||||
[AOAv2]: https://source.android.com/devices/accessories/aoa2#hid-support
|
||||
|
||||
To enable AOA keyboard, use:
|
||||
|
||||
```bash
|
||||
scrcpy --keyboard=aoa
|
||||
```
|
||||
|
||||
Contrary to the other modes, it works at the USB level directly (so it only
|
||||
works over USB).
|
||||
|
||||
It does not use the scrcpy server, and does not require `adb` (USB debugging).
|
||||
Therefore, it is possible to control the device (but not mirror) even with USB
|
||||
debugging disabled (see [OTG](otg.md)).
|
||||
|
||||
Note: On Windows, it may only work in [OTG mode](otg.md), not while mirroring
|
||||
(it is not possible to open a USB device if it is already open by another
|
||||
process like the _adb daemon_).
|
||||
|
|
@ -0,0 +1,70 @@
|
|||
# Mouse
|
||||
|
||||
Several mouse input modes are available:
|
||||
|
||||
- `--mouse=sdk` (default)
|
||||
- `--mouse=uhid` (or `-M`): simulates a physical HID mouse using the UHID
|
||||
kernel module on the device
|
||||
- `--mouse=aoa`: simulates a physical HID mouse using the AOAv2 protocol
|
||||
- `--mouse=disabled`
|
||||
|
||||
|
||||
## SDK mouse
|
||||
|
||||
In this mode (`--mouse=sdk`, or if the parameter is omitted), mouse input events
|
||||
are injected at the Android API level with absolute coordinates.
|
||||
|
||||
Note that on some devices, an additional option must be enabled in developer
|
||||
options for this mouse mode to work. See
|
||||
[prerequisites](/README.md#prerequisites).
|
||||
|
||||
|
||||
## Physical mouse simulation
|
||||
|
||||
Two modes allow to simulate a physical HID mouse on the device.
|
||||
|
||||
In these modes, the computer mouse is "captured": the mouse pointer disappears
|
||||
from the computer and appears on the Android device instead.
|
||||
|
||||
Special capture keys, either <kbd>Alt</kbd> or <kbd>Super</kbd>, toggle
|
||||
(disable or enable) the mouse capture. Use one of them to give the control of
|
||||
the mouse back to the computer.
|
||||
|
||||
|
||||
### UHID
|
||||
|
||||
This mode simulates a physical HID mouse using the [UHID] kernel module on the
|
||||
device.
|
||||
|
||||
[UHID]: https://kernel.org/doc/Documentation/hid/uhid.txt
|
||||
|
||||
To enable UHID mouse, use:
|
||||
|
||||
```bash
|
||||
scrcpy --mouse=uhid
|
||||
scrcpy -M # short version
|
||||
```
|
||||
|
||||
|
||||
### AOA
|
||||
|
||||
This mode simulates a physical HID mouse using the [AOAv2] protocol.
|
||||
|
||||
[AOAv2]: https://source.android.com/devices/accessories/aoa2#hid-support
|
||||
|
||||
To enable AOA mouse, use:
|
||||
|
||||
```bash
|
||||
scrcpy --mouse=aoa
|
||||
```
|
||||
|
||||
Contrary to the other modes, it works at the USB level directly (so it only
|
||||
works over USB).
|
||||
|
||||
It does not use the scrcpy server, and does not require `adb` (USB debugging).
|
||||
Therefore, it is possible to control the device (but not mirror) even with USB
|
||||
debugging disabled (see [OTG](otg.md)).
|
||||
|
||||
Note: On Windows, it may only work in [OTG mode](otg.md), not while mirroring
|
||||
(it is not possible to open a USB device if it is already open by another
|
||||
process like the _adb daemon_).
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
# OTG
|
||||
|
||||
By default, _scrcpy_ injects input events at the Android API level. As an
|
||||
alternative, when connected over USB, it is possible to send HID events, so that
|
||||
scrcpy behaves as if it was a physical keyboard and/or mouse connected to the
|
||||
Android device.
|
||||
|
||||
A special mode allows to control the device without mirroring, using AOA
|
||||
[keyboard](keyboard.md#aoa) and [mouse](mouse.md#aoa). Therefore, it is possible
|
||||
to run _scrcpy_ with only physical keyboard and mouse simulation (HID), as if
|
||||
the computer keyboard and mouse were plugged directly to the device via an OTG
|
||||
cable.
|
||||
|
||||
In this mode, `adb` (USB debugging) is not necessary, and mirroring is disabled.
|
||||
|
||||
This is similar to `--keyboard=aoa --mouse=aoa`, but without mirroring.
|
||||
|
||||
To enable OTG mode:
|
||||
|
||||
```bash
|
||||
scrcpy --otg
|
||||
# Pass the serial if several USB devices are available
|
||||
scrcpy --otg -s 0123456789abcdef
|
||||
```
|
||||
|
||||
It is possible to disable HID keyboard or HID mouse:
|
||||
|
||||
```bash
|
||||
scrcpy --otg --keyboard=disabled
|
||||
scrcpy --otg --mouse=disabled
|
||||
```
|
||||
|
||||
It only works if the device is connected over USB.
|
||||
|
||||
## OTG issues on Windows
|
||||
|
||||
See [FAQ](/FAQ.md#otg-issues-on-windows).
|
||||
Loading…
Reference in New Issue