Update developer documentation for v1.8

This commit is contained in:
Romain Vimont 2019-03-07 20:21:13 +01:00
parent d795144a36
commit 71fd238b0a
1 changed files with 54 additions and 25 deletions

View File

@ -32,7 +32,7 @@ The server is a Java application (with a [`public static void main(String...
args)`][main] method), compiled against the Android framework, and executed as args)`][main] method), compiled against the Android framework, and executed as
`shell` on the Android device. `shell` on the Android device.
[main]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/Server.java#L61 [main]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/Server.java#L100
To run such a Java application, the classes must be [_dexed_][dex] (typically, To run such a Java application, the classes must be [_dexed_][dex] (typically,
to `classes.dex`). If `my.package.MainClass` is the main class, compiled to to `classes.dex`). If `my.package.MainClass` is the main class, compiled to
@ -65,8 +65,8 @@ They can be called using reflection though. The communication with hidden
components is provided by [_wrappers_ classes][wrappers] and [aidl]. components is provided by [_wrappers_ classes][wrappers] and [aidl].
[hidden]: https://stackoverflow.com/a/31908373/1987178 [hidden]: https://stackoverflow.com/a/31908373/1987178
[wrappers]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/wrappers [wrappers]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/wrappers
[aidl]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/aidl/android/view [aidl]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/aidl/android/view
### Threading ### Threading
@ -89,9 +89,9 @@ The video is encoded using the [`MediaCodec`] API. The codec takes its input
from a [surface] associated to the display, and writes the resulting H.264 from a [surface] associated to the display, and writes the resulting H.264
stream to the provided output stream (the socket connected to the client). stream to the provided output stream (the socket connected to the client).
[`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java [`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java
[`MediaCodec`]: https://developer.android.com/reference/android/media/MediaCodec.html [`MediaCodec`]: https://developer.android.com/reference/android/media/MediaCodec.html
[surface]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L63-L64 [surface]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L69-L70
On device [rotation], the codec, surface and display are reinitialized, and a On device [rotation], the codec, surface and display are reinitialized, and a
new video stream is produced. new video stream is produced.
@ -105,8 +105,9 @@ because it avoids to send unnecessary frames, but there are drawbacks:
Both problems are [solved][repeat] by the flag Both problems are [solved][repeat] by the flag
[`KEY_REPEAT_PREVIOUS_FRAME_AFTER`][repeat-flag]. [`KEY_REPEAT_PREVIOUS_FRAME_AFTER`][repeat-flag].
[rotation]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L89-L92 [rotation]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L90
[repeat]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L125-L126 [repeat]:
https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L147-L148
[repeat-flag]: https://developer.android.com/reference/android/media/MediaFormat.html#KEY_REPEAT_PREVIOUS_FRAME_AFTER [repeat-flag]: https://developer.android.com/reference/android/media/MediaFormat.html#KEY_REPEAT_PREVIOUS_FRAME_AFTER
@ -124,11 +125,11 @@ All of them may need to inject input events to the system. To do so, they use
the _hidden_ method [`InputManager.injectInputEvent`] (exposed by our the _hidden_ method [`InputManager.injectInputEvent`] (exposed by our
[`InputManager` wrapper][inject-wrapper]). [`InputManager` wrapper][inject-wrapper]).
[`EventController`]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/EventController.java#L70 [`EventController`]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/EventController.java#L66
[`KeyEvent`]: https://developer.android.com/reference/android/view/KeyEvent.html [`KeyEvent`]: https://developer.android.com/reference/android/view/KeyEvent.html
[`MotionEvent`]: https://developer.android.com/reference/android/view/MotionEvent.html [`MotionEvent`]: https://developer.android.com/reference/android/view/MotionEvent.html
[`InputManager.injectInputEvent`]: https://android.googlesource.com/platform/frameworks/base/+/oreo-release/core/java/android/hardware/input/InputManager.java#857 [`InputManager.injectInputEvent`]: https://android.googlesource.com/platform/frameworks/base/+/oreo-release/core/java/android/hardware/input/InputManager.java#857
[inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27 [inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27
@ -153,7 +154,7 @@ Note that the client-server roles are expressed at the application level:
- the server _serves_ video stream and handle requests from the client, - the server _serves_ video stream and handle requests from the client,
- the client _controls_ the device through the server. - the client _controls_ the device through the server.
However, the roles are inverted at the network level: However, the roles are reversed at the network level:
- the client opens a server socket and listen on a port before starting the - the client opens a server socket and listen on a port before starting the
server, server,
@ -162,6 +163,9 @@ However, the roles are inverted at the network level:
This role inversion guarantees that the connection will not fail due to race This role inversion guarantees that the connection will not fail due to race
conditions, and avoids polling. conditions, and avoids polling.
_(Note that over TCP/IP, the roles are not reversed, due to a bug in `adb
reverse`. See commit [1038bad] and [issue #5].)_
Once the server is connected, it sends the device information (name and initial Once the server is connected, it sends the device information (name and initial
screen dimensions). Thus, the client may init the window and renderer, before screen dimensions). Thus, the client may init the window and renderer, before
the first frame is available. the first frame is available.
@ -169,6 +173,8 @@ the first frame is available.
To minimize startup time, SDL initialization is performed while listening for To minimize startup time, SDL initialization is performed while listening for
the connection from the server (see commit [90a46b4]). the connection from the server (see commit [90a46b4]).
[1038bad]: https://github.com/Genymobile/scrcpy/commit/1038bad3850f18717a048a4d5c0f8110e54ee172
[issue #5]: https://github.com/Genymobile/scrcpy/issues/5
[90a46b4]: https://github.com/Genymobile/scrcpy/commit/90a46b4c45637d083e877020d85ade52a9a5fa8e [90a46b4]: https://github.com/Genymobile/scrcpy/commit/90a46b4c45637d083e877020d85ade52a9a5fa8e
@ -177,17 +183,25 @@ the connection from the server (see commit [90a46b4]).
The client uses 3 threads: The client uses 3 threads:
- the **main** thread, executing the SDL event loop, - the **main** thread, executing the SDL event loop,
- the **decoder** thread, decoding video frames, - the **stream** thread, receiving the video and used for decoding and
recording,
- the **controller** thread, sending _control events_ to the server. - the **controller** thread, sending _control events_ to the server.
In addition, another thread can be started if necessary to handle APK
installation or file push requests (via drag&drop on the main window).
### Decoder
The [decoder] runs in a separate thread. It uses _libav_ to decode the H.264
stream from the socket, and notifies the main thread when a new frame is
available.
There are two [frames] simultaneously in memory: ### Stream
The video [stream] is received from the socket (connected to the server on the
device) in a separate thread.
If a [decoder] is present (i.e. `--no-display` is not set), then it uses _libav_
to decode the H.264 stream from the socket, and notifies the main thread when a
new frame is available.
There are two [frames][video_buffer] simultaneously in memory:
- the **decoding** frame, written by the decoder from the decoder thread, - the **decoding** frame, written by the decoder from the decoder thread,
- the **rendering** frame, rendered in a texture from the main thread. - the **rendering** frame, rendered in a texture from the main thread.
@ -195,9 +209,23 @@ When a new decoded frame is available, the decoder _swaps_ the decoding and
rendering frame (with proper synchronization). Thus, it immediatly starts rendering frame (with proper synchronization). Thus, it immediatly starts
to decode a new frame while the main thread renders the last one. to decode a new frame while the main thread renders the last one.
[decoder]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/decoder.c If a [recorder] is present (i.e. `--record` is enabled), then its muxes the raw
[frames]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/frames.h H.264 packet to the output video file.
[stream]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/stream.h
[decoder]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/decoder.h
[video_buffer]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/video_buffer.h
[recorder]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/recorder.h
```
+----------+ +----------+
---> | decoder | ---> | screen |
+---------+ / +----------+ +----------+
socket ---> | stream | ----
+---------+ \ +----------+
---> | recorder |
+----------+
```
### Controller ### Controller
@ -211,10 +239,10 @@ events_ to a blocking queue hold by the controller. On its own thread, the
controller takes events from the queue, that it serializes and sends to the controller takes events from the queue, that it serializes and sends to the
client. client.
[controller]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/controller.h [controller]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/controller.h
[controlevent]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/controlevent.h [controlevent]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/control_event.h
[inputmanager]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/inputmanager.h [inputmanager]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/input_manager.h
[convert]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/convert.h [convert]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/convert.h
### UI and event loop ### UI and event loop
@ -225,9 +253,10 @@ thread.
Events are handled in the [event loop], which either updates the [screen] or Events are handled in the [event loop], which either updates the [screen] or
delegates to the [input manager][inputmanager]. delegates to the [input manager][inputmanager].
[scrcpy]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/scrcpy.c [scrcpy]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/scrcpy.c
[event loop]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/scrcpy.c#L38 [event loop]:
[screen]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/screen.h https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/scrcpy.c#L187
[screen]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/screen.h
## Hack ## Hack