mirror of https://github.com/Genymobile/scrcpy
Update developer documentation for v1.8
This commit is contained in:
parent
d795144a36
commit
71fd238b0a
79
DEVELOP.md
79
DEVELOP.md
|
@ -32,7 +32,7 @@ The server is a Java application (with a [`public static void main(String...
|
||||||
args)`][main] method), compiled against the Android framework, and executed as
|
args)`][main] method), compiled against the Android framework, and executed as
|
||||||
`shell` on the Android device.
|
`shell` on the Android device.
|
||||||
|
|
||||||
[main]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/Server.java#L61
|
[main]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/Server.java#L100
|
||||||
|
|
||||||
To run such a Java application, the classes must be [_dexed_][dex] (typically,
|
To run such a Java application, the classes must be [_dexed_][dex] (typically,
|
||||||
to `classes.dex`). If `my.package.MainClass` is the main class, compiled to
|
to `classes.dex`). If `my.package.MainClass` is the main class, compiled to
|
||||||
|
@ -65,8 +65,8 @@ They can be called using reflection though. The communication with hidden
|
||||||
components is provided by [_wrappers_ classes][wrappers] and [aidl].
|
components is provided by [_wrappers_ classes][wrappers] and [aidl].
|
||||||
|
|
||||||
[hidden]: https://stackoverflow.com/a/31908373/1987178
|
[hidden]: https://stackoverflow.com/a/31908373/1987178
|
||||||
[wrappers]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/wrappers
|
[wrappers]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/wrappers
|
||||||
[aidl]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/aidl/android/view
|
[aidl]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/aidl/android/view
|
||||||
|
|
||||||
|
|
||||||
### Threading
|
### Threading
|
||||||
|
@ -89,9 +89,9 @@ The video is encoded using the [`MediaCodec`] API. The codec takes its input
|
||||||
from a [surface] associated to the display, and writes the resulting H.264
|
from a [surface] associated to the display, and writes the resulting H.264
|
||||||
stream to the provided output stream (the socket connected to the client).
|
stream to the provided output stream (the socket connected to the client).
|
||||||
|
|
||||||
[`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java
|
[`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java
|
||||||
[`MediaCodec`]: https://developer.android.com/reference/android/media/MediaCodec.html
|
[`MediaCodec`]: https://developer.android.com/reference/android/media/MediaCodec.html
|
||||||
[surface]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L63-L64
|
[surface]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L69-L70
|
||||||
|
|
||||||
On device [rotation], the codec, surface and display are reinitialized, and a
|
On device [rotation], the codec, surface and display are reinitialized, and a
|
||||||
new video stream is produced.
|
new video stream is produced.
|
||||||
|
@ -105,8 +105,9 @@ because it avoids to send unnecessary frames, but there are drawbacks:
|
||||||
Both problems are [solved][repeat] by the flag
|
Both problems are [solved][repeat] by the flag
|
||||||
[`KEY_REPEAT_PREVIOUS_FRAME_AFTER`][repeat-flag].
|
[`KEY_REPEAT_PREVIOUS_FRAME_AFTER`][repeat-flag].
|
||||||
|
|
||||||
[rotation]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L89-L92
|
[rotation]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L90
|
||||||
[repeat]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L125-L126
|
[repeat]:
|
||||||
|
https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L147-L148
|
||||||
[repeat-flag]: https://developer.android.com/reference/android/media/MediaFormat.html#KEY_REPEAT_PREVIOUS_FRAME_AFTER
|
[repeat-flag]: https://developer.android.com/reference/android/media/MediaFormat.html#KEY_REPEAT_PREVIOUS_FRAME_AFTER
|
||||||
|
|
||||||
|
|
||||||
|
@ -124,11 +125,11 @@ All of them may need to inject input events to the system. To do so, they use
|
||||||
the _hidden_ method [`InputManager.injectInputEvent`] (exposed by our
|
the _hidden_ method [`InputManager.injectInputEvent`] (exposed by our
|
||||||
[`InputManager` wrapper][inject-wrapper]).
|
[`InputManager` wrapper][inject-wrapper]).
|
||||||
|
|
||||||
[`EventController`]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/EventController.java#L70
|
[`EventController`]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/EventController.java#L66
|
||||||
[`KeyEvent`]: https://developer.android.com/reference/android/view/KeyEvent.html
|
[`KeyEvent`]: https://developer.android.com/reference/android/view/KeyEvent.html
|
||||||
[`MotionEvent`]: https://developer.android.com/reference/android/view/MotionEvent.html
|
[`MotionEvent`]: https://developer.android.com/reference/android/view/MotionEvent.html
|
||||||
[`InputManager.injectInputEvent`]: https://android.googlesource.com/platform/frameworks/base/+/oreo-release/core/java/android/hardware/input/InputManager.java#857
|
[`InputManager.injectInputEvent`]: https://android.googlesource.com/platform/frameworks/base/+/oreo-release/core/java/android/hardware/input/InputManager.java#857
|
||||||
[inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/v1.0/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27
|
[inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
@ -153,7 +154,7 @@ Note that the client-server roles are expressed at the application level:
|
||||||
- the server _serves_ video stream and handle requests from the client,
|
- the server _serves_ video stream and handle requests from the client,
|
||||||
- the client _controls_ the device through the server.
|
- the client _controls_ the device through the server.
|
||||||
|
|
||||||
However, the roles are inverted at the network level:
|
However, the roles are reversed at the network level:
|
||||||
|
|
||||||
- the client opens a server socket and listen on a port before starting the
|
- the client opens a server socket and listen on a port before starting the
|
||||||
server,
|
server,
|
||||||
|
@ -162,6 +163,9 @@ However, the roles are inverted at the network level:
|
||||||
This role inversion guarantees that the connection will not fail due to race
|
This role inversion guarantees that the connection will not fail due to race
|
||||||
conditions, and avoids polling.
|
conditions, and avoids polling.
|
||||||
|
|
||||||
|
_(Note that over TCP/IP, the roles are not reversed, due to a bug in `adb
|
||||||
|
reverse`. See commit [1038bad] and [issue #5].)_
|
||||||
|
|
||||||
Once the server is connected, it sends the device information (name and initial
|
Once the server is connected, it sends the device information (name and initial
|
||||||
screen dimensions). Thus, the client may init the window and renderer, before
|
screen dimensions). Thus, the client may init the window and renderer, before
|
||||||
the first frame is available.
|
the first frame is available.
|
||||||
|
@ -169,6 +173,8 @@ the first frame is available.
|
||||||
To minimize startup time, SDL initialization is performed while listening for
|
To minimize startup time, SDL initialization is performed while listening for
|
||||||
the connection from the server (see commit [90a46b4]).
|
the connection from the server (see commit [90a46b4]).
|
||||||
|
|
||||||
|
[1038bad]: https://github.com/Genymobile/scrcpy/commit/1038bad3850f18717a048a4d5c0f8110e54ee172
|
||||||
|
[issue #5]: https://github.com/Genymobile/scrcpy/issues/5
|
||||||
[90a46b4]: https://github.com/Genymobile/scrcpy/commit/90a46b4c45637d083e877020d85ade52a9a5fa8e
|
[90a46b4]: https://github.com/Genymobile/scrcpy/commit/90a46b4c45637d083e877020d85ade52a9a5fa8e
|
||||||
|
|
||||||
|
|
||||||
|
@ -177,17 +183,25 @@ the connection from the server (see commit [90a46b4]).
|
||||||
The client uses 3 threads:
|
The client uses 3 threads:
|
||||||
|
|
||||||
- the **main** thread, executing the SDL event loop,
|
- the **main** thread, executing the SDL event loop,
|
||||||
- the **decoder** thread, decoding video frames,
|
- the **stream** thread, receiving the video and used for decoding and
|
||||||
|
recording,
|
||||||
- the **controller** thread, sending _control events_ to the server.
|
- the **controller** thread, sending _control events_ to the server.
|
||||||
|
|
||||||
|
In addition, another thread can be started if necessary to handle APK
|
||||||
|
installation or file push requests (via drag&drop on the main window).
|
||||||
|
|
||||||
### Decoder
|
|
||||||
|
|
||||||
The [decoder] runs in a separate thread. It uses _libav_ to decode the H.264
|
|
||||||
stream from the socket, and notifies the main thread when a new frame is
|
|
||||||
available.
|
|
||||||
|
|
||||||
There are two [frames] simultaneously in memory:
|
### Stream
|
||||||
|
|
||||||
|
The video [stream] is received from the socket (connected to the server on the
|
||||||
|
device) in a separate thread.
|
||||||
|
|
||||||
|
If a [decoder] is present (i.e. `--no-display` is not set), then it uses _libav_
|
||||||
|
to decode the H.264 stream from the socket, and notifies the main thread when a
|
||||||
|
new frame is available.
|
||||||
|
|
||||||
|
There are two [frames][video_buffer] simultaneously in memory:
|
||||||
- the **decoding** frame, written by the decoder from the decoder thread,
|
- the **decoding** frame, written by the decoder from the decoder thread,
|
||||||
- the **rendering** frame, rendered in a texture from the main thread.
|
- the **rendering** frame, rendered in a texture from the main thread.
|
||||||
|
|
||||||
|
@ -195,9 +209,23 @@ When a new decoded frame is available, the decoder _swaps_ the decoding and
|
||||||
rendering frame (with proper synchronization). Thus, it immediatly starts
|
rendering frame (with proper synchronization). Thus, it immediatly starts
|
||||||
to decode a new frame while the main thread renders the last one.
|
to decode a new frame while the main thread renders the last one.
|
||||||
|
|
||||||
[decoder]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/decoder.c
|
If a [recorder] is present (i.e. `--record` is enabled), then its muxes the raw
|
||||||
[frames]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/frames.h
|
H.264 packet to the output video file.
|
||||||
|
|
||||||
|
[stream]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/stream.h
|
||||||
|
[decoder]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/decoder.h
|
||||||
|
[video_buffer]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/video_buffer.h
|
||||||
|
[recorder]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/recorder.h
|
||||||
|
|
||||||
|
```
|
||||||
|
+----------+ +----------+
|
||||||
|
---> | decoder | ---> | screen |
|
||||||
|
+---------+ / +----------+ +----------+
|
||||||
|
socket ---> | stream | ----
|
||||||
|
+---------+ \ +----------+
|
||||||
|
---> | recorder |
|
||||||
|
+----------+
|
||||||
|
```
|
||||||
|
|
||||||
### Controller
|
### Controller
|
||||||
|
|
||||||
|
@ -211,10 +239,10 @@ events_ to a blocking queue hold by the controller. On its own thread, the
|
||||||
controller takes events from the queue, that it serializes and sends to the
|
controller takes events from the queue, that it serializes and sends to the
|
||||||
client.
|
client.
|
||||||
|
|
||||||
[controller]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/controller.h
|
[controller]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/controller.h
|
||||||
[controlevent]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/controlevent.h
|
[controlevent]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/control_event.h
|
||||||
[inputmanager]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/inputmanager.h
|
[inputmanager]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/input_manager.h
|
||||||
[convert]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/convert.h
|
[convert]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/convert.h
|
||||||
|
|
||||||
|
|
||||||
### UI and event loop
|
### UI and event loop
|
||||||
|
@ -225,9 +253,10 @@ thread.
|
||||||
Events are handled in the [event loop], which either updates the [screen] or
|
Events are handled in the [event loop], which either updates the [screen] or
|
||||||
delegates to the [input manager][inputmanager].
|
delegates to the [input manager][inputmanager].
|
||||||
|
|
||||||
[scrcpy]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/scrcpy.c
|
[scrcpy]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/scrcpy.c
|
||||||
[event loop]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/scrcpy.c#L38
|
[event loop]:
|
||||||
[screen]: https://github.com/Genymobile/scrcpy/blob/v1.0/app/src/screen.h
|
https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/scrcpy.c#L187
|
||||||
|
[screen]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/screen.h
|
||||||
|
|
||||||
|
|
||||||
## Hack
|
## Hack
|
||||||
|
|
Loading…
Reference in New Issue