RepoMirrors
/
mpv
mirror of https://github.com/mpv-player/mpv
1
0
Fork 0
Code Issues Releases Wiki Activity
mpv/filters/f_decoder_wrapper.c

1327 lines
40 KiB
C
Raw Normal View History

video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
/*
* This file is part of mpv.
*
* mpv is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* mpv is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with mpv. If not, see <http://www.gnu.org/licenses/>.
*/
options: change how option range min/max is handled Before this commit, option declarations used M_OPT_MIN/M_OPT_MAX (and some other identifiers based on these) to signal whether an option had min/max values. Remove these flags, and make it use a range implicitly on the condition if min<max is true. This requires care in all cases when only M_OPT_MIN or M_OPT_MAX were set (instead of both). Generally, the commit replaces all these instances with using DBL_MAX/DBL_MIN for the "unset" part of the range. This also happens to fix some cases where you could pass over-large values to integer options, which were silently truncated, but now cause an error. This commit has some higher potential for regressions.
2020-03-13 15:49:39 +00:00
#include <float.h>
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
#include <math.h>
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
#include <assert.h>
#include <libavutil/buffer.h>
ad_lavc, vd_lavc: return full error codes to shared decoder loop ad_lavc and vd_lavc use the lavc_process() helper to translate the FFmpeg push/pull API to the internal filter API (which completely mismatch, even though I'm responsible for both, just fucking kill me). This interface was "slightly" too tight. It returned only a bool indicating "progress", which was not enough to handle some cases (see following commit). While we're at it, move all state into a struct. This is only a single bool, but we get the chance to add more if needed. This fixes mpv falling asleep if decoding returns an error during draining. If decoding fails when we already sent EOF, the state machine stopped making progress. This left mpv just sitting around and doing nothing. A test case can be created with: echo $RANDOM >> image.png This makes libavformat read a proper packet plus a packet of garbage. libavcodec will decode a frame, and then return an error code. The lavc_process() wrapper could not deal with this, because there was no way to differentiate between "retry" and "send new packet". Normally, it would send a new packet, so decoding would make progress anyway. If there was "progress", we couldn't just retry, because it'd retry forever. This is made worse by the fact that it tries to decode at least two frames before starting display, meaning it will "sit around and do nothing" before the picture is displayed. Change it so that on error return, "receiving" a frame is retried. This will make it return the EOF, so everything works properly. This is a high-risk change, because all these funny bullshit exceptions for hardware decoding are in the way, and I didn't retest them. For example, if hardware decoding is enabled, it keeps a list of packets, that are fed into the decoder again if hardware decoding fails, and a software fallback is performed. Another case of horrifying accidental complexity. Fixes: #6618
2019-10-24 16:40:46 +00:00
#include <libavutil/common.h>
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
#include <libavutil/rational.h>
#include "options/options.h"
#include "common/msg.h"
player: get rid of mpv_global.opts This was always a legacy thing. Remove it by applying an orgy of mp_get_config_group() calls, and sometimes m_config_cache_alloc() or mp_read_option_raw(). win32 changes untested.
2018-05-21 14:25:52 +00:00
#include "options/m_config.h"
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
#include "osdep/timer.h"
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
#include "osdep/threads.h"
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
#include "demux/demux.h"
#include "demux/packet.h"
#include "common/codecs.h"
#include "common/global.h"
#include "common/recorder.h"
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
#include "misc/dispatch.h"
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
#include "audio/aframe.h"
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
#include "video/out/vo.h"
#include "video/csputils.h"
#include "demux/stheader.h"
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
#include "f_async_queue.h"
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
#include "f_decoder_wrapper.h"
#include "f_demux_in.h"
#include "filter_internal.h"
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
struct dec_queue_opts {
options: transition options from OPT_FLAG to OPT_BOOL c78482045444c488bb7948305d583a55d17cd236 introduced a bool option type as a replacement for the flag type, but didn't actually transition and remove the flag type because it would have been too much mundane work.
2023-02-20 03:32:50 +00:00
bool use_queue;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
int64_t max_bytes;
int64_t max_samples;
double max_duration;
};
#define OPT_BASE_STRUCT struct dec_queue_opts
static const struct m_option dec_queue_opts_list[] = {
options: transition options from OPT_FLAG to OPT_BOOL c78482045444c488bb7948305d583a55d17cd236 introduced a bool option type as a replacement for the flag type, but didn't actually transition and remove the flag type because it would have been too much mundane work.
2023-02-20 03:32:50 +00:00
{"enable", OPT_BOOL(use_queue)},
options: change option macros and all option declarations Change all OPT_* macros such that they don't define the entire m_option initializer, and instead expand only to a part of it, which sets certain fields. This requires changing almost every option declaration, because they all use these macros. A declaration now always starts with {"name", ... followed by designated initializers only (possibly wrapped in macros). The OPT_* macros now initialize the .offset and .type fields only, sometimes also .priv and others. I think this change makes the option macros less tricky. The old code had to stuff everything into macro arguments (and attempted to allow setting arbitrary fields by letting the user pass designated initializers in the vararg parts). Some of this was made messy due to C99 and C11 not allowing 0-sized varargs with ',' removal. It's also possible that this change is pointless, other than cosmetic preferences. Not too happy about some things. For example, the OPT_CHOICE() indentation I applied looks a bit ugly. Much of this change was done with regex search&replace, but some places required manual editing. In particular, code in "obscure" areas (which I didn't include in compilation) might be broken now. In wayland_common.c the author of some option declarations confused the flags parameter with the default value (though the default value was also properly set below). I fixed this with this change.
2020-03-14 20:28:01 +00:00
{"max-secs", OPT_DOUBLE(max_duration), M_RANGE(0, DBL_MAX)},
options: fix OPT_BYTE_SIZE upper limits As an unfortunate disaster, min/max values use the type double, which causes tons of issues with int64_t types. Anyway, OPT_BYTE_SIZE is often used as maximum for size_t quantities, which can have a size different from (u)int64_t. OPT_BYTE_SIZE still uses in64_t, because in theory, you could use it for file sizes. (demux.c would for example be capable of caching more than 2GB on 32 bit platforms if a file cache is used. Though for some reason the accounting code still uses size_t, so that use case is broken. But still insist that it _could_ be used this way.) There were various inconsistent attempts to set m_option.max to a value such that the size_t/int64_t upper limit is not exceeded. Due to the double max field, this didn't really work correctly. Try to fix this with the M_MAX_MEM_BYTES constant. It's a good approximation, because on 32 bit it should allow 2GB (untested, also would probably exhaust address space in practice but whatever), and something "high enough" in 64 bit. For some reason, clang 11 still warns. But I think this might be a clang bug, or I'm crazy. The result is correct anyway.
2020-03-18 19:51:38 +00:00
{"max-bytes", OPT_BYTE_SIZE(max_bytes), M_RANGE(0, M_MAX_MEM_BYTES)},
options: change option macros and all option declarations Change all OPT_* macros such that they don't define the entire m_option initializer, and instead expand only to a part of it, which sets certain fields. This requires changing almost every option declaration, because they all use these macros. A declaration now always starts with {"name", ... followed by designated initializers only (possibly wrapped in macros). The OPT_* macros now initialize the .offset and .type fields only, sometimes also .priv and others. I think this change makes the option macros less tricky. The old code had to stuff everything into macro arguments (and attempted to allow setting arbitrary fields by letting the user pass designated initializers in the vararg parts). Some of this was made messy due to C99 and C11 not allowing 0-sized varargs with ',' removal. It's also possible that this change is pointless, other than cosmetic preferences. Not too happy about some things. For example, the OPT_CHOICE() indentation I applied looks a bit ugly. Much of this change was done with regex search&replace, but some places required manual editing. In particular, code in "obscure" areas (which I didn't include in compilation) might be broken now. In wayland_common.c the author of some option declarations confused the flags parameter with the default value (though the default value was also properly set below). I fixed this with this change.
2020-03-14 20:28:01 +00:00
{"max-samples", OPT_INT64(max_samples), M_RANGE(0, DBL_MAX)},
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
{0}
};
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
static const struct m_sub_options vdec_queue_conf = {
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
.opts = dec_queue_opts_list,
.size = sizeof(struct dec_queue_opts),
.defaults = &(const struct dec_queue_opts){
.max_bytes = 512 * 1024 * 1024,
.max_samples = 50,
.max_duration = 2,
},
};
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
static const struct m_sub_options adec_queue_conf = {
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
.opts = dec_queue_opts_list,
.size = sizeof(struct dec_queue_opts),
.defaults = &(const struct dec_queue_opts){
.max_bytes = 1 * 1024 * 1024,
.max_samples = 48000,
.max_duration = 1,
},
};
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
#undef OPT_BASE_STRUCT
#define OPT_BASE_STRUCT struct dec_wrapper_opts
struct dec_wrapper_opts {
m_option: change m_option_type_aspect to double This specific option type is only used for the video aspect. The underlying type was a float to represent the inputted value, but it's actually not precise enough. When using something like 4:3, the values of the incorrect digits are actually significant enough to make av_d2q return a very funky numerator and denominator which is close to 4/3 but not quite. This leads to some "off by one pixel" errors. Weirdly, mpv's actual calculations for this were already being done as double, but then converted to floats for this specific type. Just drop the conversion step and leave it all as double which has the precision we need (i.e. AVRational is now 4/3 for the this case). Fixes #8190.
2023-08-08 22:24:28 +00:00
double movie_aspect;
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
int aspect_method;
options: rename --fps to --container-fps-override This better reflects what it actually does. As a bonus, script writers won't be misled into thinking that fps displays the actual video or display fps.
2023-10-15 14:36:56 +00:00
double fps_override;
options: transition options from OPT_FLAG to OPT_BOOL c78482045444c488bb7948305d583a55d17cd236 introduced a bool option type as a replacement for the flag type, but didn't actually transition and remove the flag type because it would have been too much mundane work.
2023-02-20 03:32:50 +00:00
bool correct_pts;
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
int video_rotate;
char *audio_decoders;
char *video_decoders;
char *audio_spdif;
struct dec_queue_opts *vdec_queue_opts;
struct dec_queue_opts *adec_queue_opts;
int64_t video_reverse_size;
int64_t audio_reverse_size;
};
options: Make validation and help possible for all option types Today, validation is only possible for string type options. But there's no particular reason why it needs to be restricted in this way, and there are potential uses, to allow other options to be validated without forcing the option to have to reimplement parsing from scratch. The first part, simply making the validation function an explicit field instead of overloading priv is simple enough. But if we only do that, then the validation function still needs to deal with the raw pre-parsed string. Instead, we want to allow the value to be parsed before it is validated. That in turn leads to us having validator functions that should be type aware. Unfortunately, that means we need to keep the explicit macro like OPT_STRING_VALIDATE() as a way to enforce the correct typing of the function. Otherwise, we'd have to have the validator take a void * and hope the implementation can cast it correctly. For help, we don't have this problem, as help doesn't look at the value. Then, we turn validators that are really help generators into explicit help functions and where a validator is help + validation, we split them into two parts. I have, however, left functions that need to query information for both help and validation as single functions to avoid code duplication. In this change, I have not added an other OPT_FOO_VALIDATE() macros as they are not needed, but I will add some in a separate change to illustrate the pattern.
2021-02-21 00:41:44 +00:00
static int decoder_list_help(struct mp_log *log, const m_option_t *opt,
struct bstr name);
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
const struct m_sub_options dec_wrapper_conf = {
.opts = (const struct m_option[]){
options: transition options from OPT_FLAG to OPT_BOOL c78482045444c488bb7948305d583a55d17cd236 introduced a bool option type as a replacement for the flag type, but didn't actually transition and remove the flag type because it would have been too much mundane work.
2023-02-20 03:32:50 +00:00
{"correct-pts", OPT_BOOL(correct_pts)},
options: rename --fps to --container-fps-override This better reflects what it actually does. As a bonus, script writers won't be misled into thinking that fps displays the actual video or display fps.
2023-10-15 14:36:56 +00:00
{"container-fps-override", OPT_DOUBLE(fps_override), M_RANGE(0, DBL_MAX)},
options: Make validation and help possible for all option types Today, validation is only possible for string type options. But there's no particular reason why it needs to be restricted in this way, and there are potential uses, to allow other options to be validated without forcing the option to have to reimplement parsing from scratch. The first part, simply making the validation function an explicit field instead of overloading priv is simple enough. But if we only do that, then the validation function still needs to deal with the raw pre-parsed string. Instead, we want to allow the value to be parsed before it is validated. That in turn leads to us having validator functions that should be type aware. Unfortunately, that means we need to keep the explicit macro like OPT_STRING_VALIDATE() as a way to enforce the correct typing of the function. Otherwise, we'd have to have the validator take a void * and hope the implementation can cast it correctly. For help, we don't have this problem, as help doesn't look at the value. Then, we turn validators that are really help generators into explicit help functions and where a validator is help + validation, we split them into two parts. I have, however, left functions that need to query information for both help and validation as single functions to avoid code duplication. In this change, I have not added an other OPT_FOO_VALIDATE() macros as they are not needed, but I will add some in a separate change to illustrate the pattern.
2021-02-21 00:41:44 +00:00
{"ad", OPT_STRING(audio_decoders),
.help = decoder_list_help},
{"vd", OPT_STRING(video_decoders),
.help = decoder_list_help},
{"audio-spdif", OPT_STRING(audio_spdif),
.help = decoder_list_help},
options: change option macros and all option declarations Change all OPT_* macros such that they don't define the entire m_option initializer, and instead expand only to a part of it, which sets certain fields. This requires changing almost every option declaration, because they all use these macros. A declaration now always starts with {"name", ... followed by designated initializers only (possibly wrapped in macros). The OPT_* macros now initialize the .offset and .type fields only, sometimes also .priv and others. I think this change makes the option macros less tricky. The old code had to stuff everything into macro arguments (and attempted to allow setting arbitrary fields by letting the user pass designated initializers in the vararg parts). Some of this was made messy due to C99 and C11 not allowing 0-sized varargs with ',' removal. It's also possible that this change is pointless, other than cosmetic preferences. Not too happy about some things. For example, the OPT_CHOICE() indentation I applied looks a bit ugly. Much of this change was done with regex search&replace, but some places required manual editing. In particular, code in "obscure" areas (which I didn't include in compilation) might be broken now. In wayland_common.c the author of some option declarations confused the flags parameter with the default value (though the default value was also properly set below). I fixed this with this change.
2020-03-14 20:28:01 +00:00
{"video-rotate", OPT_CHOICE(video_rotate, {"no", -1}),
.flags = UPDATE_IMGPAR, M_RANGE(0, 359)},
{"video-aspect-override", OPT_ASPECT(movie_aspect),
.flags = UPDATE_IMGPAR, M_RANGE(-1, 10)},
{"video-aspect-method", OPT_CHOICE(aspect_method,
{"bitstream", 1}, {"container", 2}),
.flags = UPDATE_IMGPAR},
{"vd-queue", OPT_SUBSTRUCT(vdec_queue_opts, vdec_queue_conf)},
{"ad-queue", OPT_SUBSTRUCT(adec_queue_opts, adec_queue_conf)},
{"video-reversal-buffer", OPT_BYTE_SIZE(video_reverse_size),
options: fix OPT_BYTE_SIZE upper limits As an unfortunate disaster, min/max values use the type double, which causes tons of issues with int64_t types. Anyway, OPT_BYTE_SIZE is often used as maximum for size_t quantities, which can have a size different from (u)int64_t. OPT_BYTE_SIZE still uses in64_t, because in theory, you could use it for file sizes. (demux.c would for example be capable of caching more than 2GB on 32 bit platforms if a file cache is used. Though for some reason the accounting code still uses size_t, so that use case is broken. But still insist that it _could_ be used this way.) There were various inconsistent attempts to set m_option.max to a value such that the size_t/int64_t upper limit is not exceeded. Due to the double max field, this didn't really work correctly. Try to fix this with the M_MAX_MEM_BYTES constant. It's a good approximation, because on 32 bit it should allow 2GB (untested, also would probably exhaust address space in practice but whatever), and something "high enough" in 64 bit. For some reason, clang 11 still warns. But I think this might be a clang bug, or I'm crazy. The result is correct anyway.
2020-03-18 19:51:38 +00:00
M_RANGE(0, M_MAX_MEM_BYTES)},
options: change option macros and all option declarations Change all OPT_* macros such that they don't define the entire m_option initializer, and instead expand only to a part of it, which sets certain fields. This requires changing almost every option declaration, because they all use these macros. A declaration now always starts with {"name", ... followed by designated initializers only (possibly wrapped in macros). The OPT_* macros now initialize the .offset and .type fields only, sometimes also .priv and others. I think this change makes the option macros less tricky. The old code had to stuff everything into macro arguments (and attempted to allow setting arbitrary fields by letting the user pass designated initializers in the vararg parts). Some of this was made messy due to C99 and C11 not allowing 0-sized varargs with ',' removal. It's also possible that this change is pointless, other than cosmetic preferences. Not too happy about some things. For example, the OPT_CHOICE() indentation I applied looks a bit ugly. Much of this change was done with regex search&replace, but some places required manual editing. In particular, code in "obscure" areas (which I didn't include in compilation) might be broken now. In wayland_common.c the author of some option declarations confused the flags parameter with the default value (though the default value was also properly set below). I fixed this with this change.
2020-03-14 20:28:01 +00:00
{"audio-reversal-buffer", OPT_BYTE_SIZE(audio_reverse_size),
options: rename --fps to --container-fps-override This better reflects what it actually does. As a bonus, script writers won't be misled into thinking that fps displays the actual video or display fps.
2023-10-15 14:36:56 +00:00
M_RANGE(0, M_MAX_MEM_BYTES)},
{"fps", OPT_REPLACED("container-fps-override")},
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
{0}
},
.size = sizeof(struct dec_wrapper_opts),
.defaults = &(const struct dec_wrapper_opts){
options: transition options from OPT_FLAG to OPT_BOOL c78482045444c488bb7948305d583a55d17cd236 introduced a bool option type as a replacement for the flag type, but didn't actually transition and remove the flag type because it would have been too much mundane work.
2023-02-20 03:32:50 +00:00
.correct_pts = true,
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
.movie_aspect = -1.,
.aspect_method = 2,
.video_reverse_size = 1 * 1024 * 1024 * 1024,
.audio_reverse_size = 64 * 1024 * 1024,
},
};
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
struct priv {
struct mp_log *log;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
struct sh_stream *header;
// --- The following fields are to be accessed by dec_dispatch (or if that
// field is NULL, by the mp_decoder_wrapper user thread).
// Use thread_lock() for access outside of the decoder thread.
bool request_terminate_dec_thread;
struct mp_filter *dec_root_filter; // thread root filter; no thread => NULL
struct mp_filter *decf; // wrapper filter which drives the decoder
player: get rid of mpv_global.opts This was always a legacy thing. Remove it by applying an orgy of mp_get_config_group() calls, and sometimes m_config_cache_alloc() or mp_read_option_raw(). win32 changes untested.
2018-05-21 14:25:52 +00:00
struct m_config_cache *opt_cache;
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
struct dec_wrapper_opts *opts;
f_decoder_wrapper: make most queue options runtime changeable Why not.
2020-02-29 23:38:34 +00:00
struct dec_queue_opts *queue_opts;
f_decoder_wrapper: enable DR and hwdec with --vd-queue-enable This was forgotten. Hardware decoding typically breaks immediately, because many hw decoding APIs require allocating all surfaces in advance (and/or libavcodec was not made flexible enough to add new surfaces later). If the queue is large enough, it will run out of surfaces, fail decoding, and fall back to software decoding. We consider this the user's fault. --hwdec-extra-frames can be used to avoid this, if you have enough GPU memory, and the needed number of frames is lower than the arbitrary mpv-set maximum limit of that option.
2020-03-05 10:33:37 +00:00
struct mp_stream_info stream_info;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
struct mp_codec_params *codec;
struct mp_decoder *decoder;
// Demuxer output.
struct mp_pin *demux;
// Last PTS from decoder (set with each vd_driver->decode() call)
double codec_pts;
int num_codec_pts_problems;
// Last packet DTS from decoder (passed through from source packets)
double codec_dts;
int num_codec_dts_problems;
// PTS or DTS of packet first read
double first_packet_pdts;
// There was at least one packet with nonsense timestamps.
f_decoder_wrapper: fully reset timestamp fixup logic on seeks This could lead to nonsense when backward playback is involved. Better reduce the possible interactions. Besides, it's better to fully reset things on seeks in general. The only exception is has_broken_packet_pts, which enables hr-seek if everything looks good. It's intended to trigger at the second hr-seek or so if the file is normal, and to disable it if the file is broken. It tries to avoid enabling the hr-seek logic before it can know about whether things are "good", so resetting it on seeks would obviously never enable it. Document it as explicit exception.
2019-05-31 15:51:46 +00:00
// Intentionally not reset on seeks; its whole purpose is to enable faster
// future seeks.
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
int has_broken_packet_pts; // <0: uninitialized, 0: no problems, 1: broken
int has_broken_decoded_pts;
int packets_without_output; // number packets sent without frame received
// Final PTS of previously decoded frame
double pts;
struct mp_image_params dec_format, last_format, fixed_format;
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
double fps;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
double start_pts;
double start, end;
struct demux_packet *new_segment;
struct mp_frame packet;
f_decoder_wrapper: hack for discarding preroll in backward playback mode Some audio codecs will discard or cut the first frames when starting decoding. While some of that works through well-defined mechanisms (like initial padding), it's in general very codec/decoder specific, and not really predictable. In addition, libavcodec isn't very good with reporting "dropped" frames (and our internal interface reflects this). It seems our only chance to handle this is through timestamps. In theory, it would be best to discard frames that have timestamps before the "resume" position. But since video has reordered timestamps, we'd need to put some effort into finding this position. The first video packet doesn't necessarily contain this timestamp. (In theory, we could just do this in the demuxer with some trivial additional work, and set it on the packet's kf_seek_pts field. Although this field is supposed to contain just this value, the field is considered demuxer-internal, and I didn't want to make matters worse by reusing it for the interface to the decoder. With some more effort and buffering, we could calculate this value within the decoder, but fuck that.) The approach chosen in this commit is setting the timestamp to NOPTS. This will break in some obscure situations, but backward playback is a pretty obscure feature to begin with, so I considered this a reasonable implementation choice. Before passing a preroll packet to the decoder, its timestamps are set to NOPTS. Frames that are returned from the decoder and have the NOPTS timestamp are considered preroll and are discarded. This happens only during "preroll" mode (preroll_discard==true), so it doesn't affect normal forward playback. It's disabled on the first packet with a timestamp, so it can tolerate some crap even in backward playback mode. We don't check the dts fields out of laziness (decoded audio frames don't even have this field). I considered using an approach using the EDL clipping infrastructure (as mentioned in the last two paragraphs in the commit message of commit " demux_lavf: implement bad hack for backward playback of wav"). This didn't work, and I blamed timestamp rounding within mpv for it. But the problem was actually due to Matroska-rounded timestamps. Since the audio frame size isn't exactly aligned to 1ms, there will be an overlap (or gap) in the timestamps. This overlap is much smaller than 1ms, since it's just the sub-millisecond remainder part of the audio frame size. This makes the timestamps discontinuous and unreliable for the purpose we wanted to use it. We can't just smooth the timestamps in the demuxer either.
2019-05-31 14:07:13 +00:00
bool packet_fed, preroll_discard;
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
size_t reverse_queue_byte_size;
struct mp_frame *reverse_queue;
int num_reverse_queue;
bool reverse_queue_complete;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
struct mp_frame decoded_coverart;
int coverart_returned; // 0: no, 1: coverart frame itself, 2: EOF returned
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
int play_dir;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
// --- The following fields can be accessed only from the mp_decoder_wrapper
// user thread.
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
struct mp_decoder_wrapper public;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
// --- Specific access depending on threading stuff.
struct mp_async_queue *queue; // decoded frame output queue
struct mp_dispatch_queue *dec_dispatch; // non-NULL if decoding thread used
bool dec_thread_lock; // debugging (esp. for no-thread case)
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_thread dec_thread;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
bool dec_thread_valid;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex cache_lock;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
// --- Protected by cache_lock.
char *cur_hwdec;
char *decoder_desc;
bool try_spdif;
player: let frontend decide whether to use cover-art mode Essentially, this lets video.c decide whether to consider a video track cover art, instead of having the decoder wrapper use the lower level sh_stream flag. Some pain because of the dumb threading shit. Moving the code further down to make some of it part of the lock should not change behavior, although it could (framedrop nonsense). This commit should not change actual behavior, and is only preparation for the following commit.
2020-09-27 22:04:21 +00:00
bool attached_picture;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
bool pts_reset;
int attempt_framedrops; // try dropping this many frames
int dropped_frames; // total frames _probably_ dropped
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
};
options: Make validation and help possible for all option types Today, validation is only possible for string type options. But there's no particular reason why it needs to be restricted in this way, and there are potential uses, to allow other options to be validated without forcing the option to have to reimplement parsing from scratch. The first part, simply making the validation function an explicit field instead of overloading priv is simple enough. But if we only do that, then the validation function still needs to deal with the raw pre-parsed string. Instead, we want to allow the value to be parsed before it is validated. That in turn leads to us having validator functions that should be type aware. Unfortunately, that means we need to keep the explicit macro like OPT_STRING_VALIDATE() as a way to enforce the correct typing of the function. Otherwise, we'd have to have the validator take a void * and hope the implementation can cast it correctly. For help, we don't have this problem, as help doesn't look at the value. Then, we turn validators that are really help generators into explicit help functions and where a validator is help + validation, we split them into two parts. I have, however, left functions that need to query information for both help and validation as single functions to avoid code duplication. In this change, I have not added an other OPT_FOO_VALIDATE() macros as they are not needed, but I will add some in a separate change to illustrate the pattern.
2021-02-21 00:41:44 +00:00
static int decoder_list_help(struct mp_log *log, const m_option_t *opt,
struct bstr name)
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
{
if (strcmp(opt->name, "ad") == 0) {
struct mp_decoder_list *list = audio_decoder_list();
mp_print_decoders(log, MSGL_INFO, "Audio decoders:", list);
talloc_free(list);
return M_OPT_EXIT;
}
if (strcmp(opt->name, "vd") == 0) {
struct mp_decoder_list *list = video_decoder_list();
mp_print_decoders(log, MSGL_INFO, "Video decoders:", list);
talloc_free(list);
return M_OPT_EXIT;
}
if (strcmp(opt->name, "audio-spdif") == 0) {
mp_info(log, "Choices: ac3,dts-hd,dts (and possibly more)\n");
return M_OPT_EXIT;
}
return 1;
}
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
// Update cached values for main thread which require access to the decoder
// thread state. Must run on/locked with decoder thread.
static void update_cached_values(struct priv *p)
{
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
p->cur_hwdec = NULL;
if (p->decoder && p->decoder->control)
p->decoder->control(p->decoder->f, VDCTRL_GET_HWDEC, &p->cur_hwdec);
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
}
// Lock the decoder thread. This may synchronously wait until the decoder thread
// is done with its current work item (such as waiting for a frame), and thus
// may block for a while. (I.e. avoid during normal playback.)
// If no decoder thread is running, this is a no-op, except for some debug stuff.
static void thread_lock(struct priv *p)
{
if (p->dec_dispatch)
mp_dispatch_lock(p->dec_dispatch);
assert(!p->dec_thread_lock);
p->dec_thread_lock = true;
}
// Undo thread_lock().
static void thread_unlock(struct priv *p)
{
assert(p->dec_thread_lock);
p->dec_thread_lock = false;
if (p->dec_dispatch)
mp_dispatch_unlock(p->dec_dispatch);
}
f_decoder_wrapper: avoid full reset on timeline switch etc. Slightly cleaner. We don't need to awkwardly backup "some" state on backwards playback. Due to not resetting last_format, normal timeline switches don't unconditionally trigger recomputing of certain image parameters. Also probably doesn't reset framedrop parameters, although I don't care about that part.
2019-05-31 16:56:56 +00:00
// This resets only the decoder. Unlike a full reset(), this doesn't imply a
// seek reset. This distinction exists only when using timeline stuff (EDL and
// ordered chapters). timeline stuff needs to reset the decoder state, but keep
// some of the user-relevant state.
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
static void reset_decoder(struct priv *p)
{
p->first_packet_pdts = MP_NOPTS_VALUE;
p->start_pts = MP_NOPTS_VALUE;
p->codec_pts = MP_NOPTS_VALUE;
p->codec_dts = MP_NOPTS_VALUE;
f_decoder_wrapper: fully reset timestamp fixup logic on seeks This could lead to nonsense when backward playback is involved. Better reduce the possible interactions. Besides, it's better to fully reset things on seeks in general. The only exception is has_broken_packet_pts, which enables hr-seek if everything looks good. It's intended to trigger at the second hr-seek or so if the file is normal, and to disable it if the file is broken. It tries to avoid enabling the hr-seek logic before it can know about whether things are "good", so resetting it on seeks would obviously never enable it. Document it as explicit exception.
2019-05-31 15:51:46 +00:00
p->num_codec_pts_problems = 0;
p->num_codec_dts_problems = 0;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
p->has_broken_decoded_pts = 0;
p->packets_without_output = 0;
mp_frame_unref(&p->packet);
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
p->packet_fed = false;
f_decoder_wrapper: hack for discarding preroll in backward playback mode Some audio codecs will discard or cut the first frames when starting decoding. While some of that works through well-defined mechanisms (like initial padding), it's in general very codec/decoder specific, and not really predictable. In addition, libavcodec isn't very good with reporting "dropped" frames (and our internal interface reflects this). It seems our only chance to handle this is through timestamps. In theory, it would be best to discard frames that have timestamps before the "resume" position. But since video has reordered timestamps, we'd need to put some effort into finding this position. The first video packet doesn't necessarily contain this timestamp. (In theory, we could just do this in the demuxer with some trivial additional work, and set it on the packet's kf_seek_pts field. Although this field is supposed to contain just this value, the field is considered demuxer-internal, and I didn't want to make matters worse by reusing it for the interface to the decoder. With some more effort and buffering, we could calculate this value within the decoder, but fuck that.) The approach chosen in this commit is setting the timestamp to NOPTS. This will break in some obscure situations, but backward playback is a pretty obscure feature to begin with, so I considered this a reasonable implementation choice. Before passing a preroll packet to the decoder, its timestamps are set to NOPTS. Frames that are returned from the decoder and have the NOPTS timestamp are considered preroll and are discarded. This happens only during "preroll" mode (preroll_discard==true), so it doesn't affect normal forward playback. It's disabled on the first packet with a timestamp, so it can tolerate some crap even in backward playback mode. We don't check the dts fields out of laziness (decoded audio frames don't even have this field). I considered using an approach using the EDL clipping infrastructure (as mentioned in the last two paragraphs in the commit message of commit " demux_lavf: implement bad hack for backward playback of wav"). This didn't work, and I blamed timestamp rounding within mpv for it. But the problem was actually due to Matroska-rounded timestamps. Since the audio frame size isn't exactly aligned to 1ms, there will be an overlap (or gap) in the timestamps. This overlap is much smaller than 1ms, since it's just the sub-millisecond remainder part of the audio frame size. This makes the timestamps discontinuous and unreliable for the purpose we wanted to use it. We can't just smooth the timestamps in the demuxer either.
2019-05-31 14:07:13 +00:00
p->preroll_discard = false;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
talloc_free(p->new_segment);
p->new_segment = NULL;
p->start = p->end = MP_NOPTS_VALUE;
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
if (p->decoder)
mp_filter_reset(p->decoder->f);
}
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
static void decf_reset(struct mp_filter *f)
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
{
struct priv *p = f->priv;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
assert(p->decf == f);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
p->pts = MP_NOPTS_VALUE;
f_decoder_wrapper: avoid full reset on timeline switch etc. Slightly cleaner. We don't need to awkwardly backup "some" state on backwards playback. Due to not resetting last_format, normal timeline switches don't unconditionally trigger recomputing of certain image parameters. Also probably doesn't reset framedrop parameters, although I don't care about that part.
2019-05-31 16:56:56 +00:00
p->last_format = p->fixed_format = (struct mp_image_params){0};
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
p->pts_reset = false;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
p->attempt_framedrops = 0;
p->dropped_frames = 0;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
f_decoder_wrapper: avoid full reset on timeline switch etc. Slightly cleaner. We don't need to awkwardly backup "some" state on backwards playback. Due to not resetting last_format, normal timeline switches don't unconditionally trigger recomputing of certain image parameters. Also probably doesn't reset framedrop parameters, although I don't care about that part.
2019-05-31 16:56:56 +00:00
p->coverart_returned = 0;
for (int n = 0; n < p->num_reverse_queue; n++)
mp_frame_unref(&p->reverse_queue[n]);
p->num_reverse_queue = 0;
p->reverse_queue_byte_size = 0;
p->reverse_queue_complete = false;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
reset_decoder(p);
}
int mp_decoder_wrapper_control(struct mp_decoder_wrapper *d,
enum dec_ctrl cmd, void *arg)
{
struct priv *p = d->f->priv;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
int res = CONTROL_UNKNOWN;
if (cmd == VDCTRL_GET_HWDEC) {
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
*(char **)arg = p->cur_hwdec;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
} else {
thread_lock(p);
if (p->decoder && p->decoder->control)
res = p->decoder->control(p->decoder->f, cmd, arg);
update_cached_values(p);
thread_unlock(p);
}
return res;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
}
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
static void decf_destroy(struct mp_filter *f)
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
{
struct priv *p = f->priv;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
assert(p->decf == f);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
if (p->decoder) {
f_decoder_wrapper: reduce uninit message log level For vd/ad.
2019-11-01 00:55:06 +00:00
MP_DBG(f, "Uninit decoder.\n");
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
talloc_free(p->decoder->f);
p->decoder = NULL;
}
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
decf_reset(f);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
mp_frame_unref(&p->decoded_coverart);
}
struct mp_decoder_list *video_decoder_list(void)
{
struct mp_decoder_list *list = talloc_zero(NULL, struct mp_decoder_list);
vd_lavc.add_decoders(list);
return list;
}
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
struct mp_decoder_list *audio_decoder_list(void)
{
struct mp_decoder_list *list = talloc_zero(NULL, struct mp_decoder_list);
ad_lavc.add_decoders(list);
return list;
}
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
static bool reinit_decoder(struct priv *p)
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
{
if (p->decoder)
talloc_free(p->decoder->f);
p->decoder = NULL;
reset_decoder(p);
p->has_broken_packet_pts = -10; // needs 10 packets to reach decision
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
talloc_free(p->decoder_desc);
p->decoder_desc = NULL;
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
const struct mp_decoder_fns *driver = NULL;
struct mp_decoder_list *list = NULL;
char *user_list = NULL;
f_decoder_wrapper, sd_add: accept "null" codec This is for easier use with the "delay_open" feature added in the previous commit. The "null" codec is reported if the codec is unknown (because the stream was not opened yet at time the tracks were added). The rest of the timeline mechanism will set the correct codec at runtime. But this means every time a delay-loaded track is selected, it wants to initialize a decoder for the "null" codec. Accept a "null" decoder. But since FFmpeg has no such codec, and out of my own laziness, just let it fall back to "common" codecs that need no other initialization data.
2020-02-15 17:12:12 +00:00
char *fallback = NULL;
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
if (p->codec->type == STREAM_VIDEO) {
driver = &vd_lavc;
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
user_list = p->opts->video_decoders;
f_decoder_wrapper, sd_add: accept "null" codec This is for easier use with the "delay_open" feature added in the previous commit. The "null" codec is reported if the codec is unknown (because the stream was not opened yet at time the tracks were added). The rest of the timeline mechanism will set the correct codec at runtime. But this means every time a delay-loaded track is selected, it wants to initialize a decoder for the "null" codec. Accept a "null" decoder. But since FFmpeg has no such codec, and out of my own laziness, just let it fall back to "common" codecs that need no other initialization data.
2020-02-15 17:12:12 +00:00
fallback = "h264";
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
} else if (p->codec->type == STREAM_AUDIO) {
driver = &ad_lavc;
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
user_list = p->opts->audio_decoders;
f_decoder_wrapper, sd_add: accept "null" codec This is for easier use with the "delay_open" feature added in the previous commit. The "null" codec is reported if the codec is unknown (because the stream was not opened yet at time the tracks were added). The rest of the timeline mechanism will set the correct codec at runtime. But this means every time a delay-loaded track is selected, it wants to initialize a decoder for the "null" codec. Accept a "null" decoder. But since FFmpeg has no such codec, and out of my own laziness, just let it fall back to "common" codecs that need no other initialization data.
2020-02-15 17:12:12 +00:00
fallback = "aac";
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
bool try_spdif = p->try_spdif;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
if (try_spdif && p->codec->codec) {
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
struct mp_decoder_list *spdif =
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
select_spdif_codec(p->codec->codec, p->opts->audio_spdif);
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
if (spdif->num_entries) {
driver = &ad_spdif;
list = spdif;
} else {
talloc_free(spdif);
}
}
}
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
various: add some missing error checks
2023-11-18 22:30:42 +00:00
if (!driver)
return false;
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
if (!list) {
struct mp_decoder_list *full = talloc_zero(NULL, struct mp_decoder_list);
various: add some missing error checks
2023-11-18 22:30:42 +00:00
driver->add_decoders(full);
f_decoder_wrapper, sd_add: accept "null" codec This is for easier use with the "delay_open" feature added in the previous commit. The "null" codec is reported if the codec is unknown (because the stream was not opened yet at time the tracks were added). The rest of the timeline mechanism will set the correct codec at runtime. But this means every time a delay-loaded track is selected, it wants to initialize a decoder for the "null" codec. Accept a "null" decoder. But since FFmpeg has no such codec, and out of my own laziness, just let it fall back to "common" codecs that need no other initialization data.
2020-02-15 17:12:12 +00:00
const char *codec = p->codec->codec;
if (codec && strcmp(codec, "null") == 0)
codec = fallback;
list = mp_select_decoders(p->log, full, codec, user_list);
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
talloc_free(full);
}
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
mp_print_decoders(p->log, MSGL_V, "Codec list:", list);
for (int n = 0; n < list->num_entries; n++) {
struct mp_decoder_entry *sel = &list->entries[n];
MP_VERBOSE(p, "Opening decoder %s\n", sel->decoder);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
p->decoder = driver->create(p->decf, p->codec, sel->decoder);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
if (p->decoder) {
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
f_decoder_wrapper: change video-codec to show description or name Not both of them. Formating it as `<name> (<desc>)` produced arguably silly string like `hevc (HEVC (High Efficiency Video Coding))`. Unpack this to show only description if available or name otherwise. Produces way nicer results in stats.lua and similar places where this name is printed.
2023-09-01 01:15:10 +00:00
const char *d = sel->desc && sel->desc[0] ? sel->desc : sel->decoder;
p->decoder_desc = talloc_strdup(p, d);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
MP_VERBOSE(p, "Selected codec: %s\n", p->decoder_desc);
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
break;
}
MP_WARN(p, "Decoder init failed for %s\n", sel->decoder);
}
if (!p->decoder) {
MP_ERR(p, "Failed to initialize a decoder for codec '%s'.\n",
p->codec->codec ? p->codec->codec : "<?>");
}
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
update_cached_values(p);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
talloc_free(list);
return !!p->decoder;
}
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
bool mp_decoder_wrapper_reinit(struct mp_decoder_wrapper *d)
{
struct priv *p = d->f->priv;
thread_lock(p);
bool res = reinit_decoder(p);
thread_unlock(p);
return res;
}
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
void mp_decoder_wrapper_get_desc(struct mp_decoder_wrapper *d,
char *buf, size_t buf_size)
{
struct priv *p = d->f->priv;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
snprintf(buf, buf_size, "%s", p->decoder_desc ? p->decoder_desc : "");
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
}
void mp_decoder_wrapper_set_frame_drops(struct mp_decoder_wrapper *d, int num)
{
struct priv *p = d->f->priv;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
p->attempt_framedrops = num;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
}
int mp_decoder_wrapper_get_frames_dropped(struct mp_decoder_wrapper *d)
{
struct priv *p = d->f->priv;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
int res = p->dropped_frames;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
return res;
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
}
double mp_decoder_wrapper_get_container_fps(struct mp_decoder_wrapper *d)
{
struct priv *p = d->f->priv;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
thread_lock(p);
double res = p->fps;
thread_unlock(p);
return res;
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
}
void mp_decoder_wrapper_set_spdif_flag(struct mp_decoder_wrapper *d, bool spdif)
{
struct priv *p = d->f->priv;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
p->try_spdif = spdif;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
}
player: let frontend decide whether to use cover-art mode Essentially, this lets video.c decide whether to consider a video track cover art, instead of having the decoder wrapper use the lower level sh_stream flag. Some pain because of the dumb threading shit. Moving the code further down to make some of it part of the lock should not change behavior, although it could (framedrop nonsense). This commit should not change actual behavior, and is only preparation for the following commit.
2020-09-27 22:04:21 +00:00
void mp_decoder_wrapper_set_coverart_flag(struct mp_decoder_wrapper *d, bool c)
{
struct priv *p = d->f->priv;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
player: let frontend decide whether to use cover-art mode Essentially, this lets video.c decide whether to consider a video track cover art, instead of having the decoder wrapper use the lower level sh_stream flag. Some pain because of the dumb threading shit. Moving the code further down to make some of it part of the lock should not change behavior, although it could (framedrop nonsense). This commit should not change actual behavior, and is only preparation for the following commit.
2020-09-27 22:04:21 +00:00
p->attached_picture = c;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
player: let frontend decide whether to use cover-art mode Essentially, this lets video.c decide whether to consider a video track cover art, instead of having the decoder wrapper use the lower level sh_stream flag. Some pain because of the dumb threading shit. Moving the code further down to make some of it part of the lock should not change behavior, although it could (framedrop nonsense). This commit should not change actual behavior, and is only preparation for the following commit.
2020-09-27 22:04:21 +00:00
}
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
bool mp_decoder_wrapper_get_pts_reset(struct mp_decoder_wrapper *d)
{
struct priv *p = d->f->priv;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
bool res = p->pts_reset;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
return res;
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
}
void mp_decoder_wrapper_set_play_dir(struct mp_decoder_wrapper *d, int dir)
{
struct priv *p = d->f->priv;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
thread_lock(p);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
p->play_dir = dir;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
thread_unlock(p);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
}
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
static void fix_image_params(struct priv *p,
struct mp_image_params *params)
{
struct mp_image_params m = *params;
struct mp_codec_params *c = p->codec;
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
struct dec_wrapper_opts *opts = p->opts;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
MP_VERBOSE(p, "Decoder format: %s\n", mp_image_params_to_str(params));
p->dec_format = *params;
// While mp_image_params normally always have to have d_w/d_h set, the
// decoder signals unknown bitstream aspect ratio with both set to 0.
bool use_container = true;
if (opts->aspect_method == 1 && m.p_w > 0 && m.p_h > 0) {
MP_VERBOSE(p, "Using bitstream aspect ratio.\n");
use_container = false;
}
if (use_container && c->par_w > 0 && c->par_h) {
MP_VERBOSE(p, "Using container aspect ratio.\n");
m.p_w = c->par_w;
m.p_h = c->par_h;
}
if (opts->movie_aspect >= 0) {
MP_VERBOSE(p, "Forcing user-set aspect ratio.\n");
if (opts->movie_aspect == 0) {
m.p_w = m.p_h = 1;
} else {
AVRational a = av_d2q(opts->movie_aspect, INT_MAX);
mp_image_params_set_dsize(&m, a.num, a.den);
}
}
// Assume square pixels if no aspect ratio is set at all.
if (m.p_w <= 0 || m.p_h <= 0)
m.p_w = m.p_h = 1;
video: remove internal stereo_out flag Also rename stereo3d to stereo_in. The only real change is that the vo_gpu OSD code now uses the actual stereo 3D mode, instead of the --video-steroe-mode value. (Why does this vo_gpu code even exist?)
2018-04-21 11:31:00 +00:00
m.stereo3d = p->codec->stereo_mode;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
demux: add crop to mp_codec_params
2023-09-01 20:26:05 +00:00
if (!mp_rect_equals(&p->codec->crop, &(struct mp_rect){0})) {
struct mp_rect crop = p->codec->crop;
// Offset to respect existing decoder crop.
crop.x0 += m.crop.x0;
crop.x1 += m.crop.x0;
crop.y0 += m.crop.y0;
crop.y1 += m.crop.y0;
// Crop has to be inside existing image bounds.
if (mp_image_crop_valid(&(struct mp_image_params) {
.w = mp_rect_w(m.crop), .h = mp_rect_h(m.crop), .crop = crop }))
{
m.crop = crop;
} else {
MP_WARN(p, "Invalid container crop %dx%d+%d+%d for %dx%d image\n",
mp_rect_w(crop), mp_rect_h(crop), crop.x0, crop.y0,
mp_rect_w(m.crop), mp_rect_h(m.crop));
}
}
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
if (opts->video_rotate < 0) {
m.rotate = 0;
} else {
mp_image, f_decoder_wrapper: implement AV_FRAME_DATA_DISPLAYMATRIX fixes #9249 (JPEG orientation support) with ffmpeg commit [0]. [0] https://github.com/FFmpeg/FFmpeg/commit/e93c9986027d17917c3b4f533b28ee4a2ce7cd4c
2021-10-02 17:19:10 +00:00
// ffmpeg commit 535a835e51 says that frame rotate takes priority
if (!m.rotate)
m.rotate = p->codec->rotate;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
m.rotate = (m.rotate + opts->video_rotate) % 360;
}
mp_colorspace_merge(&m.color, &c->color);
// Guess missing colorspace fields from metadata. This guarantees all
// fields are at least set to legal values afterwards.
mp_image_params_guess_csp(&m);
p->last_format = *params;
p->fixed_format = m;
}
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
void mp_decoder_wrapper_reset_params(struct mp_decoder_wrapper *d)
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
{
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
struct priv *p = d->f->priv;
p->last_format = (struct mp_image_params){0};
}
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
void mp_decoder_wrapper_get_video_dec_params(struct mp_decoder_wrapper *d,
struct mp_image_params *m)
{
struct priv *p = d->f->priv;
*m = p->dec_format;
}
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
// This code exists only because multimedia is so god damn crazy. In a sane
// world, the video decoder would always output a video frame with a valid PTS;
// this deals with cases where it doesn't.
static void crazy_video_pts_stuff(struct priv *p, struct mp_image *mpi)
{
// Note: the PTS is reordered, but the DTS is not. Both must be monotonic.
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
if (mpi->pts != MP_NOPTS_VALUE) {
if (mpi->pts < p->codec_pts)
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
p->num_codec_pts_problems++;
p->codec_pts = mpi->pts;
}
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
if (mpi->dts != MP_NOPTS_VALUE) {
if (mpi->dts <= p->codec_dts)
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
p->num_codec_dts_problems++;
p->codec_dts = mpi->dts;
}
if (p->has_broken_packet_pts < 0)
p->has_broken_packet_pts++;
if (p->num_codec_pts_problems)
p->has_broken_packet_pts = 1;
// If PTS is unset, or non-monotonic, fall back to DTS.
if ((p->num_codec_pts_problems > p->num_codec_dts_problems ||
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
mpi->pts == MP_NOPTS_VALUE) && mpi->dts != MP_NOPTS_VALUE)
mpi->pts = mpi->dts;
// Compensate for incorrectly using mpeg-style DTS for avi timestamps.
if (p->decoder && p->decoder->control && p->codec->avi_dts &&
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
mpi->pts != MP_NOPTS_VALUE && p->fps > 0)
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
{
int delay = -1;
p->decoder->control(p->decoder->f, VDCTRL_GET_BFRAMES, &delay);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
mpi->pts -= MPMAX(delay, 0) / p->fps;
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
}
}
// Return true if the current frame is outside segment range.
static bool process_decoded_frame(struct priv *p, struct mp_frame *frame)
{
if (frame->type == MP_FRAME_EOF) {
// if we were just draining current segment, don't propagate EOF
if (p->new_segment)
mp_frame_unref(frame);
return true;
}
bool segment_ended = false;
if (frame->type == MP_FRAME_VIDEO) {
struct mp_image *mpi = frame->data;
crazy_video_pts_stuff(p, mpi);
struct demux_packet *ccpkt = new_demux_packet_from_buf(mpi->a53_cc);
if (ccpkt) {
av_buffer_unref(&mpi->a53_cc);
ccpkt->pts = mpi->pts;
ccpkt->dts = mpi->dts;
demuxer_feed_caption(p->header, ccpkt);
}
// Stop hr-seek logic.
if (mpi->pts == MP_NOPTS_VALUE || mpi->pts >= p->start_pts)
p->start_pts = MP_NOPTS_VALUE;
if (mpi->pts != MP_NOPTS_VALUE) {
segment_ended = p->end != MP_NOPTS_VALUE && mpi->pts >= p->end;
if ((p->start != MP_NOPTS_VALUE && mpi->pts < p->start) ||
segment_ended)
{
mp_frame_unref(frame);
goto done;
}
}
} else if (frame->type == MP_FRAME_AUDIO) {
struct mp_aframe *aframe = frame->data;
mp_aframe_clip_timestamps(aframe, p->start, p->end);
double pts = mp_aframe_get_pts(aframe);
if (pts != MP_NOPTS_VALUE && p->start != MP_NOPTS_VALUE)
segment_ended = pts >= p->end;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
if (mp_aframe_get_size(aframe) == 0) {
mp_frame_unref(frame);
goto done;
}
} else {
MP_ERR(p, "unknown frame type from decoder\n");
}
done:
return segment_ended;
}
static void correct_video_pts(struct priv *p, struct mp_image *mpi)
{
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
mpi->pts *= p->play_dir;
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
if (!p->opts->correct_pts || mpi->pts == MP_NOPTS_VALUE) {
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
double fps = p->fps > 0 ? p->fps : 25;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
if (p->opts->correct_pts) {
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
if (p->has_broken_decoded_pts <= 1) {
f_decoder_wrapper: fix a typo in log message
2018-04-26 16:29:41 +00:00
MP_WARN(p, "No video PTS! Making something up. Using "
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
"%f FPS.\n", fps);
if (p->has_broken_decoded_pts == 1)
MP_WARN(p, "Ignoring further missing PTS warnings.\n");
p->has_broken_decoded_pts++;
}
}
double frame_time = 1.0f / fps;
double base = p->first_packet_pdts;
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
mpi->pts = p->pts;
if (mpi->pts == MP_NOPTS_VALUE) {
mpi->pts = base == MP_NOPTS_VALUE ? 0 : base;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
} else {
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
mpi->pts += frame_time;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
}
}
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
p->pts = mpi->pts;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
}
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
static void correct_audio_pts(struct priv *p, struct mp_aframe *aframe)
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
{
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
double dir = p->play_dir;
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
double frame_pts = mp_aframe_get_pts(aframe);
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
double frame_len = mp_aframe_duration(aframe);
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
if (frame_pts != MP_NOPTS_VALUE) {
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
if (dir < 0)
frame_pts = -(frame_pts + frame_len);
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
if (p->pts != MP_NOPTS_VALUE)
MP_STATS(p, "value %f audio-pts-err", p->pts - frame_pts);
audio: move back PTS jump detection to before filter chain The recent changes to player/audio.c moved PTS jump detection to after audio filtering. This was mostly done for convenience, because dataflow between decoder and filters was made "automatic", and jump detection would have to be done as filter. Now move it back to after decoders, again out of convenience. The future direction is to make the dataflow between filters and AO automatic, so this is a bit in the way. Another reason is that speed changes tend to cause jumps - these are legitimate, but get annoying quickly.
2018-02-12 17:00:23 +00:00
double diff = fabs(p->pts - frame_pts);
// Attempt to detect jumps in PTS. Even for the lowest sample rates and
// with worst container rounded timestamp, this should be a margin more
// than enough.
if (p->pts != MP_NOPTS_VALUE && diff > 0.1) {
MP_WARN(p, "Invalid audio PTS: %f -> %f\n", p->pts, frame_pts);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
if (diff >= 5) {
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
p->pts_reset = true;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
}
audio: move back PTS jump detection to before filter chain The recent changes to player/audio.c moved PTS jump detection to after audio filtering. This was mostly done for convenience, because dataflow between decoder and filters was made "automatic", and jump detection would have to be done as filter. Now move it back to after decoders, again out of convenience. The future direction is to make the dataflow between filters and AO automatic, so this is a bit in the way. Another reason is that speed changes tend to cause jumps - these are legitimate, but get annoying quickly.
2018-02-12 17:00:23 +00:00
}
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
// Keep the interpolated timestamp if it doesn't deviate more
// than 1 ms from the real one. (MKV rounded timestamps.)
audio: move back PTS jump detection to before filter chain The recent changes to player/audio.c moved PTS jump detection to after audio filtering. This was mostly done for convenience, because dataflow between decoder and filters was made "automatic", and jump detection would have to be done as filter. Now move it back to after decoders, again out of convenience. The future direction is to make the dataflow between filters and AO automatic, so this is a bit in the way. Another reason is that speed changes tend to cause jumps - these are legitimate, but get annoying quickly.
2018-02-12 17:00:23 +00:00
if (p->pts == MP_NOPTS_VALUE || diff > 0.001)
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
p->pts = frame_pts;
}
if (p->pts == MP_NOPTS_VALUE && p->header->missing_timestamps)
p->pts = 0;
mp_aframe_set_pts(aframe, p->pts);
if (p->pts != MP_NOPTS_VALUE)
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
p->pts += frame_len;
}
static void process_output_frame(struct priv *p, struct mp_frame frame)
{
if (frame.type == MP_FRAME_VIDEO) {
struct mp_image *mpi = frame.data;
correct_video_pts(p, mpi);
if (!mp_image_params_equal(&p->last_format, &mpi->params))
fix_image_params(p, &mpi->params);
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
mpi->params = p->fixed_format;
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
mpi->nominal_fps = p->fps;
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
} else if (frame.type == MP_FRAME_AUDIO) {
struct mp_aframe *aframe = frame.data;
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
if (p->play_dir < 0 && !mp_aframe_reverse(aframe))
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
MP_ERR(p, "Couldn't reverse audio frame.\n");
correct_audio_pts(p, aframe);
}
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
}
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
void mp_decoder_wrapper_set_start_pts(struct mp_decoder_wrapper *d, double pts)
{
struct priv *p = d->f->priv;
p->start_pts = pts;
}
static bool is_new_segment(struct priv *p, struct mp_frame frame)
{
if (frame.type != MP_FRAME_PACKET)
return false;
struct demux_packet *pkt = frame.data;
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
return (pkt->segmented && (pkt->start != p->start || pkt->end != p->end ||
pkt->codec != p->codec)) ||
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
(p->play_dir < 0 && pkt->back_restart && p->packet_fed);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
}
static void feed_packet(struct priv *p)
{
if (!p->decoder || !mp_pin_in_needs_data(p->decoder->f->pins[0]))
return;
demux, f_decoder_wrapper: fix coverart in backward mode Shitty ancient hack that wastes my time all the time. demux.c: always return the coverart packet as soon as possible, and don't let the backward demux state machine possibly stop it. f_decoder_wrapper.c: mess with some shit until it somehow starts to work. I think the old code tried to let it cleverly fall through so the packet was processed "normally"; just make it run the "usual" code instead.
2019-05-25 19:57:31 +00:00
if (p->decoded_coverart.type)
return;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
if (!p->packet.type && !p->new_segment) {
p->packet = mp_pin_out_read(p->demux);
if (!p->packet.type)
return;
if (p->packet.type != MP_FRAME_EOF && p->packet.type != MP_FRAME_PACKET) {
MP_ERR(p, "invalid frame type from demuxer\n");
mp_frame_unref(&p->packet);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
mp_filter_internal_mark_failed(p->decf);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
return;
}
}
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
if (!p->packet.type)
return;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
// Flush current data if the packet is a new segment.
if (is_new_segment(p, p->packet)) {
assert(!p->new_segment);
p->new_segment = p->packet.data;
p->packet = MP_EOF_FRAME;
}
assert(p->packet.type == MP_FRAME_PACKET || p->packet.type == MP_FRAME_EOF);
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
struct demux_packet *packet =
p->packet.type == MP_FRAME_PACKET ? p->packet.data : NULL;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
// For video framedropping, including parts of the hr-seek logic.
if (p->decoder->control) {
double start_pts = p->start_pts;
if (p->start != MP_NOPTS_VALUE && (start_pts == MP_NOPTS_VALUE ||
p->start > start_pts))
start_pts = p->start;
int framedrop_type = 0;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
if (p->attempt_framedrops)
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
framedrop_type = 1;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
if (start_pts != MP_NOPTS_VALUE && packet && p->play_dir > 0 &&
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
packet->pts < start_pts - .005 && !p->has_broken_packet_pts)
framedrop_type = 2;
p->decoder->control(p->decoder->f, VDCTRL_SET_FRAMEDROP, &framedrop_type);
}
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
if (!p->dec_dispatch && p->public.recorder_sink)
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
mp_recorder_feed_packet(p->public.recorder_sink, packet);
double pkt_pts = packet ? packet->pts : MP_NOPTS_VALUE;
double pkt_dts = packet ? packet->dts : MP_NOPTS_VALUE;
if (pkt_pts == MP_NOPTS_VALUE)
p->has_broken_packet_pts = 1;
if (packet && packet->dts == MP_NOPTS_VALUE && !p->codec->avi_dts)
packet->dts = packet->pts;
double pkt_pdts = pkt_pts == MP_NOPTS_VALUE ? pkt_dts : pkt_pts;
if (p->first_packet_pdts == MP_NOPTS_VALUE)
p->first_packet_pdts = pkt_pdts;
f_decoder_wrapper: hack for discarding preroll in backward playback mode Some audio codecs will discard or cut the first frames when starting decoding. While some of that works through well-defined mechanisms (like initial padding), it's in general very codec/decoder specific, and not really predictable. In addition, libavcodec isn't very good with reporting "dropped" frames (and our internal interface reflects this). It seems our only chance to handle this is through timestamps. In theory, it would be best to discard frames that have timestamps before the "resume" position. But since video has reordered timestamps, we'd need to put some effort into finding this position. The first video packet doesn't necessarily contain this timestamp. (In theory, we could just do this in the demuxer with some trivial additional work, and set it on the packet's kf_seek_pts field. Although this field is supposed to contain just this value, the field is considered demuxer-internal, and I didn't want to make matters worse by reusing it for the interface to the decoder. With some more effort and buffering, we could calculate this value within the decoder, but fuck that.) The approach chosen in this commit is setting the timestamp to NOPTS. This will break in some obscure situations, but backward playback is a pretty obscure feature to begin with, so I considered this a reasonable implementation choice. Before passing a preroll packet to the decoder, its timestamps are set to NOPTS. Frames that are returned from the decoder and have the NOPTS timestamp are considered preroll and are discarded. This happens only during "preroll" mode (preroll_discard==true), so it doesn't affect normal forward playback. It's disabled on the first packet with a timestamp, so it can tolerate some crap even in backward playback mode. We don't check the dts fields out of laziness (decoded audio frames don't even have this field). I considered using an approach using the EDL clipping infrastructure (as mentioned in the last two paragraphs in the commit message of commit " demux_lavf: implement bad hack for backward playback of wav"). This didn't work, and I blamed timestamp rounding within mpv for it. But the problem was actually due to Matroska-rounded timestamps. Since the audio frame size isn't exactly aligned to 1ms, there will be an overlap (or gap) in the timestamps. This overlap is much smaller than 1ms, since it's just the sub-millisecond remainder part of the audio frame size. This makes the timestamps discontinuous and unreliable for the purpose we wanted to use it. We can't just smooth the timestamps in the demuxer either.
2019-05-31 14:07:13 +00:00
if (packet && packet->back_preroll) {
p->preroll_discard = true;
packet->pts = packet->dts = MP_NOPTS_VALUE;
}
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
mp_pin_in_write(p->decoder->f->pins[0], p->packet);
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
p->packet_fed = true;
f_decoder_wrapper: hack for discarding preroll in backward playback mode Some audio codecs will discard or cut the first frames when starting decoding. While some of that works through well-defined mechanisms (like initial padding), it's in general very codec/decoder specific, and not really predictable. In addition, libavcodec isn't very good with reporting "dropped" frames (and our internal interface reflects this). It seems our only chance to handle this is through timestamps. In theory, it would be best to discard frames that have timestamps before the "resume" position. But since video has reordered timestamps, we'd need to put some effort into finding this position. The first video packet doesn't necessarily contain this timestamp. (In theory, we could just do this in the demuxer with some trivial additional work, and set it on the packet's kf_seek_pts field. Although this field is supposed to contain just this value, the field is considered demuxer-internal, and I didn't want to make matters worse by reusing it for the interface to the decoder. With some more effort and buffering, we could calculate this value within the decoder, but fuck that.) The approach chosen in this commit is setting the timestamp to NOPTS. This will break in some obscure situations, but backward playback is a pretty obscure feature to begin with, so I considered this a reasonable implementation choice. Before passing a preroll packet to the decoder, its timestamps are set to NOPTS. Frames that are returned from the decoder and have the NOPTS timestamp are considered preroll and are discarded. This happens only during "preroll" mode (preroll_discard==true), so it doesn't affect normal forward playback. It's disabled on the first packet with a timestamp, so it can tolerate some crap even in backward playback mode. We don't check the dts fields out of laziness (decoded audio frames don't even have this field). I considered using an approach using the EDL clipping infrastructure (as mentioned in the last two paragraphs in the commit message of commit " demux_lavf: implement bad hack for backward playback of wav"). This didn't work, and I blamed timestamp rounding within mpv for it. But the problem was actually due to Matroska-rounded timestamps. Since the audio frame size isn't exactly aligned to 1ms, there will be an overlap (or gap) in the timestamps. This overlap is much smaller than 1ms, since it's just the sub-millisecond remainder part of the audio frame size. This makes the timestamps discontinuous and unreliable for the purpose we wanted to use it. We can't just smooth the timestamps in the demuxer either.
2019-05-31 14:07:13 +00:00
p->packet = MP_NO_FRAME;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
p->packets_without_output += 1;
}
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
static void enqueue_backward_frame(struct priv *p, struct mp_frame frame)
{
bool eof = frame.type == MP_FRAME_EOF;
if (!eof) {
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
struct dec_wrapper_opts *opts = p->opts;
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
uint64_t queue_size = 0;
switch (p->header->type) {
case STREAM_VIDEO: queue_size = opts->video_reverse_size; break;
case STREAM_AUDIO: queue_size = opts->audio_reverse_size; break;
}
if (p->reverse_queue_byte_size >= queue_size) {
MP_ERR(p, "Reversal queue overflow, discarding frame.\n");
mp_frame_unref(&frame);
return;
}
p->reverse_queue_byte_size += mp_frame_approx_size(frame);
}
// Note: EOF (really BOF) is propagated, but not reversed.
MP_TARRAY_INSERT_AT(p, p->reverse_queue, p->num_reverse_queue,
eof ? 0 : p->num_reverse_queue, frame);
p->reverse_queue_complete = eof;
}
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
static void read_frame(struct priv *p)
{
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
struct mp_pin *pin = p->decf->ppins[0];
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
struct mp_frame frame = {0};
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
if (!p->decoder || !mp_pin_in_needs_data(pin))
return;
if (p->decoded_coverart.type) {
if (p->coverart_returned == 0) {
f_decoder_wrapper: put coverart through image output logic This wasn't done, probably regression from one of the last dozen of times this special code path was touched. This meant coverart images ignored the user-set aspect ratio completely, and some other things.
2019-11-16 22:21:03 +00:00
frame = mp_frame_ref(p->decoded_coverart);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
p->coverart_returned = 1;
f_decoder_wrapper: put coverart through image output logic This wasn't done, probably regression from one of the last dozen of times this special code path was touched. This meant coverart images ignored the user-set aspect ratio completely, and some other things.
2019-11-16 22:21:03 +00:00
goto output_frame;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
} else if (p->coverart_returned == 1) {
f_decoder_wrapper: put coverart through image output logic This wasn't done, probably regression from one of the last dozen of times this special code path was touched. This meant coverart images ignored the user-set aspect ratio completely, and some other things.
2019-11-16 22:21:03 +00:00
frame = MP_EOF_FRAME;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
p->coverart_returned = 2;
f_decoder_wrapper: put coverart through image output logic This wasn't done, probably regression from one of the last dozen of times this special code path was touched. This meant coverart images ignored the user-set aspect ratio completely, and some other things.
2019-11-16 22:21:03 +00:00
goto output_frame;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
}
return;
}
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
if (p->reverse_queue_complete && p->num_reverse_queue) {
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
frame = p->reverse_queue[p->num_reverse_queue - 1];
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
p->num_reverse_queue -= 1;
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
goto output_frame;
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
}
p->reverse_queue_complete = false;
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
frame = mp_pin_out_read(p->decoder->f->pins[1]);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
if (!frame.type)
return;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_lock(&p->cache_lock);
player: let frontend decide whether to use cover-art mode Essentially, this lets video.c decide whether to consider a video track cover art, instead of having the decoder wrapper use the lower level sh_stream flag. Some pain because of the dumb threading shit. Moving the code further down to make some of it part of the lock should not change behavior, although it could (framedrop nonsense). This commit should not change actual behavior, and is only preparation for the following commit.
2020-09-27 22:04:21 +00:00
if (p->attached_picture && frame.type == MP_FRAME_VIDEO)
p->decoded_coverart = frame;
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
if (p->attempt_framedrops) {
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
int dropped = MPMAX(0, p->packets_without_output - 1);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
p->attempt_framedrops = MPMAX(0, p->attempt_framedrops - dropped);
p->dropped_frames += dropped;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
}
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_unlock(&p->cache_lock);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
player: let frontend decide whether to use cover-art mode Essentially, this lets video.c decide whether to consider a video track cover art, instead of having the decoder wrapper use the lower level sh_stream flag. Some pain because of the dumb threading shit. Moving the code further down to make some of it part of the lock should not change behavior, although it could (framedrop nonsense). This commit should not change actual behavior, and is only preparation for the following commit.
2020-09-27 22:04:21 +00:00
if (p->decoded_coverart.type) {
mp_filter_internal_mark_progress(p->decf);
return;
}
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
p->packets_without_output = 0;
f_decoder_wrapper: hack for discarding preroll in backward playback mode Some audio codecs will discard or cut the first frames when starting decoding. While some of that works through well-defined mechanisms (like initial padding), it's in general very codec/decoder specific, and not really predictable. In addition, libavcodec isn't very good with reporting "dropped" frames (and our internal interface reflects this). It seems our only chance to handle this is through timestamps. In theory, it would be best to discard frames that have timestamps before the "resume" position. But since video has reordered timestamps, we'd need to put some effort into finding this position. The first video packet doesn't necessarily contain this timestamp. (In theory, we could just do this in the demuxer with some trivial additional work, and set it on the packet's kf_seek_pts field. Although this field is supposed to contain just this value, the field is considered demuxer-internal, and I didn't want to make matters worse by reusing it for the interface to the decoder. With some more effort and buffering, we could calculate this value within the decoder, but fuck that.) The approach chosen in this commit is setting the timestamp to NOPTS. This will break in some obscure situations, but backward playback is a pretty obscure feature to begin with, so I considered this a reasonable implementation choice. Before passing a preroll packet to the decoder, its timestamps are set to NOPTS. Frames that are returned from the decoder and have the NOPTS timestamp are considered preroll and are discarded. This happens only during "preroll" mode (preroll_discard==true), so it doesn't affect normal forward playback. It's disabled on the first packet with a timestamp, so it can tolerate some crap even in backward playback mode. We don't check the dts fields out of laziness (decoded audio frames don't even have this field). I considered using an approach using the EDL clipping infrastructure (as mentioned in the last two paragraphs in the commit message of commit " demux_lavf: implement bad hack for backward playback of wav"). This didn't work, and I blamed timestamp rounding within mpv for it. But the problem was actually due to Matroska-rounded timestamps. Since the audio frame size isn't exactly aligned to 1ms, there will be an overlap (or gap) in the timestamps. This overlap is much smaller than 1ms, since it's just the sub-millisecond remainder part of the audio frame size. This makes the timestamps discontinuous and unreliable for the purpose we wanted to use it. We can't just smooth the timestamps in the demuxer either.
2019-05-31 14:07:13 +00:00
if (p->preroll_discard && frame.type != MP_FRAME_EOF) {
double ts = mp_frame_get_pts(frame);
if (ts == MP_NOPTS_VALUE) {
mp_frame_unref(&frame);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
mp_filter_internal_mark_progress(p->decf);
f_decoder_wrapper: hack for discarding preroll in backward playback mode Some audio codecs will discard or cut the first frames when starting decoding. While some of that works through well-defined mechanisms (like initial padding), it's in general very codec/decoder specific, and not really predictable. In addition, libavcodec isn't very good with reporting "dropped" frames (and our internal interface reflects this). It seems our only chance to handle this is through timestamps. In theory, it would be best to discard frames that have timestamps before the "resume" position. But since video has reordered timestamps, we'd need to put some effort into finding this position. The first video packet doesn't necessarily contain this timestamp. (In theory, we could just do this in the demuxer with some trivial additional work, and set it on the packet's kf_seek_pts field. Although this field is supposed to contain just this value, the field is considered demuxer-internal, and I didn't want to make matters worse by reusing it for the interface to the decoder. With some more effort and buffering, we could calculate this value within the decoder, but fuck that.) The approach chosen in this commit is setting the timestamp to NOPTS. This will break in some obscure situations, but backward playback is a pretty obscure feature to begin with, so I considered this a reasonable implementation choice. Before passing a preroll packet to the decoder, its timestamps are set to NOPTS. Frames that are returned from the decoder and have the NOPTS timestamp are considered preroll and are discarded. This happens only during "preroll" mode (preroll_discard==true), so it doesn't affect normal forward playback. It's disabled on the first packet with a timestamp, so it can tolerate some crap even in backward playback mode. We don't check the dts fields out of laziness (decoded audio frames don't even have this field). I considered using an approach using the EDL clipping infrastructure (as mentioned in the last two paragraphs in the commit message of commit " demux_lavf: implement bad hack for backward playback of wav"). This didn't work, and I blamed timestamp rounding within mpv for it. But the problem was actually due to Matroska-rounded timestamps. Since the audio frame size isn't exactly aligned to 1ms, there will be an overlap (or gap) in the timestamps. This overlap is much smaller than 1ms, since it's just the sub-millisecond remainder part of the audio frame size. This makes the timestamps discontinuous and unreliable for the purpose we wanted to use it. We can't just smooth the timestamps in the demuxer either.
2019-05-31 14:07:13 +00:00
return;
}
p->preroll_discard = false;
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
}
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
bool segment_ended = process_decoded_frame(p, &frame);
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
if (p->play_dir < 0 && frame.type) {
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
enqueue_backward_frame(p, frame);
frame = MP_NO_FRAME;
}
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
// If there's a new segment, start it as soon as we're drained/finished.
if (segment_ended && p->new_segment) {
struct demux_packet *new_segment = p->new_segment;
p->new_segment = NULL;
reset_decoder(p);
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
if (new_segment->segmented) {
if (p->codec != new_segment->codec) {
p->codec = new_segment->codec;
if (!mp_decoder_wrapper_reinit(&p->public))
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
mp_filter_internal_mark_failed(p->decf);
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
}
p->start = new_segment->start;
p->end = new_segment->end;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
}
f_decoder_wrapper: avoid full reset on timeline switch etc. Slightly cleaner. We don't need to awkwardly backup "some" state on backwards playback. Due to not resetting last_format, normal timeline switches don't unconditionally trigger recomputing of certain image parameters. Also probably doesn't reset framedrop parameters, although I don't care about that part.
2019-05-31 16:56:56 +00:00
p->reverse_queue_byte_size = 0;
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
p->reverse_queue_complete = p->num_reverse_queue > 0;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
p->packet = MAKE_FRAME(MP_FRAME_PACKET, new_segment);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
mp_filter_internal_mark_progress(p->decf);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
}
if (!frame.type) {
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
mp_filter_internal_mark_progress(p->decf); // make it retry
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
return;
}
f_decoder_wrapper: reorganize, fix EDL/ordered chapters backward playback Before this commit, there was a single process_decoded_frame() function. It handled various aspects of dealing with a newly decoded frame. Move some of these to a separate process_output_frame() function. This new function is called in the order the frames are returned to the playback core. Some correct_audio_pts() (was process_audio_frame()) becomes slightly less awkward due to this, and the timestamp smoothing can actually work in backward playback mode now (thus moving p->pts out of reset_decoder()). Behavior for normal playback also changes subtly. This shouldn't matter in sane cases, but if you mix broken files, --no-correct-pts, and timeline stuff, differences in behavior might be visible. Timeline clipping (EDL/ordered chapters) works now, because it's done before "transforming" the timestamps. Audio timestamp smoothing happens after it, which is a behavior change, but should be more correct. This still runs crazy_video_pts_stuff() before everything else. On the pther hand, --no-correct-pts or missing timestamp processing is done last. But these things didn't really work with timeline before.
2019-05-31 17:52:26 +00:00
output_frame:
process_output_frame(p, frame);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
mp_pin_in_write(pin, frame);
}
f_decoder_wrapper: make most queue options runtime changeable Why not.
2020-02-29 23:38:34 +00:00
static void update_queue_config(struct priv *p)
{
if (!p->queue)
return;
struct mp_async_queue_config cfg = {
.max_bytes = p->queue_opts->max_bytes,
.sample_unit = AQUEUE_UNIT_SAMPLES,
.max_samples = p->queue_opts->max_samples,
.max_duration = p->queue_opts->max_duration,
};
mp_async_queue_set_config(p->queue, cfg);
}
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
static void decf_process(struct mp_filter *f)
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
{
struct priv *p = f->priv;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
assert(p->decf == f);
f_decoder_wrapper: make most queue options runtime changeable Why not.
2020-02-29 23:38:34 +00:00
if (m_config_cache_update(p->opt_cache))
update_queue_config(p);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
feed_packet(p);
read_frame(p);
}
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
static MP_THREAD_VOID dec_thread(void *ptr)
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
{
struct priv *p = ptr;
mp_threads: rename threads for consistent naming across all of them I'd like some names to be more descriptive, but to work with 15 chars limit we have to make some sacrifice. Also because of the limit, remove the `mpv/` prefix and prioritize actuall thread name.
2023-10-22 18:59:34 +00:00
char *t_name = "dec/?";
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
switch (p->header->type) {
mp_threads: rename threads for consistent naming across all of them I'd like some names to be more descriptive, but to work with 15 chars limit we have to make some sacrifice. Also because of the limit, remove the `mpv/` prefix and prioritize actuall thread name.
2023-10-22 18:59:34 +00:00
case STREAM_VIDEO: t_name = "dec/video"; break;
case STREAM_AUDIO: t_name = "dec/audio"; break;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
}
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_thread_set_name(t_name);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
while (!p->request_terminate_dec_thread) {
filter: minor cosmetic naming issue Just putting some more lipstick on the pig, maybe it looks a bit nicer now.
2020-03-08 18:37:20 +00:00
mp_filter_graph_run(p->dec_root_filter);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
update_cached_values(p);
mp_dispatch_queue_process(p->dec_dispatch, INFINITY);
}
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
MP_THREAD_RETURN();
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
}
static void public_f_reset(struct mp_filter *f)
{
struct priv *p = f->priv;
assert(p->public.f == f);
if (p->queue) {
mp_async_queue_reset(p->queue);
thread_lock(p);
if (p->dec_root_filter)
mp_filter_reset(p->dec_root_filter);
mp_dispatch_interrupt(p->dec_dispatch);
thread_unlock(p);
mp_async_queue_resume(p->queue);
}
}
static void public_f_destroy(struct mp_filter *f)
{
struct priv *p = f->priv;
assert(p->public.f == f);
if (p->dec_thread_valid) {
assert(p->dec_dispatch);
thread_lock(p);
p->request_terminate_dec_thread = 1;
mp_dispatch_interrupt(p->dec_dispatch);
thread_unlock(p);
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_thread_join(p->dec_thread);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
p->dec_thread_valid = false;
}
f_decoder_wrapper: fix use of destroyed mutex After calling the main filter's destroy callback, all child filters are destroyed. But one of them still tried to access the cache_lock mutex (which is destroyed in said destroy callback). This actually caused a crash on Android with _FORTIFY_SOURCE. Fix this by destroying the child filters first.
2020-03-18 21:31:17 +00:00
mp_filter_free_children(f);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
talloc_free(p->dec_root_filter);
talloc_free(p->queue);
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_destroy(&p->cache_lock);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
}
static const struct mp_filter_info decf_filter = {
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
.name = "decode",
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
.process = decf_process,
.reset = decf_reset,
.destroy = decf_destroy,
};
static const struct mp_filter_info decode_wrapper_filter = {
.name = "decode_wrapper",
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
.priv_size = sizeof(struct priv),
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
.reset = public_f_reset,
.destroy = public_f_destroy,
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
};
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
static void wakeup_dec_thread(void *ptr)
{
struct priv *p = ptr;
mp_dispatch_interrupt(p->dec_dispatch);
}
f_decoder_wrapper: make decoder thread responsive while filling queue The mp_filter_run() invocation blocks as long as the demuxer provides packets and the queue can be filled. That means it may block quite a long time of the decoder queue size is large (since we use libavcodec in a blocking manner; it regrettably does not have an async. API). This made the main thread freeze in certain situations, because it has to wait on the decoder thread. Other than I suspected (I wrote that code, but that doesn't mean I know how the hell it works), this did not freeze during seeking: seek resets flushed the queue, which also prevented the decoder thread from adding more frames to it, thus stopping decoding and responding to the main thread in time. But it does fix the issue that exiting the player waited for the decoder to finish filling the queue when stopping playback. (This happened because it called mp_decoder_wrapper_set_play_dir() before any resets. Related to the somewhat messy way play_dir is generally set. But it affects all "synchronous" decoder wrapper API calls.) This uses pretty weird mechanisms in filter.h and dispatch.h. The resulting durr hurr interactions are probably hard to follow, and this whole thing is a sin. On the other hand, this is a _very_ user visible issue, and I'm happy that it can be fixed in such an unintrusive way.
2020-03-05 20:30:21 +00:00
static void onlock_dec_thread(void *ptr)
{
struct priv *p = ptr;
mp_filter_graph_interrupt(p->dec_root_filter);
}
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
struct mp_decoder_wrapper *mp_decoder_wrapper_create(struct mp_filter *parent,
struct sh_stream *src)
{
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
struct mp_filter *public_f = mp_filter_create(parent, &decode_wrapper_filter);
if (!public_f)
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
return NULL;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
struct priv *p = public_f->priv;
p->public.f = public_f;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
mp_mutex_init(&p->cache_lock);
options: make decoder options local to decoder wrapper Instead of having f_decoder_wrapper create its own copy of the entire mpv option tree, create a struct local to that file and move all used options to there. movie_aspect is used by the "video-aspect" deprecated property code. I think it's probably better not to remove the property yet, but fortunately it's easy to work around without needing special handling for this option or so. correct_pts is used to prevent use of hr-seek in playloop.c. Ignore that, if you use --no-correct-pts you're asking for trouble anyway. This is the only behavior change.
2020-02-29 23:28:09 +00:00
p->opt_cache = m_config_cache_alloc(p, public_f->global, &dec_wrapper_conf);
p->opts = p->opt_cache->opts;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
p->header = src;
p->codec = p->header->codec;
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
p->play_dir = 1;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
mp_filter_add_pin(public_f, MP_PIN_OUT, "out");
player: get rid of mpv_global.opts This was always a legacy thing. Remove it by applying an orgy of mp_get_config_group() calls, and sometimes m_config_cache_alloc() or mp_read_option_raw(). win32 changes untested.
2018-05-21 14:25:52 +00:00
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
if (p->header->type == STREAM_VIDEO) {
f_decoder_wrapper: make log prefix less verbose
2020-09-20 19:33:40 +00:00
p->log = mp_log_new(p, parent->global->log, "!vd");
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
p->fps = src->codec->fps;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
f_decoder_wrapper: replace most public fields with setters/getters I may (optionally) move decoding to a separate thread in a future change. It's a bit attractive to move the entire decoder wrapper to there, so if the demuxer has a new packet, it doesn't have to wake up the main thread, and can directly wake up the decoder. (Although that's bullshit, since there's a queue in between, and libavcodec's multi-threaded decoding plays cross-threads ping pong with packets anyway. On the other hand, the main thread would still have to shuffle the packets around, so whatever, just seems like better design.) As preparation, there shouldn't be any mutable state exposed by the wrapper. But there's still a large number of corner-caseish crap, so just use setters/getters for them. This recorder thing will inherently not work, so it'll have to be disabled if threads are used. This is a bit painful, but probably still the right thing. Like speculatively pulling teeth.
2020-02-28 20:17:55 +00:00
MP_VERBOSE(p, "Container reported FPS: %f\n", p->fps);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
options: rename --fps to --container-fps-override This better reflects what it actually does. As a bonus, script writers won't be misled into thinking that fps displays the actual video or display fps.
2023-10-15 14:36:56 +00:00
if (p->opts->fps_override) {
p->fps = p->opts->fps_override;
MP_INFO(p, "Container FPS forced to %5.3f.\n", p->fps);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
MP_INFO(p, "Use --no-correct-pts to force FPS based timing.\n");
}
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
f_decoder_wrapper: make most queue options runtime changeable Why not.
2020-02-29 23:38:34 +00:00
p->queue_opts = p->opts->vdec_queue_opts;
audio: move to decoder wrapper Use the decoder wrapper that was introduced for video. This removes all code duplication the old audio decoder wrapper had with the video code. (The audio wrapper was copy pasted from the video one over a decade ago, and has been kept in sync ever since by the power of copy&paste. Since the original copy&paste was possibly done by someone who did not answer to the LGPL relicensing, this should also remove all doubts about whether any of this code is left, since we now completely remove any code that could possibly have been based on it.) There is some complication with spdif handling, and a minor behavior change (it will restrict the list of codecs to spdif if spdif is to be used), but there should not be any difference in practice.
2018-01-29 05:18:33 +00:00
} else if (p->header->type == STREAM_AUDIO) {
f_decoder_wrapper: make log prefix less verbose
2020-09-20 19:33:40 +00:00
p->log = mp_log_new(p, parent->global->log, "!ad");
f_decoder_wrapper: make most queue options runtime changeable Why not.
2020-02-29 23:38:34 +00:00
p->queue_opts = p->opts->adec_queue_opts;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
} else {
goto error;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
}
f_decoder_wrapper: make most queue options runtime changeable Why not.
2020-02-29 23:38:34 +00:00
if (p->queue_opts && p->queue_opts->use_queue) {
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
p->queue = mp_async_queue_create();
p->dec_dispatch = mp_dispatch_create(p);
p->dec_root_filter = mp_filter_create_root(public_f->global);
filter: minor cosmetic naming issue Just putting some more lipstick on the pig, maybe it looks a bit nicer now.
2020-03-08 18:37:20 +00:00
mp_filter_graph_set_wakeup_cb(p->dec_root_filter, wakeup_dec_thread, p);
f_decoder_wrapper: make decoder thread responsive while filling queue The mp_filter_run() invocation blocks as long as the demuxer provides packets and the queue can be filled. That means it may block quite a long time of the decoder queue size is large (since we use libavcodec in a blocking manner; it regrettably does not have an async. API). This made the main thread freeze in certain situations, because it has to wait on the decoder thread. Other than I suspected (I wrote that code, but that doesn't mean I know how the hell it works), this did not freeze during seeking: seek resets flushed the queue, which also prevented the decoder thread from adding more frames to it, thus stopping decoding and responding to the main thread in time. But it does fix the issue that exiting the player waited for the decoder to finish filling the queue when stopping playback. (This happened because it called mp_decoder_wrapper_set_play_dir() before any resets. Related to the somewhat messy way play_dir is generally set. But it affects all "synchronous" decoder wrapper API calls.) This uses pretty weird mechanisms in filter.h and dispatch.h. The resulting durr hurr interactions are probably hard to follow, and this whole thing is a sin. On the other hand, this is a _very_ user visible issue, and I'm happy that it can be fixed in such an unintrusive way.
2020-03-05 20:30:21 +00:00
mp_dispatch_set_onlock_fn(p->dec_dispatch, onlock_dec_thread, p);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
f_decoder_wrapper: enable DR and hwdec with --vd-queue-enable This was forgotten. Hardware decoding typically breaks immediately, because many hw decoding APIs require allocating all surfaces in advance (and/or libavcodec was not made flexible enough to add new surfaces later). If the queue is large enough, it will run out of surfaces, fail decoding, and fall back to software decoding. We consider this the user's fault. --hwdec-extra-frames can be used to avoid this, if you have enough GPU memory, and the needed number of frames is lower than the arbitrary mpv-set maximum limit of that option.
2020-03-05 10:33:37 +00:00
struct mp_stream_info *sinfo = mp_filter_find_stream_info(parent);
if (sinfo) {
p->dec_root_filter->stream_info = &p->stream_info;
p->stream_info = (struct mp_stream_info){
.dr_vo = sinfo->dr_vo,
.hwdec_devs = sinfo->hwdec_devs,
};
}
f_decoder_wrapper: make most queue options runtime changeable Why not.
2020-02-29 23:38:34 +00:00
update_queue_config(p);
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
}
p->decf = mp_filter_create(p->dec_root_filter ? p->dec_root_filter : public_f,
&decf_filter);
various: add some missing error checks
2023-11-18 22:30:42 +00:00
if (!p->decf)
goto error;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
p->decf->priv = p;
f_decoder_wrapper: use proper log prefix for all involved filters p->log has a prefix set that gives some context and distinguishes audio and video decoders. The "public" wrapper filter didn't use it, which is a regression since commit a3823ce0e03.
2020-03-05 20:20:50 +00:00
p->decf->log = public_f->log = p->log;
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
mp_filter_add_pin(p->decf, MP_PIN_OUT, "out");
struct mp_filter *demux = mp_demux_in_create(p->decf, p->header);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
if (!demux)
goto error;
p->demux = demux->pins[0];
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
decf_reset(p->decf);
if (p->queue) {
struct mp_filter *f_in =
mp_async_queue_create_filter(public_f, MP_PIN_OUT, p->queue);
struct mp_filter *f_out =
mp_async_queue_create_filter(p->decf, MP_PIN_IN, p->queue);
mp_pin_connect(public_f->ppins[0], f_in->pins[0]);
mp_pin_connect(f_out->pins[0], p->decf->pins[0]);
p->dec_thread_valid = true;
ALL: use new mp_thread abstraction
2023-10-21 02:55:41 +00:00
if (mp_thread_create(&p->dec_thread, dec_thread, p)) {
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
p->dec_thread_valid = false;
goto error;
}
} else {
mp_pin_connect(public_f->ppins[0], p->decf->pins[0]);
}
public_f_reset(public_f);
f_decoder_wrapper: fix initialization state Some state wasn't reset when decoding was started without a seek reset before it. The code used to rely on reset_decoder() resetting this state, but since the commit referenced below, reset_decoder() does less than reset(). Fix this by explicitly calling reset() on initialization. Fixes: "f_decoder_wrapper: avoid full reset on timeline switch etc."
2019-06-01 12:12:26 +00:00
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
return &p->public;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
error:
player: add optional separate video decoding thread See manpage additions. This has been a topic in MPlayer/mplayer2/mpv since forever. But since libavcodec multi-threaded decoding was added, I've always considered this pointless. libavcodec requires you to "preload" it with packets, and then you can pretty much avoid blocking on it, if decoding is fast enough. But in some cases, a decoupled decoder thread _might_ help. Users have for example come up with cases where decoding video in a separate process and piping it as raw video to mpv helped. (Or my memory is false, and it was about vapoursynth filtering, who knows.) So let's just see whether this helps with anything. Note that this would have been _much_ easier if libavcodec had an asynchronous (or rather, non-blocking) API. It could probably have easily gained that with a small change to its multi-threading code and a small extension to its API, but I guess not. Unfortunately, this uglifies f_decoder_wrapper quite a lot. Part of this is due to annoying corner cases like legacy frame dropping and hardware decoder state. These could probably be prettified later on. There is also a change in playloop.c: this is because there is a need to coordinate playback resets between demuxer thread, decoder thread, and playback logic. I think this SEEK_BLOCK idea worked out reasonably well. There are still a number of problems. For example, if the demuxer cache is full, the decoder thread will simply block hard until the output queue is full, which interferes with seeking. Could also be improved later. Hardware decoding will probably die in a fire, because it will run out of surfaces quickly. We could reduce the queue to size 1... maybe later. We could update the queue options at runtime easily, but currently I'm not going to bother. I could only have put the lavc wrapper itself on a separate thread. But there is some annoying interaction with EDL and backward playback shit, and also you would have had to loop demuxer packets through the playloop, so this sounded less annoying. The food my mother made for us today was delicious. Because audio uses the same code, also for audio (even if completely pointless). Fixes: #6926
2020-02-29 20:40:52 +00:00
talloc_free(public_f);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
return NULL;
}
ad_lavc, vd_lavc: return full error codes to shared decoder loop ad_lavc and vd_lavc use the lavc_process() helper to translate the FFmpeg push/pull API to the internal filter API (which completely mismatch, even though I'm responsible for both, just fucking kill me). This interface was "slightly" too tight. It returned only a bool indicating "progress", which was not enough to handle some cases (see following commit). While we're at it, move all state into a struct. This is only a single bool, but we get the chance to add more if needed. This fixes mpv falling asleep if decoding returns an error during draining. If decoding fails when we already sent EOF, the state machine stopped making progress. This left mpv just sitting around and doing nothing. A test case can be created with: echo $RANDOM >> image.png This makes libavformat read a proper packet plus a packet of garbage. libavcodec will decode a frame, and then return an error code. The lavc_process() wrapper could not deal with this, because there was no way to differentiate between "retry" and "send new packet". Normally, it would send a new packet, so decoding would make progress anyway. If there was "progress", we couldn't just retry, because it'd retry forever. This is made worse by the fact that it tries to decode at least two frames before starting display, meaning it will "sit around and do nothing" before the picture is displayed. Change it so that on error return, "receiving" a frame is retried. This will make it return the EOF, so everything works properly. This is a high-risk change, because all these funny bullshit exceptions for hardware decoding are in the way, and I didn't retest them. For example, if hardware decoding is enabled, it keeps a list of packets, that are fed into the decoder again if hardware decoding fails, and a software fallback is performed. Another case of horrifying accidental complexity. Fixes: #6618
2019-10-24 16:40:46 +00:00
void lavc_process(struct mp_filter *f, struct lavc_state *state,
int (*send)(struct mp_filter *f, struct demux_packet *pkt),
int (*receive)(struct mp_filter *f, struct mp_frame *res))
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
{
if (!mp_pin_in_needs_data(f->ppins[1]))
return;
struct mp_frame frame = {0};
ad_lavc, vd_lavc: return full error codes to shared decoder loop ad_lavc and vd_lavc use the lavc_process() helper to translate the FFmpeg push/pull API to the internal filter API (which completely mismatch, even though I'm responsible for both, just fucking kill me). This interface was "slightly" too tight. It returned only a bool indicating "progress", which was not enough to handle some cases (see following commit). While we're at it, move all state into a struct. This is only a single bool, but we get the chance to add more if needed. This fixes mpv falling asleep if decoding returns an error during draining. If decoding fails when we already sent EOF, the state machine stopped making progress. This left mpv just sitting around and doing nothing. A test case can be created with: echo $RANDOM >> image.png This makes libavformat read a proper packet plus a packet of garbage. libavcodec will decode a frame, and then return an error code. The lavc_process() wrapper could not deal with this, because there was no way to differentiate between "retry" and "send new packet". Normally, it would send a new packet, so decoding would make progress anyway. If there was "progress", we couldn't just retry, because it'd retry forever. This is made worse by the fact that it tries to decode at least two frames before starting display, meaning it will "sit around and do nothing" before the picture is displayed. Change it so that on error return, "receiving" a frame is retried. This will make it return the EOF, so everything works properly. This is a high-risk change, because all these funny bullshit exceptions for hardware decoding are in the way, and I didn't retest them. For example, if hardware decoding is enabled, it keeps a list of packets, that are fed into the decoder again if hardware decoding fails, and a software fallback is performed. Another case of horrifying accidental complexity. Fixes: #6618
2019-10-24 16:40:46 +00:00
int ret_recv = receive(f, &frame);
f_decoder_wapper: trust frame return over error code lavc_process() calls the receive/send callbacks, which mirror libavcodec's send/receive API. The receive API in particular can return both a status code and a frame. Normally, libavcodec is pretty explicit that it can't return both a frame and an error. But the receive callback does more stuff in addition (vd_lavc does hardware decoding fallbacks etc.). The previous commit shows an instance where this happened, and where it leaked a frame in this case. For robustness, check whether the frame is set first, i.e. trust it over the status code. Before this, it checked for an EOF status first. Hopefully is of no consequence otherwise. I made this change after testing everything (can someone implement a test suite which tests this exhaustively).
2019-10-25 19:50:10 +00:00
if (frame.type) {
state->eof_returned = false;
mp_pin_in_write(f->ppins[1], frame);
} else if (ret_recv == AVERROR_EOF) {
ad_lavc, vd_lavc: return full error codes to shared decoder loop ad_lavc and vd_lavc use the lavc_process() helper to translate the FFmpeg push/pull API to the internal filter API (which completely mismatch, even though I'm responsible for both, just fucking kill me). This interface was "slightly" too tight. It returned only a bool indicating "progress", which was not enough to handle some cases (see following commit). While we're at it, move all state into a struct. This is only a single bool, but we get the chance to add more if needed. This fixes mpv falling asleep if decoding returns an error during draining. If decoding fails when we already sent EOF, the state machine stopped making progress. This left mpv just sitting around and doing nothing. A test case can be created with: echo $RANDOM >> image.png This makes libavformat read a proper packet plus a packet of garbage. libavcodec will decode a frame, and then return an error code. The lavc_process() wrapper could not deal with this, because there was no way to differentiate between "retry" and "send new packet". Normally, it would send a new packet, so decoding would make progress anyway. If there was "progress", we couldn't just retry, because it'd retry forever. This is made worse by the fact that it tries to decode at least two frames before starting display, meaning it will "sit around and do nothing" before the picture is displayed. Change it so that on error return, "receiving" a frame is retried. This will make it return the EOF, so everything works properly. This is a high-risk change, because all these funny bullshit exceptions for hardware decoding are in the way, and I didn't retest them. For example, if hardware decoding is enabled, it keeps a list of packets, that are fed into the decoder again if hardware decoding fails, and a software fallback is performed. Another case of horrifying accidental complexity. Fixes: #6618
2019-10-24 16:40:46 +00:00
if (!state->eof_returned)
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
mp_pin_in_write(f->ppins[1], MP_EOF_FRAME);
ad_lavc, vd_lavc: return full error codes to shared decoder loop ad_lavc and vd_lavc use the lavc_process() helper to translate the FFmpeg push/pull API to the internal filter API (which completely mismatch, even though I'm responsible for both, just fucking kill me). This interface was "slightly" too tight. It returned only a bool indicating "progress", which was not enough to handle some cases (see following commit). While we're at it, move all state into a struct. This is only a single bool, but we get the chance to add more if needed. This fixes mpv falling asleep if decoding returns an error during draining. If decoding fails when we already sent EOF, the state machine stopped making progress. This left mpv just sitting around and doing nothing. A test case can be created with: echo $RANDOM >> image.png This makes libavformat read a proper packet plus a packet of garbage. libavcodec will decode a frame, and then return an error code. The lavc_process() wrapper could not deal with this, because there was no way to differentiate between "retry" and "send new packet". Normally, it would send a new packet, so decoding would make progress anyway. If there was "progress", we couldn't just retry, because it'd retry forever. This is made worse by the fact that it tries to decode at least two frames before starting display, meaning it will "sit around and do nothing" before the picture is displayed. Change it so that on error return, "receiving" a frame is retried. This will make it return the EOF, so everything works properly. This is a high-risk change, because all these funny bullshit exceptions for hardware decoding are in the way, and I didn't retest them. For example, if hardware decoding is enabled, it keeps a list of packets, that are fed into the decoder again if hardware decoding fails, and a software fallback is performed. Another case of horrifying accidental complexity. Fixes: #6618
2019-10-24 16:40:46 +00:00
state->eof_returned = true;
f_decoder_wrapper: pass through EOF after EOF It's relevant in some obscure corner cases (EDL file that has a segment without audio). Didn't test what's actually going on (is ad_lavc.c behaving wrong? is libavcodec behaving wrong or in an unexpected way? is lavc_process wrong?) and just patched it over with some bullshit, so the fix might be too complicated, and could be reworked at some later point. This sure is a real data flow fuckmess.
2020-08-27 16:40:12 +00:00
state->packets_sent = false;
ad_lavc, vd_lavc: return full error codes to shared decoder loop ad_lavc and vd_lavc use the lavc_process() helper to translate the FFmpeg push/pull API to the internal filter API (which completely mismatch, even though I'm responsible for both, just fucking kill me). This interface was "slightly" too tight. It returned only a bool indicating "progress", which was not enough to handle some cases (see following commit). While we're at it, move all state into a struct. This is only a single bool, but we get the chance to add more if needed. This fixes mpv falling asleep if decoding returns an error during draining. If decoding fails when we already sent EOF, the state machine stopped making progress. This left mpv just sitting around and doing nothing. A test case can be created with: echo $RANDOM >> image.png This makes libavformat read a proper packet plus a packet of garbage. libavcodec will decode a frame, and then return an error code. The lavc_process() wrapper could not deal with this, because there was no way to differentiate between "retry" and "send new packet". Normally, it would send a new packet, so decoding would make progress anyway. If there was "progress", we couldn't just retry, because it'd retry forever. This is made worse by the fact that it tries to decode at least two frames before starting display, meaning it will "sit around and do nothing" before the picture is displayed. Change it so that on error return, "receiving" a frame is retried. This will make it return the EOF, so everything works properly. This is a high-risk change, because all these funny bullshit exceptions for hardware decoding are in the way, and I didn't retest them. For example, if hardware decoding is enabled, it keeps a list of packets, that are fed into the decoder again if hardware decoding fails, and a software fallback is performed. Another case of horrifying accidental complexity. Fixes: #6618
2019-10-24 16:40:46 +00:00
} else if (ret_recv == AVERROR(EAGAIN)) {
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
// Need to feed a packet.
frame = mp_pin_out_read(f->ppins[0]);
struct demux_packet *pkt = NULL;
if (frame.type == MP_FRAME_PACKET) {
pkt = frame.data;
} else if (frame.type != MP_FRAME_EOF) {
if (frame.type) {
MP_ERR(f, "unexpected frame type\n");
mp_frame_unref(&frame);
mp_filter_internal_mark_failed(f);
}
return;
f_decoder_wrapper: pass through EOF after EOF It's relevant in some obscure corner cases (EDL file that has a segment without audio). Didn't test what's actually going on (is ad_lavc.c behaving wrong? is libavcodec behaving wrong or in an unexpected way? is lavc_process wrong?) and just patched it over with some bullshit, so the fix might be too complicated, and could be reworked at some later point. This sure is a real data flow fuckmess.
2020-08-27 16:40:12 +00:00
} else if (!state->packets_sent) {
// EOF only; just return it, without requiring send/receive to
// pass it through properly.
mp_pin_in_write(f->ppins[1], MP_EOF_FRAME);
return;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
}
ad_lavc, vd_lavc: return full error codes to shared decoder loop ad_lavc and vd_lavc use the lavc_process() helper to translate the FFmpeg push/pull API to the internal filter API (which completely mismatch, even though I'm responsible for both, just fucking kill me). This interface was "slightly" too tight. It returned only a bool indicating "progress", which was not enough to handle some cases (see following commit). While we're at it, move all state into a struct. This is only a single bool, but we get the chance to add more if needed. This fixes mpv falling asleep if decoding returns an error during draining. If decoding fails when we already sent EOF, the state machine stopped making progress. This left mpv just sitting around and doing nothing. A test case can be created with: echo $RANDOM >> image.png This makes libavformat read a proper packet plus a packet of garbage. libavcodec will decode a frame, and then return an error code. The lavc_process() wrapper could not deal with this, because there was no way to differentiate between "retry" and "send new packet". Normally, it would send a new packet, so decoding would make progress anyway. If there was "progress", we couldn't just retry, because it'd retry forever. This is made worse by the fact that it tries to decode at least two frames before starting display, meaning it will "sit around and do nothing" before the picture is displayed. Change it so that on error return, "receiving" a frame is retried. This will make it return the EOF, so everything works properly. This is a high-risk change, because all these funny bullshit exceptions for hardware decoding are in the way, and I didn't retest them. For example, if hardware decoding is enabled, it keeps a list of packets, that are fed into the decoder again if hardware decoding fails, and a software fallback is performed. Another case of horrifying accidental complexity. Fixes: #6618
2019-10-24 16:40:46 +00:00
int ret_send = send(f, pkt);
if (ret_send == AVERROR(EAGAIN)) {
f_decoder_wrapper: retry decoding if libavcodec returns invalid state At least the libavcodec MediaCodec wrapper sometimes seems to break the libavcodec API, and does the following sequence: send_packet() -> EAGAIN receive_frame() -> EAGAIN send_packet() -> OK The libavcodec API never allows returning EAGAIN from both functions, so we discarded packets in this case. Change it to retrying decoding, for the sake of MediaCodec. Note that it could also happen due to internal bugs in the vd_lavc.c hw fallback code, but if there are any remaining, they should be fixed properly instead. Requested.
2018-03-21 13:55:37 +00:00
// Should never happen, but can happen with broken decoders.
MP_WARN(f, "could not consume packet\n");
mp_pin_out_unread(f->ppins[0], frame);
mp_filter_wakeup(f);
return;
}
f_decoder_wrapper: pass through EOF after EOF It's relevant in some obscure corner cases (EDL file that has a segment without audio). Didn't test what's actually going on (is ad_lavc.c behaving wrong? is libavcodec behaving wrong or in an unexpected way? is lavc_process wrong?) and just patched it over with some bullshit, so the fix might be too complicated, and could be reworked at some later point. This sure is a real data flow fuckmess.
2020-08-27 16:40:12 +00:00
state->packets_sent = true;
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
talloc_free(pkt);
mp_filter_internal_mark_progress(f);
ad_lavc, vd_lavc: return full error codes to shared decoder loop ad_lavc and vd_lavc use the lavc_process() helper to translate the FFmpeg push/pull API to the internal filter API (which completely mismatch, even though I'm responsible for both, just fucking kill me). This interface was "slightly" too tight. It returned only a bool indicating "progress", which was not enough to handle some cases (see following commit). While we're at it, move all state into a struct. This is only a single bool, but we get the chance to add more if needed. This fixes mpv falling asleep if decoding returns an error during draining. If decoding fails when we already sent EOF, the state machine stopped making progress. This left mpv just sitting around and doing nothing. A test case can be created with: echo $RANDOM >> image.png This makes libavformat read a proper packet plus a packet of garbage. libavcodec will decode a frame, and then return an error code. The lavc_process() wrapper could not deal with this, because there was no way to differentiate between "retry" and "send new packet". Normally, it would send a new packet, so decoding would make progress anyway. If there was "progress", we couldn't just retry, because it'd retry forever. This is made worse by the fact that it tries to decode at least two frames before starting display, meaning it will "sit around and do nothing" before the picture is displayed. Change it so that on error return, "receiving" a frame is retried. This will make it return the EOF, so everything works properly. This is a high-risk change, because all these funny bullshit exceptions for hardware decoding are in the way, and I didn't retest them. For example, if hardware decoding is enabled, it keeps a list of packets, that are fed into the decoder again if hardware decoding fails, and a software fallback is performed. Another case of horrifying accidental complexity. Fixes: #6618
2019-10-24 16:40:46 +00:00
} else {
vd_lavc: simplify fallback handling for full stream hw decoder Shovel the code around to make the data flow slightly simpler (?). At least there's only one send_packet function now. The old code had the problem that send_packet() could be called even if there were queued packets; due to sending the queued packets in the receive_frame function, this should not happen anymore (the code checking for this case in send_packet should normally never be called). Untested with actual full stream hw decoders (none available here); I created a test case by making hwaccel decoding fail.
2019-11-02 21:37:14 +00:00
// Decoding error, or hwdec fallback recovery. Just try again.
ad_lavc, vd_lavc: return full error codes to shared decoder loop ad_lavc and vd_lavc use the lavc_process() helper to translate the FFmpeg push/pull API to the internal filter API (which completely mismatch, even though I'm responsible for both, just fucking kill me). This interface was "slightly" too tight. It returned only a bool indicating "progress", which was not enough to handle some cases (see following commit). While we're at it, move all state into a struct. This is only a single bool, but we get the chance to add more if needed. This fixes mpv falling asleep if decoding returns an error during draining. If decoding fails when we already sent EOF, the state machine stopped making progress. This left mpv just sitting around and doing nothing. A test case can be created with: echo $RANDOM >> image.png This makes libavformat read a proper packet plus a packet of garbage. libavcodec will decode a frame, and then return an error code. The lavc_process() wrapper could not deal with this, because there was no way to differentiate between "retry" and "send new packet". Normally, it would send a new packet, so decoding would make progress anyway. If there was "progress", we couldn't just retry, because it'd retry forever. This is made worse by the fact that it tries to decode at least two frames before starting display, meaning it will "sit around and do nothing" before the picture is displayed. Change it so that on error return, "receiving" a frame is retried. This will make it return the EOF, so everything works properly. This is a high-risk change, because all these funny bullshit exceptions for hardware decoding are in the way, and I didn't retest them. For example, if hardware decoding is enabled, it keeps a list of packets, that are fed into the decoder again if hardware decoding fails, and a software fallback is performed. Another case of horrifying accidental complexity. Fixes: #6618
2019-10-24 16:40:46 +00:00
mp_filter_internal_mark_progress(f);
video: make decoder wrapper a filter Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes a source filter. vd.h mostly disappears, because mp_filter takes care of the dataflow, but its remains are in struct mp_decoder_fns. One goal is to simplify dataflow by letting the filter framework handle it (or more accurately, using its conventions). One result is that the decode calls disappear from video.c, because we simply connect the decoder wrapper and the filter chain with mp_pin_connect(). Another goal is to eventually remove the code duplication between the audio and video paths for this. This commit prepares for this by trying to make f_decoder_wrapper.c extensible, so it can be used for audio as well later. Decoder framedropping changes a bit. It doesn't seem to be worse than before, and it's an obscure feature, so I'm content with its new state. Some special code that was apparently meant to avoid dropping too many frames in a row is removed, though. I'm not sure how the source code tree should be organized. For one, video/decode/vd_lavc.c is the only file in its directory, which is a bit annoying.
2018-01-28 09:08:45 +00:00
}
}