video: make decoder wrapper a filter
Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes
a source filter. vd.h mostly disappears, because mp_filter takes care of
the dataflow, but its remains are in struct mp_decoder_fns.
One goal is to simplify dataflow by letting the filter framework handle
it (or more accurately, using its conventions). One result is that the
decode calls disappear from video.c, because we simply connect the
decoder wrapper and the filter chain with mp_pin_connect().
Another goal is to eventually remove the code duplication between the
audio and video paths for this. This commit prepares for this by trying
to make f_decoder_wrapper.c extensible, so it can be used for audio as
well later.
Decoder framedropping changes a bit. It doesn't seem to be worse than
before, and it's an obscure feature, so I'm content with its new state.
Some special code that was apparently meant to avoid dropping too many
frames in a row is removed, though.
I'm not sure how the source code tree should be organized. For one,
video/decode/vd_lavc.c is the only file in its directory, which is a bit
annoying.
2018-01-28 09:08:45 +00:00
|
|
|
/*
|
|
|
|
* This file is part of mpv.
|
|
|
|
*
|
|
|
|
* mpv is free software; you can redistribute it and/or
|
|
|
|
* modify it under the terms of the GNU Lesser General Public
|
|
|
|
* License as published by the Free Software Foundation; either
|
|
|
|
* version 2.1 of the License, or (at your option) any later version.
|
|
|
|
*
|
|
|
|
* mpv is distributed in the hope that it will be useful,
|
|
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
* GNU Lesser General Public License for more details.
|
|
|
|
*
|
|
|
|
* You should have received a copy of the GNU Lesser General Public
|
|
|
|
* License along with mpv. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
*/
|
|
|
|
|
|
|
|
#pragma once
|
|
|
|
|
|
|
|
#include <stdbool.h>
|
|
|
|
|
|
|
|
#include "filter.h"
|
|
|
|
|
|
|
|
struct sh_stream;
|
|
|
|
struct mp_codec_params;
|
|
|
|
struct mp_image_params;
|
|
|
|
struct mp_decoder_list;
|
|
|
|
struct demux_packet;
|
|
|
|
|
|
|
|
// (free with talloc_free(mp_decoder_wrapper.f)
|
|
|
|
struct mp_decoder_wrapper {
|
|
|
|
// Filter with no input and 1 output, which returns the decoded data.
|
|
|
|
struct mp_filter *f;
|
|
|
|
|
|
|
|
// For informational purposes.
|
|
|
|
char *decoder_desc;
|
|
|
|
|
|
|
|
// Can be set by user.
|
|
|
|
struct mp_recorder_sink *recorder_sink;
|
Implement backwards playback
See manpage additions. This is a huge hack. You can bet there are shit
tons of bugs. It's literally forcing square pegs into round holes.
Hopefully, the manpage wall of text makes it clear enough that the whole
shit can easily crash and burn. (Although it shouldn't literally crash.
That would be a bug. It possibly _could_ start a fire by entering some
sort of endless loop, not a literal one, just something where it tries
to do work without making progress.)
(Some obvious bugs I simply ignored for this initial version, but
there's a number of potential bugs I can't even imagine. Normal playback
should remain completely unaffected, though.)
How this works is also described in the manpage. Basically, we demux in
reverse, then we decode in reverse, then we render in reverse.
The decoding part is the simplest: just reorder the decoder output. This
weirdly integrates with the timeline/ordered chapter code, which also
has special requirements on feeding the packets to the decoder in a
non-straightforward way (it doesn't conflict, although a bugmessmass
breaks correct slicing of segments, so EDL/ordered chapter playback is
broken in backward direction).
Backward demuxing is pretty involved. In theory, it could be much
easier: simply iterating the usual demuxer output backward. But this
just doesn't fit into our code, so there's a cthulhu nightmare of shit.
To be specific, each stream (audio, video) is reversed separately. At
least this means we can do backward playback within cached content (for
example, you could play backwards in a live stream; on that note, it
disables prefetching, which would lead to losing new live video, but
this could be avoided).
The fuckmess also meant that I didn't bother trying to support
subtitles. Subtitles are a problem because they're "sparse" streams.
They need to be "passively" demuxed: you don't try to read a subtitle
packet, you demux audio and video, and then look whether there was a
subtitle packet. This means to get subtitles for a time range, you need
to know that you demuxed video and audio over this range, which becomes
pretty messy when you demux audio and video backwards separately.
Backward display is the most weird (and potentially buggy) part. To
avoid that we need to touch a LOT of timing code, we negate all
timestamps. The basic idea is that due to the navigation, all
comparisons and subtractions of timestamps keep working, and you don't
need to touch every single of them to "reverse" them.
E.g.:
bool before = pts_a < pts_b;
would need to be:
bool before = forward
? pts_a < pts_b
: pts_a > pts_b;
or:
bool before = pts_a * dir < pts_b * dir;
or if you, as it's implemented now, just do this after decoding:
pts_a *= dir;
pts_b *= dir;
and then in the normal timing/renderer code:
bool before = pts_a < pts_b;
Consequently, we don't need many changes in the latter code. But some
assumptions inhererently true for forward playback may have been broken
anyway. What is mainly needed is fixing places where values are passed
between positive and negative "domains". For example, seeking and
timestamp user display always uses positive timestamps. The main mess is
that it's not obvious which domain a given variable should or does use.
Well, in my tests with a single file, it suddenly started to work when I
did this. I'm honestly surprised that it did, and that I didn't have to
change a single line in the timing code past decoder (just something
minor to make external/cached text subtitles display). I committed it
immediately while avoiding thinking about it. But there really likely
are subtle problems of all sorts.
As far as I'm aware, gstreamer also supports backward playback. When I
looked at this years ago, I couldn't find a way to actually try this,
and I didn't revisit it now. Back then I also read talk slides from the
person who implemented it, and I'm not sure if and which ideas I might
have taken from it. It's possible that the timestamp reversal is
inspired by it, but I didn't check. (I think it claimed that it could
avoid large changes by changing a sign?)
VapourSynth has some sort of reverse function, which provides a backward
view on a video. The function itself is trivial to implement, as
VapourSynth aims to provide random access to video by frame numbers (so
you just request decreasing frame numbers). From what I remember, it
wasn't exactly fluid, but it worked. It's implemented by creating an
index, and seeking to the target on demand, and a bunch of caching. mpv
could use it, but it would either require using VapourSynth as demuxer
and decoder for everything, or replacing the current file every time
something is supposed to be played backwards.
FFmpeg's libavfilter has reversal filters for audio and video. These
require buffering the entire media data of the file, and don't really
fit into mpv's architecture. It could be used by playing a libavfilter
graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
|
|
|
int play_dir;
|
video: make decoder wrapper a filter
Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes
a source filter. vd.h mostly disappears, because mp_filter takes care of
the dataflow, but its remains are in struct mp_decoder_fns.
One goal is to simplify dataflow by letting the filter framework handle
it (or more accurately, using its conventions). One result is that the
decode calls disappear from video.c, because we simply connect the
decoder wrapper and the filter chain with mp_pin_connect().
Another goal is to eventually remove the code duplication between the
audio and video paths for this. This commit prepares for this by trying
to make f_decoder_wrapper.c extensible, so it can be used for audio as
well later.
Decoder framedropping changes a bit. It doesn't seem to be worse than
before, and it's an obscure feature, so I'm content with its new state.
Some special code that was apparently meant to avoid dropping too many
frames in a row is removed, though.
I'm not sure how the source code tree should be organized. For one,
video/decode/vd_lavc.c is the only file in its directory, which is a bit
annoying.
2018-01-28 09:08:45 +00:00
|
|
|
|
|
|
|
// --- for STREAM_VIDEO
|
|
|
|
|
|
|
|
// FPS from demuxer or from user override
|
|
|
|
float fps;
|
|
|
|
|
|
|
|
// Framedrop control for playback (not used for hr seek etc.)
|
|
|
|
int attempt_framedrops; // try dropping this many frames
|
|
|
|
int dropped_frames; // total frames _probably_ dropped
|
2018-01-29 05:18:33 +00:00
|
|
|
|
|
|
|
// --- for STREAM_AUDIO
|
|
|
|
|
|
|
|
// Prefer spdif wrapper over real decoders.
|
|
|
|
bool try_spdif;
|
2018-02-12 17:00:23 +00:00
|
|
|
|
|
|
|
// A pts reset was observed (audio only, heuristic).
|
|
|
|
bool pts_reset;
|
video: make decoder wrapper a filter
Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes
a source filter. vd.h mostly disappears, because mp_filter takes care of
the dataflow, but its remains are in struct mp_decoder_fns.
One goal is to simplify dataflow by letting the filter framework handle
it (or more accurately, using its conventions). One result is that the
decode calls disappear from video.c, because we simply connect the
decoder wrapper and the filter chain with mp_pin_connect().
Another goal is to eventually remove the code duplication between the
audio and video paths for this. This commit prepares for this by trying
to make f_decoder_wrapper.c extensible, so it can be used for audio as
well later.
Decoder framedropping changes a bit. It doesn't seem to be worse than
before, and it's an obscure feature, so I'm content with its new state.
Some special code that was apparently meant to avoid dropping too many
frames in a row is removed, though.
I'm not sure how the source code tree should be organized. For one,
video/decode/vd_lavc.c is the only file in its directory, which is a bit
annoying.
2018-01-28 09:08:45 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
// Create the decoder wrapper for the given stream, plus underlying decoder.
|
|
|
|
// The src stream must be selected, and remain valid and selected until the
|
|
|
|
// wrapper is destroyed.
|
|
|
|
struct mp_decoder_wrapper *mp_decoder_wrapper_create(struct mp_filter *parent,
|
|
|
|
struct sh_stream *src);
|
|
|
|
|
|
|
|
struct mp_decoder_list *video_decoder_list(void);
|
2018-01-29 05:18:33 +00:00
|
|
|
struct mp_decoder_list *audio_decoder_list(void);
|
video: make decoder wrapper a filter
Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes
a source filter. vd.h mostly disappears, because mp_filter takes care of
the dataflow, but its remains are in struct mp_decoder_fns.
One goal is to simplify dataflow by letting the filter framework handle
it (or more accurately, using its conventions). One result is that the
decode calls disappear from video.c, because we simply connect the
decoder wrapper and the filter chain with mp_pin_connect().
Another goal is to eventually remove the code duplication between the
audio and video paths for this. This commit prepares for this by trying
to make f_decoder_wrapper.c extensible, so it can be used for audio as
well later.
Decoder framedropping changes a bit. It doesn't seem to be worse than
before, and it's an obscure feature, so I'm content with its new state.
Some special code that was apparently meant to avoid dropping too many
frames in a row is removed, though.
I'm not sure how the source code tree should be organized. For one,
video/decode/vd_lavc.c is the only file in its directory, which is a bit
annoying.
2018-01-28 09:08:45 +00:00
|
|
|
|
|
|
|
// For precise seeking: if possible, try to drop frames up until the given PTS.
|
|
|
|
// This is automatically unset if the target is reached, or on reset.
|
|
|
|
void mp_decoder_wrapper_set_start_pts(struct mp_decoder_wrapper *d, double pts);
|
|
|
|
|
|
|
|
enum dec_ctrl {
|
|
|
|
VDCTRL_FORCE_HWDEC_FALLBACK, // force software decoding fallback
|
|
|
|
VDCTRL_GET_HWDEC,
|
|
|
|
VDCTRL_REINIT,
|
|
|
|
VDCTRL_GET_BFRAMES,
|
|
|
|
// framedrop mode: 0=none, 1=standard, 2=hrseek
|
|
|
|
VDCTRL_SET_FRAMEDROP,
|
|
|
|
};
|
|
|
|
|
|
|
|
int mp_decoder_wrapper_control(struct mp_decoder_wrapper *d,
|
|
|
|
enum dec_ctrl cmd, void *arg);
|
|
|
|
|
|
|
|
// Force it to reevaluate output parameters (for overrides like aspect).
|
|
|
|
void mp_decoder_wrapper_reset_params(struct mp_decoder_wrapper *d);
|
|
|
|
|
|
|
|
void mp_decoder_wrapper_get_video_dec_params(struct mp_decoder_wrapper *d,
|
|
|
|
struct mp_image_params *p);
|
|
|
|
|
|
|
|
bool mp_decoder_wrapper_reinit(struct mp_decoder_wrapper *d);
|
|
|
|
|
|
|
|
struct mp_decoder {
|
|
|
|
// Bidirectional filter; takes MP_FRAME_PACKET for input.
|
|
|
|
struct mp_filter *f;
|
|
|
|
|
|
|
|
// Can be set by decoder impl. on init for "special" functionality.
|
|
|
|
int (*control)(struct mp_filter *f, enum dec_ctrl cmd, void *arg);
|
|
|
|
};
|
|
|
|
|
|
|
|
struct mp_decoder_fns {
|
|
|
|
struct mp_decoder *(*create)(struct mp_filter *parent,
|
|
|
|
struct mp_codec_params *codec,
|
|
|
|
const char *decoder);
|
|
|
|
void (*add_decoders)(struct mp_decoder_list *list);
|
|
|
|
};
|
|
|
|
|
|
|
|
extern const struct mp_decoder_fns vd_lavc;
|
2018-01-29 05:18:33 +00:00
|
|
|
extern const struct mp_decoder_fns ad_lavc;
|
|
|
|
extern const struct mp_decoder_fns ad_spdif;
|
video: make decoder wrapper a filter
Move dec_video.c to filters/f_decoder_wrapper.c. It essentially becomes
a source filter. vd.h mostly disappears, because mp_filter takes care of
the dataflow, but its remains are in struct mp_decoder_fns.
One goal is to simplify dataflow by letting the filter framework handle
it (or more accurately, using its conventions). One result is that the
decode calls disappear from video.c, because we simply connect the
decoder wrapper and the filter chain with mp_pin_connect().
Another goal is to eventually remove the code duplication between the
audio and video paths for this. This commit prepares for this by trying
to make f_decoder_wrapper.c extensible, so it can be used for audio as
well later.
Decoder framedropping changes a bit. It doesn't seem to be worse than
before, and it's an obscure feature, so I'm content with its new state.
Some special code that was apparently meant to avoid dropping too many
frames in a row is removed, though.
I'm not sure how the source code tree should be organized. For one,
video/decode/vd_lavc.c is the only file in its directory, which is a bit
annoying.
2018-01-28 09:08:45 +00:00
|
|
|
|
ad_lavc, vd_lavc: return full error codes to shared decoder loop
ad_lavc and vd_lavc use the lavc_process() helper to translate the
FFmpeg push/pull API to the internal filter API (which completely
mismatch, even though I'm responsible for both, just fucking kill me).
This interface was "slightly" too tight. It returned only a bool
indicating "progress", which was not enough to handle some cases (see
following commit).
While we're at it, move all state into a struct. This is only a single
bool, but we get the chance to add more if needed.
This fixes mpv falling asleep if decoding returns an error during
draining. If decoding fails when we already sent EOF, the state machine
stopped making progress. This left mpv just sitting around and doing
nothing.
A test case can be created with: echo $RANDOM >> image.png
This makes libavformat read a proper packet plus a packet of garbage.
libavcodec will decode a frame, and then return an error code. The
lavc_process() wrapper could not deal with this, because there was no
way to differentiate between "retry" and "send new packet". Normally, it
would send a new packet, so decoding would make progress anyway. If
there was "progress", we couldn't just retry, because it'd retry
forever.
This is made worse by the fact that it tries to decode at least two
frames before starting display, meaning it will "sit around and do
nothing" before the picture is displayed.
Change it so that on error return, "receiving" a frame is retried. This
will make it return the EOF, so everything works properly.
This is a high-risk change, because all these funny bullshit exceptions
for hardware decoding are in the way, and I didn't retest them. For
example, if hardware decoding is enabled, it keeps a list of packets,
that are fed into the decoder again if hardware decoding fails, and a
software fallback is performed. Another case of horrifying accidental
complexity.
Fixes: #6618
2019-10-24 16:40:46 +00:00
|
|
|
// Convenience wrapper for lavc based decoders. Treat lavc_state as private;
|
|
|
|
// init to all-0 on init and resets.
|
|
|
|
struct lavc_state {
|
|
|
|
bool eof_returned;
|
|
|
|
};
|
|
|
|
void lavc_process(struct mp_filter *f, struct lavc_state *state,
|
|
|
|
int (*send)(struct mp_filter *f, struct demux_packet *pkt),
|
|
|
|
int (*receive)(struct mp_filter *f, struct mp_frame *res));
|
2018-01-29 05:18:33 +00:00
|
|
|
|
|
|
|
// ad_spdif.c
|
|
|
|
struct mp_decoder_list *select_spdif_codec(const char *codec, const char *pref);
|