1
0
mirror of https://github.com/mpv-player/mpv synced 2024-12-25 00:02:13 +00:00
mpv/audio/aframe.h
wm4 b9d351f02a Implement backwards playback
See manpage additions. This is a huge hack. You can bet there are shit
tons of bugs. It's literally forcing square pegs into round holes.
Hopefully, the manpage wall of text makes it clear enough that the whole
shit can easily crash and burn. (Although it shouldn't literally crash.
That would be a bug. It possibly _could_ start a fire by entering some
sort of endless loop, not a literal one, just something where it tries
to do work without making progress.)

(Some obvious bugs I simply ignored for this initial version, but
there's a number of potential bugs I can't even imagine. Normal playback
should remain completely unaffected, though.)

How this works is also described in the manpage. Basically, we demux in
reverse, then we decode in reverse, then we render in reverse.

The decoding part is the simplest: just reorder the decoder output. This
weirdly integrates with the timeline/ordered chapter code, which also
has special requirements on feeding the packets to the decoder in a
non-straightforward way (it doesn't conflict, although a bugmessmass
breaks correct slicing of segments, so EDL/ordered chapter playback is
broken in backward direction).

Backward demuxing is pretty involved. In theory, it could be much
easier: simply iterating the usual demuxer output backward. But this
just doesn't fit into our code, so there's a cthulhu nightmare of shit.
To be specific, each stream (audio, video) is reversed separately. At
least this means we can do backward playback within cached content (for
example, you could play backwards in a live stream; on that note, it
disables prefetching, which would lead to losing new live video, but
this could be avoided).

The fuckmess also meant that I didn't bother trying to support
subtitles. Subtitles are a problem because they're "sparse" streams.
They need to be "passively" demuxed: you don't try to read a subtitle
packet, you demux audio and video, and then look whether there was a
subtitle packet. This means to get subtitles for a time range, you need
to know that you demuxed video and audio over this range, which becomes
pretty messy when you demux audio and video backwards separately.

Backward display is the most weird (and potentially buggy) part. To
avoid that we need to touch a LOT of timing code, we negate all
timestamps. The basic idea is that due to the navigation, all
comparisons and subtractions of timestamps keep working, and you don't
need to touch every single of them to "reverse" them.

E.g.:

    bool before = pts_a < pts_b;

would need to be:

    bool before = forward
        ? pts_a < pts_b
        : pts_a > pts_b;

or:

    bool before = pts_a * dir < pts_b * dir;

or if you, as it's implemented now, just do this after decoding:

    pts_a *= dir;
    pts_b *= dir;

and then in the normal timing/renderer code:

    bool before = pts_a < pts_b;

Consequently, we don't need many changes in the latter code. But some
assumptions inhererently true for forward playback may have been broken
anyway. What is mainly needed is fixing places where values are passed
between positive and negative "domains". For example, seeking and
timestamp user display always uses positive timestamps. The main mess is
that it's not obvious which domain a given variable should or does use.

Well, in my tests with a single file, it suddenly started to work when I
did this. I'm honestly surprised that it did, and that I didn't have to
change a single line in the timing code past decoder (just something
minor to make external/cached text subtitles display). I committed it
immediately while avoiding thinking about it. But there really likely
are subtle problems of all sorts.

As far as I'm aware, gstreamer also supports backward playback. When I
looked at this years ago, I couldn't find a way to actually try this,
and I didn't revisit it now. Back then I also read talk slides from the
person who implemented it, and I'm not sure if and which ideas I might
have taken from it. It's possible that the timestamp reversal is
inspired by it, but I didn't check. (I think it claimed that it could
avoid large changes by changing a sign?)

VapourSynth has some sort of reverse function, which provides a backward
view on a video. The function itself is trivial to implement, as
VapourSynth aims to provide random access to video by frame numbers (so
you just request decreasing frame numbers). From what I remember, it
wasn't exactly fluid, but it worked. It's implemented by creating an
index, and seeking to the target on demand, and a bunch of caching. mpv
could use it, but it would either require using VapourSynth as demuxer
and decoder for everything, or replacing the current file every time
something is supposed to be played backwards.

FFmpeg's libavfilter has reversal filters for audio and video. These
require buffering the entire media data of the file, and don't really
fit into mpv's architecture. It could be used by playing a libavfilter
graph that also demuxes, but that's like VapourSynth but worse.
2019-09-19 20:37:04 +02:00

74 lines
3.0 KiB
C

#pragma once
#include <stdint.h>
#include <stddef.h>
#include <stdbool.h>
struct mp_aframe;
struct AVFrame;
struct mp_chmap;
struct mp_aframe *mp_aframe_from_avframe(struct AVFrame *av_frame);
struct mp_aframe *mp_aframe_create(void);
struct mp_aframe *mp_aframe_new_ref(struct mp_aframe *frame);
void mp_aframe_reset(struct mp_aframe *frame);
void mp_aframe_unref_data(struct mp_aframe *frame);
struct AVFrame *mp_aframe_to_avframe(struct mp_aframe *frame);
struct AVFrame *mp_aframe_to_avframe_and_unref(struct mp_aframe *frame);
struct AVFrame *mp_aframe_get_raw_avframe(struct mp_aframe *frame);
bool mp_aframe_is_allocated(struct mp_aframe *frame);
void mp_aframe_config_copy(struct mp_aframe *dst, struct mp_aframe *src);
bool mp_aframe_config_equals(struct mp_aframe *a, struct mp_aframe *b);
bool mp_aframe_config_is_valid(struct mp_aframe *frame);
void mp_aframe_copy_attributes(struct mp_aframe *dst, struct mp_aframe *src);
uint8_t **mp_aframe_get_data_ro(struct mp_aframe *frame);
uint8_t **mp_aframe_get_data_rw(struct mp_aframe *frame);
int mp_aframe_get_format(struct mp_aframe *frame);
bool mp_aframe_get_chmap(struct mp_aframe *frame, struct mp_chmap *out);
int mp_aframe_get_channels(struct mp_aframe *frame);
int mp_aframe_get_rate(struct mp_aframe *frame);
int mp_aframe_get_size(struct mp_aframe *frame);
double mp_aframe_get_pts(struct mp_aframe *frame);
double mp_aframe_get_speed(struct mp_aframe *frame);
double mp_aframe_get_effective_rate(struct mp_aframe *frame);
bool mp_aframe_set_format(struct mp_aframe *frame, int format);
bool mp_aframe_set_chmap(struct mp_aframe *frame, struct mp_chmap *in);
bool mp_aframe_set_rate(struct mp_aframe *frame, int rate);
bool mp_aframe_set_size(struct mp_aframe *frame, int samples);
void mp_aframe_set_pts(struct mp_aframe *frame, double pts);
void mp_aframe_set_speed(struct mp_aframe *frame, double factor);
void mp_aframe_mul_speed(struct mp_aframe *frame, double factor);
int mp_aframe_get_planes(struct mp_aframe *frame);
int mp_aframe_get_total_plane_samples(struct mp_aframe *frame);
size_t mp_aframe_get_sstride(struct mp_aframe *frame);
bool mp_aframe_reverse(struct mp_aframe *frame);
int mp_aframe_approx_byte_size(struct mp_aframe *frame);
char *mp_aframe_format_str_buf(char *buf, size_t buf_size, struct mp_aframe *fmt);
#define mp_aframe_format_str(fmt) mp_aframe_format_str_buf((char[32]){0}, 32, (fmt))
void mp_aframe_skip_samples(struct mp_aframe *f, int samples);
double mp_aframe_end_pts(struct mp_aframe *f);
double mp_aframe_duration(struct mp_aframe *f);
void mp_aframe_clip_timestamps(struct mp_aframe *f, double start, double end);
bool mp_aframe_copy_samples(struct mp_aframe *dst, int dst_offset,
struct mp_aframe *src, int src_offset,
int samples);
bool mp_aframe_set_silence(struct mp_aframe *f, int offset, int samples);
struct mp_aframe_pool;
struct mp_aframe_pool *mp_aframe_pool_create(void *ta_parent);
int mp_aframe_pool_allocate(struct mp_aframe_pool *pool, struct mp_aframe *frame,
int samples);