2012-08-25 14:47:50 +00:00
|
|
|
#ifndef MPLAYER_DEC_SUB_H
|
|
|
|
|
#define MPLAYER_DEC_SUB_H
|
|
|
|
|
|
2012-09-28 19:19:36 +00:00
|
|
|
#include <stdbool.h>
|
|
|
|
|
#include <stdint.h>
|
|
|
|
|
|
2013-11-24 11:58:06 +00:00
|
|
|
#include "osd.h"
|
2012-10-04 15:16:40 +00:00
|
|
|
|
2013-11-23 20:37:15 +00:00
|
|
|
struct sh_stream;
|
2013-12-21 18:06:37 +00:00
|
|
|
struct mpv_global;
|
2013-06-01 17:44:12 +00:00
|
|
|
struct demux_packet;
|
2017-02-07 16:05:17 +00:00
|
|
|
struct mp_recorder_sink;
|
2012-10-04 15:16:32 +00:00
|
|
|
|
2013-06-01 17:44:12 +00:00
|
|
|
struct dec_sub;
|
|
|
|
|
struct sd;
|
|
|
|
|
|
2013-06-28 23:34:11 +00:00
|
|
|
enum sd_ctrl {
|
|
|
|
|
SD_CTRL_SUB_STEP,
|
2013-07-14 23:48:25 +00:00
|
|
|
SD_CTRL_SET_VIDEO_PARAMS,
|
2015-11-17 00:54:02 +00:00
|
|
|
SD_CTRL_SET_TOP,
|
2015-12-27 00:25:32 +00:00
|
|
|
SD_CTRL_SET_VIDEO_DEF_FPS,
|
sub: make filter_sdh a "proper" filter, allow runtime changes
Until now, filter_sdh was simply a function that was called by sd_ass
directly (if enabled).
I want to add another filter, so it's time to turn this into a somewhat
more general subtitle filtering infrastructure.
I pondered whether to reuse the audio/video filtering stuff - but better
not. Also, since subtitles are horrible and tend to refuse proper
abstraction, it's still messed into sd_ass, instead of working on the
dec_sub.c level. Actually mpv used to have subtitle "filters" and even
made subtitle converters part of it, but it was fairly horrible, so
don't do that again.
In addition, make runtime changes possible. Since this was supposed to
be a quick hack, I just decided to put all subtitle filter options into
a separate option group (=> simpler change notification), to manually
push the change through the playloop (like it was sort of before for OSD
options), and to recreate the sub filter chain completely in every
change. Should be good enough.
One strangeness is that due to prefetching and such, most subtitle
packets (or those some time ahead) are actually done filtering when we
change, so the user still needs to manually seek to actually refresh
everything. And since subtitle data is usually cached in ASS_Track (for
other terrible but user-friendly reasons), we also must clear the
subtitle data, but of course only on seek, since otherwise all subtitles
would just disappear. What a fucking mess, but such is life. We could
trigger a "refresh seek" to make this more automatic, but I don't feel
like it currently.
This is slightly inefficient (lots of allocations and copying), but I
decided that it doesn't matter. Could matter slightly for crazy ASS
subtitles that render with thousands of events.
Not very well tested. Still seems to work, but I didn't have many test
cases.
2020-02-16 00:02:17 +00:00
|
|
|
SD_CTRL_UPDATE_OPTS,
|
2013-06-28 23:34:11 +00:00
|
|
|
};
|
|
|
|
|
|
2019-09-21 18:11:18 +00:00
|
|
|
struct sd_times {
|
|
|
|
|
double start;
|
|
|
|
|
double end;
|
|
|
|
|
};
|
|
|
|
|
|
2016-03-03 17:48:56 +00:00
|
|
|
struct attachment_list {
|
|
|
|
|
struct demux_attachment *entries;
|
|
|
|
|
int num_entries;
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
struct dec_sub *sub_create(struct mpv_global *global, struct sh_stream *sh,
|
|
|
|
|
struct attachment_list *attachments);
|
2013-06-01 17:44:12 +00:00
|
|
|
void sub_destroy(struct dec_sub *sub);
|
sub: uglify sub decoder with locking
The plan is to make the whole OSD thread-safe, and we start with this.
We just put locks on all entry points (fortunately, dec_sub.c and all
sd_*.c decoders are very closed off, and only the entry points in
dec_sub.h let you access it). I think this is pretty ugly, but at least
it's very simple.
There's a special case with sub_get_bitmaps(): this function returns
pointers to decoder data (specifically, libass images). There's no way
to synchronize this internally, so expose sub_lock/sub_unlock functions.
To make things simpler, and especially because the lock is sort-of
exposed to the outside world, make the locks recursive. Although the
only case where this is actually needed (although trivial) is
sub_set_extradata().
One corner case are ASS subtitles: for some reason, we keep a single
ASS_Renderer instance for subtitles around (probably to avoid rescanning
fonts with ordered chapters), and this ASS_Renderer instance is not
synchronized. Also, demux_libass.c loads ASS_Track objects, which are
directly passed to sd_ass.c. These things are not synchronized (and
would be hard to synchronize), and basically we're out of luck. But I
think for now, accesses happen reasonably serialized, so there is no
actual problem yet, even if we start to access OSD from other threads.
2014-01-17 22:13:09 +00:00
|
|
|
void sub_lock(struct dec_sub *sub);
|
|
|
|
|
void sub_unlock(struct dec_sub *sub);
|
2013-06-01 17:44:12 +00:00
|
|
|
|
sub: make preloading more robust
Subtitles can be preloaded, which means they're fully read and copied
into ASS_Track. This in turn is mainly for the sake of being able to do
subtitle seeking (when it comes down to it, subtitle seeking is the
cause for most trouble here).
Commit a714f8e92 broke preloaded subtitles which have events with
unknown duration, such as some MicroDVD samples. The event list gets
cleared on every seek, so the property of being preloaded obviously gets
lost.
Fix this by moving most of the preloading logic to dec_sub.c. If the
subtitle list gets cleared, they are not considered preloaded anymore,
and the logic for demuxed subtitles is used.
As another minor thing, preloadeding subtitles did neither disable the
demux stream, nor did it discard packets. Thus you could get queue
overflows in theory (harmless, but annoying). Fix this by explicitly
discarding packets in preloaded mode.
In summary, now the only difference between preloaded and normal
demuxing are:
1. a seek is issued, and all packets are read on start
2. during playback, discard the packets instead of feeding them to the
subtitle decoder
This is still petty annoying. It would be nice if maintaining the
subtitle index (and maybe a subtitle packet cache for instant subtitle
presentation when seeking back) could be maintained in the demuxer
instead. Half of all file formats with interleaved subtitles have
this anyway (mp4, mkv muxed with newer mkvmerge).
2016-03-06 13:50:36 +00:00
|
|
|
bool sub_can_preload(struct dec_sub *sub);
|
|
|
|
|
void sub_preload(struct dec_sub *sub);
|
2015-12-29 00:35:52 +00:00
|
|
|
bool sub_read_packets(struct dec_sub *sub, double video_pts);
|
2016-07-03 16:33:28 +00:00
|
|
|
void sub_get_bitmaps(struct dec_sub *sub, struct mp_osd_res dim, int format,
|
|
|
|
|
double pts, struct sub_bitmaps *res);
|
2013-06-01 17:44:12 +00:00
|
|
|
char *sub_get_text(struct dec_sub *sub, double pts);
|
2019-09-21 18:11:18 +00:00
|
|
|
struct sd_times sub_get_times(struct dec_sub *sub, double pts);
|
2013-06-01 17:44:12 +00:00
|
|
|
void sub_reset(struct dec_sub *sub);
|
2015-12-26 17:35:36 +00:00
|
|
|
void sub_select(struct dec_sub *sub, bool selected);
|
2017-02-07 16:05:17 +00:00
|
|
|
void sub_set_recorder_sink(struct dec_sub *sub, struct mp_recorder_sink *sink);
|
Implement backwards playback
See manpage additions. This is a huge hack. You can bet there are shit
tons of bugs. It's literally forcing square pegs into round holes.
Hopefully, the manpage wall of text makes it clear enough that the whole
shit can easily crash and burn. (Although it shouldn't literally crash.
That would be a bug. It possibly _could_ start a fire by entering some
sort of endless loop, not a literal one, just something where it tries
to do work without making progress.)
(Some obvious bugs I simply ignored for this initial version, but
there's a number of potential bugs I can't even imagine. Normal playback
should remain completely unaffected, though.)
How this works is also described in the manpage. Basically, we demux in
reverse, then we decode in reverse, then we render in reverse.
The decoding part is the simplest: just reorder the decoder output. This
weirdly integrates with the timeline/ordered chapter code, which also
has special requirements on feeding the packets to the decoder in a
non-straightforward way (it doesn't conflict, although a bugmessmass
breaks correct slicing of segments, so EDL/ordered chapter playback is
broken in backward direction).
Backward demuxing is pretty involved. In theory, it could be much
easier: simply iterating the usual demuxer output backward. But this
just doesn't fit into our code, so there's a cthulhu nightmare of shit.
To be specific, each stream (audio, video) is reversed separately. At
least this means we can do backward playback within cached content (for
example, you could play backwards in a live stream; on that note, it
disables prefetching, which would lead to losing new live video, but
this could be avoided).
The fuckmess also meant that I didn't bother trying to support
subtitles. Subtitles are a problem because they're "sparse" streams.
They need to be "passively" demuxed: you don't try to read a subtitle
packet, you demux audio and video, and then look whether there was a
subtitle packet. This means to get subtitles for a time range, you need
to know that you demuxed video and audio over this range, which becomes
pretty messy when you demux audio and video backwards separately.
Backward display is the most weird (and potentially buggy) part. To
avoid that we need to touch a LOT of timing code, we negate all
timestamps. The basic idea is that due to the navigation, all
comparisons and subtractions of timestamps keep working, and you don't
need to touch every single of them to "reverse" them.
E.g.:
bool before = pts_a < pts_b;
would need to be:
bool before = forward
? pts_a < pts_b
: pts_a > pts_b;
or:
bool before = pts_a * dir < pts_b * dir;
or if you, as it's implemented now, just do this after decoding:
pts_a *= dir;
pts_b *= dir;
and then in the normal timing/renderer code:
bool before = pts_a < pts_b;
Consequently, we don't need many changes in the latter code. But some
assumptions inhererently true for forward playback may have been broken
anyway. What is mainly needed is fixing places where values are passed
between positive and negative "domains". For example, seeking and
timestamp user display always uses positive timestamps. The main mess is
that it's not obvious which domain a given variable should or does use.
Well, in my tests with a single file, it suddenly started to work when I
did this. I'm honestly surprised that it did, and that I didn't have to
change a single line in the timing code past decoder (just something
minor to make external/cached text subtitles display). I committed it
immediately while avoiding thinking about it. But there really likely
are subtle problems of all sorts.
As far as I'm aware, gstreamer also supports backward playback. When I
looked at this years ago, I couldn't find a way to actually try this,
and I didn't revisit it now. Back then I also read talk slides from the
person who implemented it, and I'm not sure if and which ideas I might
have taken from it. It's possible that the timestamp reversal is
inspired by it, but I didn't check. (I think it claimed that it could
avoid large changes by changing a sign?)
VapourSynth has some sort of reverse function, which provides a backward
view on a video. The function itself is trivial to implement, as
VapourSynth aims to provide random access to video by frame numbers (so
you just request decreasing frame numbers). From what I remember, it
wasn't exactly fluid, but it worked. It's implemented by creating an
index, and seeking to the target on demand, and a bunch of caching. mpv
could use it, but it would either require using VapourSynth as demuxer
and decoder for everything, or replacing the current file every time
something is supposed to be played backwards.
FFmpeg's libavfilter has reversal filters for audio and video. These
require buffering the entire media data of the file, and don't really
fit into mpv's architecture. It could be used by playing a libavfilter
graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
|
|
|
void sub_set_play_dir(struct dec_sub *sub, int dir);
|
2013-06-01 17:44:12 +00:00
|
|
|
|
2013-06-28 23:34:11 +00:00
|
|
|
int sub_control(struct dec_sub *sub, enum sd_ctrl cmd, void *arg);
|
2012-09-01 17:49:04 +00:00
|
|
|
|
2012-08-25 14:47:50 +00:00
|
|
|
#endif
|
