RepoMirrors
/
mpv
mirror of https://github.com/mpv-player/mpv
1
0
Fork 0
Code Issues Releases Wiki Activity
mpv/sub/dec_sub.h

59 lines
1.5 KiB
C
Raw Normal View History

subs: libass: use a single persistent renderer for subtitles To draw libass subtitles, the code used ASS_Renderer objects created in vf_vo (VO rendering) or vf_ass. They were destroyed and recreated together with the video filter chain. Change the code to use a single persistent renderer instance stored in the main osd_state struct. Because libass seems to misbehave if fonts are changed while a renderer exists (even if ass_set_fonts() is called on the renderer afterwards), the renderer is recreated after adding embedded fonts. The known benefits are simpler code and avoiding delays when switching between timeline parts from different files (libass fontconfig initialization, needed when creating a new renderer, can take a long time in some cases; switching between files rebuilds the video filter chain, and this required recreating the renderers). On the other hand, I'm not sure whether this could cause inefficient bitmap caching in libass; explicitly resetting the renderer in some cases could be beneficial. The new code does not keep the distinction of separate renderers for vsfilter munged aspect vs normal; this means that changing subtitle tracks can lose cache for the previous track. The new code always sets some libass parameters on each rendering call, which were previously only set if they had potentially changed. This should be harmless as libass itself has checks to see if the values differ from previous ones. Conflicts: command.c libmpcodecs/vf_ass.c libmpcodecs/vf_vo.c mplayer.c sub/ass_mp.c
2012-08-25 14:47:50 +00:00
#ifndef MPLAYER_DEC_SUB_H
#define MPLAYER_DEC_SUB_H
sub: create sub_bitmap array even when using libass One sub_bitmaps struct could contain either a libass ASS_Image list, or a mplayer native list of sub-bitmaps. This caused code duplication in vo_vdpau.c and bitmap_packer.c. Avoid this by creating such a sub_bitmap array even with libass. This basically copies the list and recreates it in mplayer's native format. It gets rid of the code duplication, and will make implementing extended subtitle and OSD rendering in other VOs easier. Also do some cosmetic changes and other preparations for the following commits.
2012-09-28 19:19:36 +00:00
#include <stdbool.h>
#include <stdint.h>
Rename sub.c/.h to osd.c/.h This was way too misleading. osd.c merely calls the subtitle renderers, instead of actually dealing with subtitles.
2013-11-24 11:58:06 +00:00
#include "osd.h"
sub: cosmetics: move things around Move sub-bitmap definitions from dec_sub.h to sub.h. While it's a bit odd that OSD data structures are in a file named sub.h, it's definitely way too strange to have them in a file about subtitle decoding. (Maybe sub.h/.c and the sub/ directory should be split out and renamed "osd" at a later point.) Remove including ass_mp.h (and the libass headers) where possible. Remove typedefs for mp_eosd_res and sub_bitmaps structs. Store a mp_eosd_res struct in osd_state instead of just w/h. Note that sbtitles might be rendered using different sizes/margins when filters are involved (the subtitle renderer is not supposed to use the OSD res directly, and the "dim" member removed in the previous commit is something different).
2012-10-04 15:16:40 +00:00
player: rearrange how subtitle context and stream headers are used Use sh_stream over sh_sub. Use dec_sub (and mpctx->d_sub) instead of the stream header. This aligns the subtitle code with the recent audio and video refactoring. sh_sub still has the decoder context, though. This is because we want to avoid reinit when switching segments with ordered chapters. (Reinit is fast, except for creating the ASS_Renderer, which in turn triggers fontconfig.) Not sure how much this matters, though, because the initial segment switch will lazily initialize the decoder anyway.
2013-11-23 20:37:15 +00:00
struct sh_stream;
sub/osd: mp_msg conversions
2013-12-21 18:06:37 +00:00
struct mpv_global;
sub: refactor Make the sub decoder stuff independent from sh_sub (except for initialization of course). Sub decoders now access a struct sd only, instead of getting access to sh_sub. The glue code in dec_sub.c is similarily independent from osd. Some simplifications are made. For example, the switch_id stuff is unneeded: the frontend code just has to make sure to call osd_changed() any time subtitles are switched. This is also preparation for introducing subtitle converters. It's much cleaner to completely separate demuxer header/renderer glue/decoders for this purpose, especially since sub converters might completely change how demuxer headers have to be interpreted. Also pass data as demux_packets. Currently, this doesn't help much, but libavcodec converters might need scary stuff like packet side data, so it's perhaps better to go with passing packets.
2013-06-01 17:44:12 +00:00
struct demux_packet;
player: add experimental stream recording feature This is basically a WIP, but it can't remain in a branch forever. A warning is print when using it as it's still a bit "shaky".
2017-02-07 16:05:17 +00:00
struct mp_recorder_sink;
sub: always go through sub.c for OSD rendering Before this commit, vf_vo.c and vf_ass.c were manually calling the subtitle decoder to retrieve images to render. In particular, this circumvented the sub-bitmap conversion & caching layer in sub.c. Change this so that subtitle decoding isn't special anymore, and draws all subtitles with the normal OSD drawing API. This is also a step towards removing the need for vf_ass auto-insertion. In fact, if auto-insertion would be disabled now, VOs with "old" OSD rendering could still render ASS subtitles in monochrome, because there is still ASS -> old-OSD bitmap conversion in the sub.c mechanism. The code is written with the assumption that the subtitle rendering filter (vf_ass) can render all subtitle formats. Since vf_ass knows the ASS format only, rendering image subs (i.e. RGBA subs) with it simply fails. This means that with vo_xv (vf_ass auto-inserted), image subs wouldn't be rendered. Use a dumb hack to disable rendering subs with a filter, if we detect that the subs are not in ASS format. (Trying to render the subs first would probably result in purging the conversion cache on every frame.)
2012-10-04 15:16:32 +00:00
sub: refactor Make the sub decoder stuff independent from sh_sub (except for initialization of course). Sub decoders now access a struct sd only, instead of getting access to sh_sub. The glue code in dec_sub.c is similarily independent from osd. Some simplifications are made. For example, the switch_id stuff is unneeded: the frontend code just has to make sure to call osd_changed() any time subtitles are switched. This is also preparation for introducing subtitle converters. It's much cleaner to completely separate demuxer header/renderer glue/decoders for this purpose, especially since sub converters might completely change how demuxer headers have to be interpreted. Also pass data as demux_packets. Currently, this doesn't help much, but libavcodec converters might need scary stuff like packet side data, so it's perhaps better to go with passing packets.
2013-06-01 17:44:12 +00:00
struct dec_sub;
struct sd;
dec_sub: introduce sub_control(), use it for sub_step This means the direct libass usage can be removed from command.c, and no weird hacks for retrieving the ASS_Track are needed. Also fix a bug when using this feature with ordered chapters.
2013-06-28 23:34:11 +00:00
enum sd_ctrl {
SD_CTRL_SUB_STEP,
sd_add: add terrible hack for (xy-)vsfilter compatibility Much has been said about this topic, we don't need to say even more. See additions to options.rst.
2013-07-14 23:48:25 +00:00
SD_CTRL_SET_VIDEO_PARAMS,
player: remove OSD subtitle render path This was used with --no-sub-ass (aka --no-ass). This option (which is not yet removed) strips all styling from the subtitles, and renders them as plaintext only. For some reason, it originally seemed convenient to reuse all the OSD text rendering code (osd_libass.c). While this was indeed simple, it had a bad influence on the rest of the code. For example, it had to decide whether to go through the OSD code path, or the proper subtitle renderer in sd_ass.c. Kill the OSD subtitle renderer. Reimplement --no-sub-ass and also "secondary" subtitles in sd_ass.c. fill_plaintext() contains some rather minor code duplication with osd_libass.c for setting up a dummy ASS_Event and escaping the stripped text. Since sd_ass.c already has to handle "normal" text subtitles, and has code for stripping ASS tags, this remains all relatively simple. Remove all the unnecessary crap from the rest of the code.
2015-11-17 00:54:02 +00:00
SD_CTRL_SET_TOP,
sub: minor refactor how video FPS for MicroDVD is set So that the video FPs is not required at initialization, and can be set later. (As for whether this MicroDVD crap is worth the trouble to handle it "correctly": MicroDVD files are unfortunately still around, and in at least one case using the video FPS seemed to help indeed.)
2015-12-27 00:25:32 +00:00
SD_CTRL_SET_VIDEO_DEF_FPS,
sub: make filter_sdh a "proper" filter, allow runtime changes Until now, filter_sdh was simply a function that was called by sd_ass directly (if enabled). I want to add another filter, so it's time to turn this into a somewhat more general subtitle filtering infrastructure. I pondered whether to reuse the audio/video filtering stuff - but better not. Also, since subtitles are horrible and tend to refuse proper abstraction, it's still messed into sd_ass, instead of working on the dec_sub.c level. Actually mpv used to have subtitle "filters" and even made subtitle converters part of it, but it was fairly horrible, so don't do that again. In addition, make runtime changes possible. Since this was supposed to be a quick hack, I just decided to put all subtitle filter options into a separate option group (=> simpler change notification), to manually push the change through the playloop (like it was sort of before for OSD options), and to recreate the sub filter chain completely in every change. Should be good enough. One strangeness is that due to prefetching and such, most subtitle packets (or those some time ahead) are actually done filtering when we change, so the user still needs to manually seek to actually refresh everything. And since subtitle data is usually cached in ASS_Track (for other terrible but user-friendly reasons), we also must clear the subtitle data, but of course only on seek, since otherwise all subtitles would just disappear. What a fucking mess, but such is life. We could trigger a "refresh seek" to make this more automatic, but I don't feel like it currently. This is slightly inefficient (lots of allocations and copying), but I decided that it doesn't matter. Could matter slightly for crazy ASS subtitles that render with thousands of events. Not very well tested. Still seems to work, but I didn't have many test cases.
2020-02-16 00:02:17 +00:00
SD_CTRL_UPDATE_OPTS,
dec_sub: introduce sub_control(), use it for sub_step This means the direct libass usage can be removed from command.c, and no weird hacks for retrieving the ASS_Track are needed. Also fix a bug when using this feature with ordered chapters.
2013-06-28 23:34:11 +00:00
};
command: add property to return text subtitles in ASS See manpage additions. This was requested, sort of. Although what has been requested might be something completely different. So this is speculative. This also changes sub_get_text() to return an allocated copy, because the buffer shit was too damn messy.
2020-05-14 20:14:49 +00:00
enum sd_text_type {
SD_TEXT_TYPE_PLAIN,
SD_TEXT_TYPE_ASS,
};
command: add sub-start & sub-end properties These properties contain the current subtitle's start and end times. Can be useful to cut sample audio through the scripting interface.
2019-09-21 18:11:18 +00:00
struct sd_times {
double start;
double end;
};
sub: pass all attachments to the subtitle decoder Commit 8d4a179c made subtitle decoders pick up fonts strictly from the same source file (i.e. the same demuxer). It breaks some fucked up use-case, and 2 people on this earth complained about the change because of this. Add it back. This copies all attached fonts on each subtitle init. I considered converting attachments to use refcounting, but it'd probably be much more complex. Since it's slightly harder to get a list of active demuxers with duplicate removed, the prev_demuxer variable serves as a hack to achieve almost the same thing, except in weird corner cases. (In which fonts could be added twice.)
2016-03-03 17:48:56 +00:00
struct attachment_list {
struct demux_attachment *entries;
int num_entries;
};
struct dec_sub *sub_create(struct mpv_global *global, struct sh_stream *sh,
struct attachment_list *attachments);
sub: refactor Make the sub decoder stuff independent from sh_sub (except for initialization of course). Sub decoders now access a struct sd only, instead of getting access to sh_sub. The glue code in dec_sub.c is similarily independent from osd. Some simplifications are made. For example, the switch_id stuff is unneeded: the frontend code just has to make sure to call osd_changed() any time subtitles are switched. This is also preparation for introducing subtitle converters. It's much cleaner to completely separate demuxer header/renderer glue/decoders for this purpose, especially since sub converters might completely change how demuxer headers have to be interpreted. Also pass data as demux_packets. Currently, this doesn't help much, but libavcodec converters might need scary stuff like packet side data, so it's perhaps better to go with passing packets.
2013-06-01 17:44:12 +00:00
void sub_destroy(struct dec_sub *sub);
sub: make preloading more robust Subtitles can be preloaded, which means they're fully read and copied into ASS_Track. This in turn is mainly for the sake of being able to do subtitle seeking (when it comes down to it, subtitle seeking is the cause for most trouble here). Commit a714f8e92 broke preloaded subtitles which have events with unknown duration, such as some MicroDVD samples. The event list gets cleared on every seek, so the property of being preloaded obviously gets lost. Fix this by moving most of the preloading logic to dec_sub.c. If the subtitle list gets cleared, they are not considered preloaded anymore, and the logic for demuxed subtitles is used. As another minor thing, preloadeding subtitles did neither disable the demux stream, nor did it discard packets. Thus you could get queue overflows in theory (harmless, but annoying). Fix this by explicitly discarding packets in preloaded mode. In summary, now the only difference between preloaded and normal demuxing are: 1. a seek is issued, and all packets are read on start 2. during playback, discard the packets instead of feeding them to the subtitle decoder This is still petty annoying. It would be nice if maintaining the subtitle index (and maybe a subtitle packet cache for instant subtitle presentation when seeking back) could be maintained in the demuxer instead. Half of all file formats with interleaved subtitles have this anyway (mp4, mkv muxed with newer mkvmerge).
2016-03-06 13:50:36 +00:00
bool sub_can_preload(struct dec_sub *sub);
void sub_preload(struct dec_sub *sub);
sub: change how subtitles are read Slightly change how it is decided when a new packet should be read. Switch to demux_read_packet_async(), and let the player "wait properly" until required subtitle packets arrive, instead of blocking everything. Move distinguishing the cases of passive and active reading into the demuxer, where it belongs.
2015-12-29 00:35:52 +00:00
bool sub_read_packets(struct dec_sub *sub, double video_pts);
video: make OSD/subtitle bitmaps refcounted (sort of) Making OSD/subtitle bitmaps refcounted was planend a longer time ago, e.g. the sub_bitmaps.packed field (which refcounts the subtitle bitmap data) was added in 2016. But nothing benefited much from it, because struct sub_bitmaps was usually stack allocated, and there was this weird callback stuff through osd_draw(). Make it possible to get actually refcounted subtitle bitmaps on the OSD API level. For this, we just copy all subtitle data other than the bitmaps with sub_bitmaps_copy(). At first, I had planned some fancy refcount shit, but when that was a big mess and hard to debug and just boiled to emulating malloc(), I made it a full allocation+copy. This affects mostly the parts array. With crazy ASS subtitles, this parts array can get pretty big (thousands of elements or more), in which case the extra alloc/copy could become performance relevant. But then again this is just pure bullshit, and I see no need to care. In practice, this extra work most likely gets drowned out by libass murdering a single core (while mpv is waiting for it) anyway. So fuck it. I just wanted this so draw_bmp.c requires only a single call to render everything. VOs also can benefit from this, because the weird callback shit isn't necessary anymore (simpler code), but I haven't done anything about it yet. In general I'd hope this will work towards simplifying the OSD layer, which is prerequisite for making actual further improvements. I haven't tested some cases such as the "overlay-add" command. Maybe it crashes now? Who knows, who cares. In addition, it might be worthwhile to reduce the code duplication between all the things that output subtitle bitmaps (with repacking, image allocation, etc.), but that's orthogonal.
2020-04-26 21:34:32 +00:00
struct sub_bitmaps *sub_get_bitmaps(struct dec_sub *sub, struct mp_osd_res dim,
int format, double pts);
command: add property to return text subtitles in ASS See manpage additions. This was requested, sort of. Although what has been requested might be something completely different. So this is speculative. This also changes sub_get_text() to return an allocated copy, because the buffer shit was too damn messy.
2020-05-14 20:14:49 +00:00
char *sub_get_text(struct dec_sub *sub, double pts, enum sd_text_type type);
command: add sub-start & sub-end properties These properties contain the current subtitle's start and end times. Can be useful to cut sample audio through the scripting interface.
2019-09-21 18:11:18 +00:00
struct sd_times sub_get_times(struct dec_sub *sub, double pts);
sub: refactor Make the sub decoder stuff independent from sh_sub (except for initialization of course). Sub decoders now access a struct sd only, instead of getting access to sh_sub. The glue code in dec_sub.c is similarily independent from osd. Some simplifications are made. For example, the switch_id stuff is unneeded: the frontend code just has to make sure to call osd_changed() any time subtitles are switched. This is also preparation for introducing subtitle converters. It's much cleaner to completely separate demuxer header/renderer glue/decoders for this purpose, especially since sub converters might completely change how demuxer headers have to be interpreted. Also pass data as demux_packets. Currently, this doesn't help much, but libavcodec converters might need scary stuff like packet side data, so it's perhaps better to go with passing packets.
2013-06-01 17:44:12 +00:00
void sub_reset(struct dec_sub *sub);
sub: destroy/recreate ASS_Renderer when disabling/enablings subs Keeping ASS_Renderers around for a potentially large number of subtitle tracks could lead to excessive memory usage, especially since the libass cache is broken (caches even unneeded data), and might consume up to ~500MB of memory for no reason.
2015-12-26 17:35:36 +00:00
void sub_select(struct dec_sub *sub, bool selected);
player: add experimental stream recording feature This is basically a WIP, but it can't remain in a branch forever. A warning is print when using it as it's still a bit "shaky".
2017-02-07 16:05:17 +00:00
void sub_set_recorder_sink(struct dec_sub *sub, struct mp_recorder_sink *sink);
Implement backwards playback See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
2019-05-18 00:10:51 +00:00
void sub_set_play_dir(struct dec_sub *sub, int dir);
sub: refactor Make the sub decoder stuff independent from sh_sub (except for initialization of course). Sub decoders now access a struct sd only, instead of getting access to sh_sub. The glue code in dec_sub.c is similarily independent from osd. Some simplifications are made. For example, the switch_id stuff is unneeded: the frontend code just has to make sure to call osd_changed() any time subtitles are switched. This is also preparation for introducing subtitle converters. It's much cleaner to completely separate demuxer header/renderer glue/decoders for this purpose, especially since sub converters might completely change how demuxer headers have to be interpreted. Also pass data as demux_packets. Currently, this doesn't help much, but libavcodec converters might need scary stuff like packet side data, so it's perhaps better to go with passing packets.
2013-06-01 17:44:12 +00:00
dec_sub: introduce sub_control(), use it for sub_step This means the direct libass usage can be removed from command.c, and no weird hacks for retrieving the ASS_Track are needed. Also fix a bug when using this feature with ordered chapters.
2013-06-28 23:34:11 +00:00
int sub_control(struct dec_sub *sub, enum sd_ctrl cmd, void *arg);
subs: restore support for sub_step command with libass This was destroyed by Uoti Urpala in commit "subs: always use sub...". Features should be either kept working or completely removed, but not just crippled, which only inflates the code and frustrates users.
2012-09-01 17:49:04 +00:00
subs: libass: use a single persistent renderer for subtitles To draw libass subtitles, the code used ASS_Renderer objects created in vf_vo (VO rendering) or vf_ass. They were destroyed and recreated together with the video filter chain. Change the code to use a single persistent renderer instance stored in the main osd_state struct. Because libass seems to misbehave if fonts are changed while a renderer exists (even if ass_set_fonts() is called on the renderer afterwards), the renderer is recreated after adding embedded fonts. The known benefits are simpler code and avoiding delays when switching between timeline parts from different files (libass fontconfig initialization, needed when creating a new renderer, can take a long time in some cases; switching between files rebuilds the video filter chain, and this required recreating the renderers). On the other hand, I'm not sure whether this could cause inefficient bitmap caching in libass; explicitly resetting the renderer in some cases could be beneficial. The new code does not keep the distinction of separate renderers for vsfilter munged aspect vs normal; this means that changing subtitle tracks can lose cache for the previous track. The new code always sets some libass parameters on each rendering call, which were previously only set if they had potentially changed. This should be harmless as libass itself has checks to see if the values differ from previous ones. Conflicts: command.c libmpcodecs/vf_ass.c libmpcodecs/vf_vo.c mplayer.c sub/ass_mp.c
2012-08-25 14:47:50 +00:00
#endif